Explore setting up an "Olmo" LLM model ("32B-Instruct") Fedora playground #24

New issue

Closed

opened 2025-12-18 15:59:51 +00:00 by kashyapc · 2 comments

kashyapc commented

2025-12-18 15:59:51 +00:00

What?

Olmo 3 is a "truly transparent" LLM model by the non-profit, Allen Institute for AI. In my limited testing, it is still very rough and currently lags behind other semi-open models in terms of performance.

The reason why I suggest Olmo is that it opens up the "the entire model flow – not just its endpoint". Meaning, "the full lifecycle of a Language Model: every stage, checkpoint, dataset, and dependency required to create and modify it" (from their announcement on Olmo 3.)

Why?

"Olmo" family of models are the closest LLM models that align with Fedora's spirit in terms of transparency and openness.

Which model to choose?

They have 32B-Base, 32B-Think, and 32B-Instruct models. Once Fedora has sufficient AI hardware, we should go for the most capable one, 32B-Instruct.

References:

Artifacts for the Olmo 3 release: https://huggingface.co/collections/allenai/olmo-3
They recently released a model that is "multimodal": https://allenai.org/molmo
Their public "playground" to try out several models (let's be gentle with it): https://playground.allenai.org/

### What? [Olmo 3](https://allenai.org/olmo) is a "truly transparent" LLM model by the non-profit, [Allen Institute for AI](https://allenai.org/about). In my limited testing, it is still very rough and currently lags behind other semi-open models in terms of performance. The reason why I suggest Olmo is that it opens up the _"the entire model flow – not just its endpoint"_. Meaning, _"the full lifecycle of a Language Model: every stage, checkpoint, dataset, and dependency required to create and modify it"_ (from their [announcement on Olmo 3](https://allenai.org/blog/olmo3).) ### Why? "Olmo" family of models are the closest LLM models that align with Fedora's spirit in terms of transparency and openness. ### Which model to choose? They have `32B-Base`, `32B-Think`, and `32B-Instruct` models. Once Fedora has sufficient AI hardware, we should go for the most capable one, `32B-Instruct`. References: - Artifacts for the Olmo 3 release: https://huggingface.co/collections/allenai/olmo-3 - They recently released a model that is "multimodal": https://allenai.org/molmo - Their public "playground" to try out several models (let's be gentle with it): https://playground.allenai.org/

👀 1

jflory7 added the

scope

new

label

2026-07-03 02:56:58 +00:00

jflory7 commented

2026-09-14 23:38:00 +00:00

Owner

This issue came up in a conversation I was in recently. It made me curious what the wider Fedora community thinks. I created a new Fedora Discussion topic to solicit input and opinions.

This issue came up in a conversation I was in recently. It made me curious what the wider Fedora community thinks. I [created a new Fedora Discussion topic](https://discussion.fedoraproject.org/t/which-llm-gen-ai-model-is-more-aligned-with-fedora-ai2-olmo-nvidia-nemotron-both-or-something-else/202060) to solicit input and opinions.

jflory7 commented

2026-10-14 18:26:02 +00:00

Owner

Discussed in 2026-10-14 FRCL meeting.

During the October 14, 2026 FRCL sync, participants evaluated Allen Institute's OLMo as part of our strategic analysis on AI model alignment. On the plus side, OLMo represents the gold standard for software and data transparency under the Four Freedoms, providing completely public training datasets, open modeling code, and permissive licensing that allows full auditability. However, OLMo currently exhibits significant drawbacks: its capability, reasoning quality, and task effectiveness lag noticeably behind leading open-weight alternatives, and its high compute and energy overhead poses real ecological and operational cost concerns. Additionally, OLMo lacks turnkey availability across upstream inference providers, creating contributor tooling hurdles unless Fedora operates its own dedicated hosting. Because deploying such a model effectively depends on shared project infrastructure, this ticket is being closed until the community is positioned to drive toward centralized contributor inference services.

Follow-up Items:

Status: Closed as deferred.
Trigger to Reopen: Revisit OLMo and fully open source AI models when Fedora evaluates or deploys shared contributor inference infrastructure, or when upstream open models achieve competitive capability parity with standard tooling.
Reference: For the full context and meeting synthesis, see the Fedora Discussion update.

_Discussed in [2026-10-14 FRCL meeting](https://discussion.fedoraproject.org/t/which-llm-gen-ai-model-is-more-aligned-with-fedora/202060/15)._ --- During the October 14, 2026 FRCL sync, participants evaluated Allen Institute's OLMo as part of our strategic analysis on AI model alignment. On the plus side, OLMo represents the gold standard for software and data transparency under the Four Freedoms, providing completely public training datasets, open modeling code, and permissive licensing that allows full auditability. However, OLMo currently exhibits significant drawbacks: its capability, reasoning quality, and task effectiveness lag noticeably behind leading open-weight alternatives, and its high compute and energy overhead poses real ecological and operational cost concerns. Additionally, OLMo lacks turnkey availability across upstream inference providers, creating contributor tooling hurdles unless Fedora operates its own dedicated hosting. Because deploying such a model effectively depends on shared project infrastructure, this ticket is being closed until the community is positioned to drive toward centralized contributor inference services. **Follow-up Items:** - **Status:** Closed as deferred. - **Trigger to Reopen:** Revisit OLMo and fully open source AI models when Fedora evaluates or deploys shared contributor inference infrastructure, or when upstream open models achieve competitive capability parity with standard tooling. - **Reference:** For the full context and meeting synthesis, see the [Fedora Discussion update](https://discussion.fedoraproject.org/t/which-llm-gen-ai-model-is-more-aligned-with-fedora/202060/15).

jflory7 closed this issue

2026-10-14 18:26:02 +00:00

jflory7 added this to the 2026 Q4 Goals milestone

2026-10-14 18:26:11 +00:00

jflory7 added the

huge

state

wontfix

labels

2026-10-14 18:26:25 +00:00