The landscape of Open Language Models (OLMos) and the advancements they bring to AI research and deployment are worth exploring. AI2 has recently released OLMos, including OLMo, a truly open large language model framework. This framework comes with pre-training data and training code, empowering researchers and developers to collectively advance the science of language models.
One of the notable OLMos is OLMo 7B, which outperforms Llama 2 on generative tasks and reading comprehension. However, it lags behind on question-answering tasks. AI2’s commitment to transparency and open research is evident in the availability of code, models, data, and evaluation code necessary for AI advancement.
The benefits of OLMo include per-token training efficiency, open training data, and licenses that enable usage for training, experimentation, and even commercialization. Additionally, OLMos provide opportunities for thought experiments that explore the landscape of language models in 2024.
Although OLMos have their limitations, such as low-quality outputs in non-English languages and weak code-generating capabilities, the release of these models by AI2 is a positive step towards open AI models. AI2 plans to expand the capabilities by releasing larger models in the future, collaborating with partners like Harvard, AMD, and Databricks.
With OLMos, the AI community can leverage open models, foster collaboration, and drive advancements in the field. The availability of OLMo and its accompanying resources sets a new standard for open and transparent language models and paves the way for future breakthroughs in AI research and deployment.

Collections

Open Language Models (OLMos) are advancing AI research and deployment. Notable OLMos include OLMo 7B, which outperforms Llama 2 on generative tasks. OLMo offers per-token training efficiency and open training data. AI2 plans to collaborate with partners like Harvard, AMD, and Databricks.

OLMo: The Truly Open Language Model Framework