‘Explore Olmo 3, the open-source LLM from AI2. Access 7B and 32B models, datasets, code, and training logs to empower your AI research contributions’

DigitalOcean Community's platform is a central hub for developers and sysadmins using DigitalOcean's cloud infrastructure, offering insights into cloud computing, DevOps practices, and open-source technologies. Through tutorials, Q&A, and community forums, DO_Community offers insights into deploying and managing applications on DigitalOcean's cloud platform. Developers can learn about Linux server administration, containerization, and automation tools to build and scale applications in the cloud.

DigitalOcean Community

Olmo 3 is Allen AI's fully open-source large language model available in 7B and 32B parameter versions. The release includes complete access to models, training datasets (Dolma 3 with 9.3 trillion tokens), code, and training logs. The model uses a three-stage training pipeline: pretraining on Dolma 3 Mix, mid-training on Dolma 3 Dolmino for skill enhancement, and long-context extension on Dolma 3 Longmino. Post-training uses the Dolci suite with SFT, DPO, and RLVR techniques. The 32B model employs grouped query attention while the 7B uses multi-head attention. OlmoTrace enables tracing text back to training sources for auditing and contamination detection.

Olmo 3: Fully Open-Source LLM from AI2 (Models, Data, & Code)

<p>This is REALLY cool. OlmoTrace as a feature seems like it solves a lot of problems when it comes to trust in AI and widespread adoption and usability of models in some fields. Being able to have what’s basically a fact check button on what AI is saying seems super valuable.</p>
