How the first LLM aggregator got started, some of the weird moments in its early growth, architecture challenges, and where we'll be taking it down the road.

OpenRouter has just raised $40m from a16z and others: https://x.com/xanderatallah/status/1937957937692938292

---
The Genesis of OpenRouter [00:00]
Initial Question [01:16]: The story begins in early 2023 with the founder, Alex Atallah, pondering if the AI inference market would be dominated by a single player. He noticed the emergence of new models beyond OpenAI and a growing desire from developers to understand the nuances of different models, including their moderation policies [01:48].

The Rise of Open Source [02:35]: The video highlights the beginning of the open-source AI race, with early models like Bloom 176B and OPT from Facebook [02:46]. A pivotal moment was the release of Meta's Llama 1 in February, which surprisingly outperformed GPT-3 on many benchmarks [03:28], signaling a shift in the landscape.

The Alpaca Moment [04:38]: A major breakthrough occurred in March 2023 with the distillation of Alpaca. Stanford researchers demonstrated that by fine-tuning Llama 1 with outputs from GPT-3, they could transfer the style and knowledge of a larger model to a smaller one for less than $600. This proved that creating powerful, specialized models no longer required massive budgets [04:58].

From a Chrome Extension to a Marketplace
Window AI [06:43]: Before OpenRouter, Atallah launched Window AI, an open-source Chrome extension that empowered users to select their preferred LLM for any web application. This project laid the groundwork for what was to come.

The Launch of OpenRouter [07:18]: OpenRouter was co-founded with Lewis, the creator of the framework that Window AI was built on. Initially, it was a simple aggregator to collect models in one place.

Growth and Evolution [07:57]: OpenRouter quickly evolved into a marketplace, driven by the proliferation of model providers with varying prices, performance, and features. The platform has seen impressive growth, with a 10-100% month-over-month increase for two years. It now offers a single API for over 400 models from more than 60 providers [08:07].

Marketplace Dynamics [08:57]: The transition to a marketplace was a response to the complexity of the growing AI ecosystem. By aggregating providers, OpenRouter helps developers achieve better uptime for both open-source and closed-source models and provides valuable data on latency and throughput [10:27].

The Future of OpenRouter
Expanding Modalities [17:02]: The future vision for OpenRouter includes incorporating models that can generate images and "transfusion models" that allow for conversations with images.

Smarter Routing [17:51]: The platform plans to implement more sophisticated routing mechanisms, including geographical routing and enterprise-level optimizations for GPU allocation.

Enhanced Discovery [18:07]: To help developers find the best models for their needs, OpenRouter aims to improve prompt observability, introduce more granular model categorization, and continue to offer competitive pricing.

---
About Alex Atallah
Cofounder & CEO of OpenRouter, the first LLM aggregator and distributor. Cofounder of OpenSea, the first NFT marketplace.

Helped grow OpenSea to over $4B in monthly volume from 2017 to 2022.

Founded OpenRouter in early 2023, which processes over 2 trillion tokens weekly across over 400 unique language models, as of May 2025.

Recorded at the AI Engineer World's Fair in San Francisco. Stay up to date on our upcoming events and content by joining our newsletter here: https://www.ai.engineer/newsletter

AI Engineer

OpenRouter founder Alex Atallah shares the journey of building a unified API marketplace for accessing multiple AI language models. Starting in 2023 as an experiment to test whether AI inference would be winner-take-all, OpenRouter evolved into a platform offering 400+ models from 60+ providers with unified pricing and switching capabilities. The talk covers key milestones like the emergence of open-source models (Llama, Alpaca), the development of middleware for AI inference, and technical challenges around latency optimization and stream cancellation. Data shows the market is becoming multi-model rather than dominated by a single provider, with Google Gemini growing from 3% to 35% usage over 12 months.

Fun stories from building OpenRouter and where all this is going - Alex Atallah, OpenRouter