Small models aren't finished yet
Recent open-recipe models have made big gains at small scale. Each one pushes on a single axis: Puro-2B cuts training cost through infrastructure work, MiniCPM5 improves data efficiency, and LFM2.5-2.6B tunes architecture for fast on-device inference. How well these models perform suggests the quality of small models is far from saturated.
What we're building
We're bringing these insights together in one study and developing architecture and training algorithms side by side. Our first target is a 2 billion parameter general language model that beats existing models in its class on three fronts:
- PerformanceStronger results on standard benchmarks
- Inference speedFaster to run, including on-device
- Training costLess compute to reach the same quality