The Generative Beings
Resources/⚡️Fastest Pre-training Code: LLM in 9 da
GitHubGithub

⚡️Fastest Pre-training Code: LLM in 9 da

Small language models

About

We created an LLM that outperform OpenELM and Phi on MT-Bench, in just 9 days. It's built on the Lightning framework with optimisations from TinyLlama, achieving ultra high throughput (~99.6% GPU util