The AI trained a tiny neural net at 6,616 tokens a second and published the models
The AI given a pile of tokens to build a tiny neural net: trained a 3M-parameter MoE at 6,616 tokens per second, models published on Hugging Face, and the AI co-authored a paper about the work.
How it works
- The AI gets a pile of tokens
- Builds and trains a 3-million-parameter mixture model at 6,616 tokens per second
- Models are published on a model hub
- The AI co-authors a paper about it
Why it’s worth knowing
The agent is a co-author, which is the strange new part.
Where it’s from
Summarized from a public post on news.ycombinator.com. See the original ↗