Zyphra released ZAYA1-8B on May 7, 2026, a Mixture of Experts reasoning model with 760M active parameters trained entirely on AMD Instinct MI300 hardware. The model uses a novel Markovian RSA test-time compute method and outperforms Claude 4.5 Sonnet on HMMT'25 while closing in on DeepSeek-V3.2 performance. Released under Apache 2.0 license, it sets a new standard for intelligence density in the small language model weight class.
ZAYA1-8B model architecture
- ▪ZAYA1-8B is a reasoning Mixture of Experts model
- ▪ZAYA1-8B uses a novel Markovian RSA test-time compute method
- ▪ZAYA1-8B has only 760M active parameters
Benchmark performance results
- ▪ZAYA1-8B outperforms open-weight models many times its size on math and coding benchmarks
- ▪ZAYA1-8B surpasses Claude 4.5 Sonnet on HMMT'25
- ▪ZAYA1-8B is closing in on DeepSeek-V3.2 in performance
- ▪ZAYA1-8B sets a new standard for intelligence density in the small language model weight class
Open-source release
- ▪ZAYA1-8B was released under Apache 2.0 license
- ▪Zyphra released ZAYA1-8B on May 7, 2026
- ▪ZAYA1-8B was trained end-to-end on AMD Instinct MI300 hardware
Story comments
Loading comments…