PrismML Releases 1-Bit Large Language Model

Caltech spinout PrismML released Bonsai 8B, a 1-bit large language model that compresses AI into just 1.15 GB of memory—14 times smaller than full-precision counterparts—while running 8 times faster and using 5 times less energy on edge hardware. Founded by Caltech professor Babak Hassibi, PrismML's architecture represents weights as simple signs {-1, +1} with shared scale factors, achieving an intelligence density score of 1.06/GB compared to 0.10/GB for competing model Qwen3 8B, though Qwen3 still leads on traditional benchmarks. The model runs natively on Apple devices and Nvidia GPUs, with PrismML positioning the technology to enable on-device AI agents, real-time robotics, and secure enterprise systems without cloud dependency.

PrismML's 1-bit architecture and technical innovation

• ▪PrismML defines intelligence density as the negative of the log of the model's average error rate divided by the model size.
• ▪PrismML released a 1-bit large language model named Bonsai 8B.
• ▪PrismML released two additional smaller models: 1-bit Bonsai 4B and 1-bit Bonsai 1.7B.
• ▪In PrismML's Bonsai model family architecture, each weight is represented only by its sign, {−1, +1}, while a shared scale factor is stored for each group of weights.
• ▪PrismML's approach is based on work done by Caltech electrical engineering professor Babak Hassibi and colleagues.

Performance benchmarks and efficiency gains

• ▪Bonsai 8B scores 1.06/GB for intelligence density.
• ▪Bonsai 8B delivers over 10x the intelligence density of its full-precision counterparts.
• ▪Bonsai 8B is 8x faster than its full-precision counterparts on edge hardware.
• ▪Bonsai 8B remains competitive with other models in its parameter-class.
• ▪Qwen3 8B comes out ahead of Bonsai 8B in various benchmarks including MMLU Redux, MuSR, and GSM8K.
• ▪Bonsai 8B is 14x smaller than its full-precision counterparts.

Applications for edge computing and on-device AI

• ▪PrismML's 1-bit architecture aims to move AI out of cloud datacenters.
• ▪PrismML expects Bonsai 8B will improve AI efficiency and viability on mobile devices.
• ▪1-bit Bonsai 8B runs natively on Apple devices including Mac, iPhone, and iPad via MLX.
• ▪Bonsai 8B model weights are available under the Apache 2.0 License.
• ▪1-bit Bonsai 8B runs on Nvidia GPUs via llama.cpp CUDA.

Perspective of PrismML

• ▪PrismML's 1-bit architecture represents a fundamental shift in how AI models can be deployed and accessed.

Perspective of Mobile device users and edge computing advocates

• ▪Smaller model sizes like Bonsai 8B's 1.15 GB footprint make AI accessible on devices with limited storage.

Perspective of Traditional AI model developers

• ▪Extreme quantization to 1-bit may limit model capabilities for complex reasoning tasks.
• ▪Full-precision models like Qwen3 8B maintain superior performance on standard benchmarks compared to 1-bit alternatives.

1 source