In October 2026, Hugging Face released Carbon-X, an open 14.8 billion parameter DNA language model developed with Zhongguancun Academy and the University of Naples Federico II. Operating under an Apache 2.0 license, Carbon-X integrates DNA sequences and natural language into a single stream, averaging 66.7% across 5 benchmarks while reducing inference costs compared to proprietary models. Carbon-X remains unvalidated by wet-lab experiments and cannot ingest full human genomes.
Carbon-X model architecture
- ▪Carbon-X incorporates a hybrid tokenizer that adds 5,515 DNA tokens, including 4,096 non-overlapping 6-mers, 1,364 tail tokens, and 3 markers, to Qwen3's vocabulary
- ▪Carbon-X features a 40-layer standard Qwen3 backbone with a hidden size of 5,120, grouped-query attention with 8 KV heads, and a 40,960-token context length
- ▪Hugging Face, Zhongguancun Academy, and TIGEM / University of Naples Federico II released Carbon-X, an open 14.8 billion parameter decoder-only DNA language model built from Qwen3-14B
- ▪Carbon-X processes raw DNA sequences and natural language within a single token stream in one decoder without requiring a separate DNA encoder
Four-stage training process
- ▪Carbon-X underwent an embedding warmup training stage using approximately 10 billion tokens with the transformer body frozen to train only new DNA embeddings
- ▪Carbon-X mid-training used 302 billion tokens at an 8,192 sequence length comprising 55% raw DNA, 20% English, 15% biology text, and 10% instruction data
- ▪Hugging Face excluded viral DNA sequences from Carbon-X's training data to prevent potential misuse in biological harm applications
- ▪Carbon-X completed supervised fine-tuning across 3 epochs over 20 dataset subsets, followed by two rounds of asynchronous GRPO reinforcement learning
DNA benchmark performance
- ▪Carbon-X achieved a self-reported 66.7% average score across 5 DNA benchmarks, placing 0.8 percentage points behind GPT-6.1 Sol and 27 percentage points above Qwen3-14B
- ▪Carbon-X achieved individual benchmark scores of 85.1% on BioReason Non-SNV, 63.6% on GenomeQA, 70.4% on FrontierVEP, and 39.4% on NextBase
- ▪Carbon-X retained general natural language performance by achieving 79.3% prompt-level strict accuracy on the IFEval benchmark
- ▪Carbon-X scored 74.9% on the BioReason Coding benchmark, outperforming GPT-6.1 Sol at 71.0% and Opus 5.5 at 70.3%
Local deployment options
- ▪Carbon-X offers official GGUF quantized models including a 15.8 GB Q8_0 build and a 9.6 GB Q4_K_M build requiring llama.cpp build b11742 or newer
- ▪Carbon-X releases its model weights under an Apache 2.0 license, permitting biological research laboratories to run the model locally on 1 x 80 GB GPUs
Cost efficiency analysis
- ▪Carbon-X achieves an estimated inference cost per problem equal to approximately 1/1,800 of GPT-6.1 Sol's cost when measured on H200 hardware
- ▪Kimi K3 improved its DNA index evaluation score from 49.4% to 60.5% when consulting Carbon-X as a sub-agent
- ▪Hugging Face estimates that scoring 2.4 million ClinVar variants of uncertain significance using Carbon-X costs approximately $300 on an H200 GPU
Research applications limitations
- ▪Carbon-X is limited to input DNA sequences of roughly 50,000 base pairs, preventing users from querying or uploading entire human genomes
- ▪Carbon-X recorded lower accuracy on specific DNA tasks, scoring 29.8% on GenomeQA splice-site questions and 34.6% on histone mark evaluations
- ▪Carbon-X developers caution that the model has not been validated through wet-lab experiments or clinical testing to verify prediction accuracy
Story comments
Loading comments…