ByteDance prepares AI model for real-time spatial video generation

ByteDance Ltd. is preparing a real-time spatial video generation model that could launch as soon as October 2026. Personally overseen by founder Zhang Yiming, the model is built on the Seedance system and targets 20 frames per second with 0.05-second latency. By rendering virtual worlds in the cloud, ByteDance aims to lower hardware costs for its Pico headsets, shifting the XR competition away from expensive devices toward cloud-rendered content and competing with Google's Genie.

ByteDance spatial video model

• ▪ByteDance Ltd. is preparing an artificial intelligence model for real-time spatial video generation that could launch as soon as October 2026
• ▪The spatial video model is designed to generate on-demand video at approximately 20 frames per second with a latency of roughly 0.05 seconds
• ▪The spatial video model is built on Seedance, ByteDance's existing video generation system, and is designed to let users create interactive virtual worlds for live streams, short-form dramas, and games

Zhang Yiming direct oversight

• ▪Bloomberg framed Zhang Yiming's involvement alongside researchers like Fei-Fei Li and Yann LeCun, who advocate that models grounded in visual and physical understanding are key to systems acting in the real world
• ▪ByteDance founder Zhang Yiming is personally overseeing the development of the new spatial video model, coordinating efforts across business units and allocating computing capacity

Cloud rendering XR strategy

• ▪ByteDance is pursuing two development routes for world models: a vision-language-action approach for robotics and embodied intelligence, and 3D simulation for entertainment and games
• ▪ByteDance's spatial video model will generate virtual worlds in the cloud rather than on-device, shifting the computational load off the headset to lower hardware costs

World model competition

• ▪ByteDance's world model direction commands an eight-figure renminbi data budget, which sources reported by 36Kr estimate is three to four times what its rivals are spending
• ▪ByteDance set world models as its top AI priority for 2026, aiming to ship at least one model by the end of the year to compete with Google's Genie

ByteDance AI infrastructure spending

• ▪ByteDance secured a $30 billion loan in early September 2026 and has been weighing capital expenditure of up to $70 billion for its artificial intelligence build-out
• ▪ByteDance's Seedance system already underpins its CapCut and Doubao applications, positioning the company as a domestic challenger against Alibaba, DeepSeek, and Moonshot AI

Pico headset integration

• ▪The spatial video model is intended to generate virtual worlds that respond directly to the voices and movements of users wearing Pico headsets, which are produced by ByteDance's extended reality arm
• ▪By utilizing cloud-rendered world models for Pico headsets, ByteDance aims to shift the extended reality competition away from high-end hardware specifications toward cloud capacity and content distribution

Debatable claims

• ▪World models are the superior path to developing embodied artificial intelligence
• ▪Cloud rendering will displace high-end hardware in the extended reality market

2 sources