Google Developing Custom AI Chips with Marvell Technology for Data Centers

Google is partnering with Marvell Technology to develop two new specialized AI chips for its data centers, planning to deploy nearly two million chips as it challenges Nvidia's market dominance in the fast-growing semiconductor category. The company will announce its new generation of tensor processing units (TPUs) at the Google Cloud Next conference in Las Vegas this week, including chips dedicated to inference workloads. Google has secured major customers including Anthropic, which accessed up to one million TPUs, and Meta Platforms, which signed a multibillion-dollar deal for TPU access over several years. With a decade of chip design experience, Google is the only major AI developer making its own chips at significant scale, though it faces supply constraints similar to Nvidia and must balance allocation between its own AI services and external customers.

Google's Strategic Push into AI Inference Chips

• ▪Google plans to announce its new generation of tensor processing units (TPUs) at the Google Cloud Next conference in Las Vegas this week
• ▪Google Chief Scientist Jeff Dean said it now becomes sensible to specialize chips more for training or more for inference workloads as demand grows for quickly processing AI queries
• ▪Gartner analyst Chirag Dekate said the battleground is shifting towards inference in AI spending

Growing Customer Demand and Major TPU Partnerships

• ▪Citadel Securities plans to present at the Google conference about how TPUs let the company train models faster than previous work with GPUs
• ▪Meta Platforms Inc. just received access to its first significant supply of TPUs and is testing them out to see what tasks they're best suited for
• ▪Anthropic PBC unveiled an expanded agreement in October to access as many as 1 million TPUs from Google
• ▪G42, the Abu Dhabi technology conglomerate, has held multiple discussions with Google about using TPUs
• ▪Anthropic signed a deal with Broadcom Inc., Google's TPU partner, for chips that will enable Anthropic to tap into about 3.5 gigawatts of computing power starting in 2027
• ▪Meta Platforms Inc. signed a multibillion-dollar deal to use TPUs through Google Cloud over several years

Evolution of TPU Development and Technical Challenges

• ▪Google debuted the more advanced Gemini 3 model in November, trained and run on TPUs
• ▪A seminal 2017 Google research paper that gave rise to today's large language models pushed the TPU team to focus on chips for training bigger AI systems
• ▪Google DeepMind and the chips team noticed that TPUs were sitting unused too often when deployed for reinforcement learning
• ▪Google Chief Scientist Jeff Dean started building an earlier AI software system to let people use language translation and voice recognition services
• ▪Nvidia began selling a chip intended for faster inference last month based on technology Nvidia acquired from Groq as part of a reported $20 billion licensing deal
• ▪Google now has to spot manufacturing flaws with hundreds of thousands of accelerator chips within 10 seconds
• ▪Jeff Dean realized there was no way that even Google could afford to deliver AI services using available chips and hardware

Balancing Internal Needs with External Customer Supply

• ▪Google's Amin Vahdat said making TPUs only for Google would eventually result in ending up on a tech island that would be limited in population and diversity
• ▪Google's Amin Vahdat said there are benefits to making TPUs only for Google, but there are substantial downsides
• ▪Google DeepMind CEO Demis Hassabis said Google is favoring what supply it has to the more elite teams who could take the most advantage of what TPUs do best

Perspective of Google TPU team

• ▪Google's Amin Vahdat believes making TPUs exclusively for Google would result in ending up on a limited tech island

Perspective of Industry analysts

• ▪Gartner analyst Chirag Dekate said the battleground is shifting towards inference in AI spending