Google Bets on New Chips to Boost AI Results, Challenging Nvidia

In a matter of months, Google’s AI chips have become one of the hottest commodities in the tech sector. Leading artificial intelligence developers, including some of the firm’s biggest rivals, are stocking up on them.

Now, the Alphabet Inc.-owned company aims to build on its momentum with the likely introduction of new chips dedicated to inference, or running AI models after they’ve been trained. With this push, Google is poised to further challenge market leader Nvidia Corp. in a fast-growing category for semiconductors that’s fueled by surging adoption of AI software.

As demand grows for quickly processing AI queries, “it now becomes sensible to specialize chips more for training or more for inference workloads,” Google Chief Scientist Jeff Dean said in an interview. “We are looking at a whole bunch of different things,” he added, including the speed of AI results it wants to enable.

The company plans to announce its new generation of custom-designed chips, known as tensor processing units, or TPUs, at the Google Cloud Next conference in Las Vegas this week. Amin Vahdat, who oversees Google’s AI infrastructure and chip work, declined to comment on plans for an inference chip that can speed up AI outputs, but said more will likely be shared “in the relatively near future.”Bloomberg News AI Infrastructure Reporter Dina Bass joins Bloomberg Businessweek Daily to discuss. She speaks with Carol Massar and Tim Stenovec. Nvidia’s graphics processing units, or GPUs, remain the gold standard for AI, particularly for training more advanced models. But a growing number of up-and-comers are vying to take on the chipmaker for inference uses, including by offering chips meant to cut down response times for chatbots and AI agents. Last month, Nvidia began selling a chip intended for faster inference based on technology it acquired from Groq as part of a reported $20 billion licensing deal.

Google brings unique strengths to that competitive landscape, including a decade of experience designing chips, vast resources from its online search profits and firsthand insights on AI models. Among the top AI developers, only Google makes its own chips at significant scale, allowing it to share vital feedback between teams to better customize hardware. (OpenAI is only now starting to design its own.)

In a recent podcast interview, Nvidia’s Jensen Huang stressed the advantages of his company’s chips, saying they can do “a whole bunch of applications” that “you can’t do with TPUs.” Google, for its part, relies on a mix of TPUs and GPUs for its own work. “A lot of people would like to run on both,” Demis Hassabis, chief executive officer of Google DeepMind, told Bloomberg. Interest in TPUs is particularly high from leading AI labs, he said.

Google has previously touted inference capabilities for its chips. It also considered releasing separate chips for training and inference early on, according to Partha Ranganathan, a vice president and engineering fellow at Google, but so far it’s resisted that approach. That might change soon as the AI spending boom moves from training to inference.

“The battleground is shifting towards inference,” said Chirag Dekate, an analyst at Gartner, who notes that in his experience Google’s Gemini model is the fastest at responding to complex reasoning tasks. “In that battleground, Google has an infrastructure advantage.”

--------

Watch Bloomberg Radio LIVE on YouTube

Weekdays 7am-6pm ET

Follow us on X

Subscribe to our Podcasts:

Bloomberg Daybreak

Bloomberg Surveillance

Bloomberg Intelligence

Balance of Power

Bloomberg Businessweek

Listen on Apple CarPlay and Android Auto with the Bloomberg Business app:

Apple CarPlay

Android Auto

Visit our YouTube channels:

Bloomberg Podcasts

Bloomberg Television

Bloomberg Originals