Google Must Double AI Serving Capacity Every 6 Months To Meet Demand

Google's AI infrastructure chief told employees the company must double its AI serving capacity every six months in order to meet demand. In a presentation earlier this month, Amin Vahdat, a vice president at Google Cloud, gave a presentation titled 'AI Infrastructure.' It included a slide on 'AI compute demand' that said: 'Now we must double every 6 months.... the next 1000x in 4-5 years.' CNBC reports: The presentation was delivered a week after Alphabet reported better-than-expected third-quarter results and raised its capital expenditures forecast for the second time this year, to a range of $91 billion to $93 billion, followed by a 'significant increase' in 2026. Hyperscaler peers Microsoft, Amazon and Meta also boosted their capex guidance, and the four companies now expect to collectively spend more than $380 billion this year.

Google's 'job is of course to build this infrastructure but it's not to outspend the competition, necessarily,' Vahdat said. 'We're going to spend a lot,' he said, adding that the real goal is to provide infrastructure that is far 'more reliable, more performant and more scalable than what's available anywhere else.' In addition to infrastructure build-outs, Vahdat said Google bolsters capacity with more efficient models and through its custom silicon. Last week, Google announced the public launch of its seventh generation Tensor Processing Unit called Ironwood, which the company says is nearly 30 times more power efficient than its first Cloud TPU from 2018.

Vahdat said the company has a big advantage with DeepMind, which has research on what AI models can look like in future years. Google needs to 'be able to deliver 1,000 times more capability, compute, storage networking for essentially the same cost and increasingly, the same power, the same energy level,' Vahdat said. 'It won't be easy but through collaboration and co-design, we're going to get there.'

Read more of this story at Slashdot.