Colin’s Note: Earlier this week, we talked about Intel and its new Gaudi 3 chip… and all the overblown headlines screaming about how Intel was gunning for Nvidia.

The financial news media made it sound like Intel was on the verge of dethroning the semiconductor giant as the king of AI hardware… which is laughable and also misses the bigger story.

Because there is a real threat to Nvidia’s industry dominance…

Other tech companies are nipping at its heels… And there’s a chink in Nvidia’s armor. If Nvidia wants to remain the top dog of the AI hardware industry, it needs to figure out a way to address it… or it could spell trouble down the line.

It’s all in today’s video…

Bleeding Edge subscribers, happy Friday, and welcome back.

On Wednesday, we talked about Intel. The company released its Gaudi 3 AI chip, and the financial news media went overboard, claiming Intel was about to bring the fight to Nvidia.

The truth is, Intel isn’t going to go away. But it’s not going to release a product that has Nvidia worried about its AI dominance coming to an end.

But that’s not to say Nvidia isn’t worried.

Competition is nipping at the AI giant’s heels. And Nvidia’s financial outlook proves it. But it’s not coming from traditional semiconductor companies like Intel or AMD… Instead, it’s coming from companies that aren’t traditional semiconductor designers.

For years, Google has been designing and using its custom silicon chips the company calls tensor processing units – or TPUs. Google’s TPUs have given the company an advantage in running custom applications like Google Maps, its world-class search engine, and serving videos on YouTube.

Instead of using off-the-shelf computer chip designs from Nvidia, AMD, or Intel… Google found that its custom designs not only performed better but also cost less.

Other tech giants are starting to take notice. Back in 2016, Amazon bought a startup computer design firm. Since then, Amazon has launched a custom silicon design that the company and its customers use daily.

Other companies like Microsoft and Apple have also developed custom silicon designs as well, with Apple now famously turning its back on Intel and offering its entire product line powered by its own in-house custom silicon.

But this week, another company made headlines… and it came much sooner than expected. On Wednesday, Meta – the company behind Facebook and Instagram – revealed its next-generation AI accelerator chips.

The MTIA, version two, is expected to power the company’s recommendation algorithms and its ad business. The most impressive aspect of Meta’s new chip is the fact that the new version of the chip came years ahead of schedule.

When Meta announced its plan to create its own line of custom silicon chips back in 2023, the inaugural version wasn’t expected to be in production until 2025. But here we are in early 2024. And not only has Meta released version one of the chip… but now, it has already released a second version of the chip.

So what does this mean for Nvidia? Are the chip giant’s best days behind it?

Not so fast.

Nvidia is still producing the best AI chips in the world. The H100 and even the A100 chips are still the gold standard in the industry. The forthcoming GB200 will enable applications like ChatGPT to reach an even new level of performance.

And despite Meta producing its custom silicon designs, the social media giant announced back in February that it would be adding 350,000 Nvidia H100 GPUs to its server infrastructure this year, bringing its total well past 600,000.

But all this comes at a cost. With each Nvidia H100 GPU costing $40,000 or more, Meta and these other tech companies are spending billions of dollars in revenue and profits over to Nvidia.

When Nvidia reported its earnings back in February, the company said it expects its profit margins to fall. You wouldn’t expect that. This was the first sign, though, that competition was starting to impact Nvidia.

In some cases, product releases from Intel and AMD gave tech giants another option… or at least a bargaining chip when it came to pricing with Nvidia.

But in reality, it’s the custom silicon designs that are coming to market much sooner and faster than expected that are giving tech companies alternatives to Nvidia.

And as I mentioned earlier, custom silicon can pay off in a major way.

Many in the tech community doubted Apple could shift away from using Intel computer chips in its popular Macintosh computers and laptops. But here we are, five years later. Not only has Apple completely ditched Intel for its silicon designs, but other laptop makers are starting to do the same over the next several years.

The same thing is going to play out in the data centers. It almost seems laughable that companies like Google, Microsoft, Amazon, and Meta can turn their backs on Nvidia. But the advancements in custom silicon… the ability to control every aspect of the design… and more importantly, to control costs… will become paramount in the years ahead.

Look, over the past year and a half, AI has been more of a race, sprint, or gold rush. Efficiencies and costs aren’t that important when it’s an all-out war to be first to market with game-changing AI products like ChatGPT.

But soon the AI race shifts from being an all-out sprint to a marathon, which will require a much more methodical approach. Not only that but investors and shareholders are going to expect a return and a normalization in data center spending.

One of the most effective ways to achieve this is by controlling your product and processes with custom silicon designs like Meta, Google, Microsoft, and Amazon.

Similar to how there’s always going to be a need for Intel’s computer chips, Nvidia is always going to be seen as a leader in the AI-accelerated computer chip market. But tech companies will be searching for every competitive advantage they can get in the coming years.

Buying off-the-shelf silicon that your competitors have access to won’t cut it in a few years. That’s why I believe Nvidia’s days of smashing results are over and reality has begun to settle in.

The world of bleeding-edge custom silicon chips is advancing. And it’s the competition that has Nvidia worried… because it will eventually become the company’s greatest challenge yet.

That was The Bleeding Edge for today. I hope you all have a wonderful weekend and we’ll see you again next week.

Regards,

Colin Tedards
Editor, The Bleeding Edge