A Reality Check on Intel’s New Chip

Colin’s Note: The mainstream financial media headlines are all screaming about the new Intel Gaudi 3 artificial intelligence (AI) chip…

About how Intel is gunning for Nvidia… going toe-to-toe with the semiconductor company leading the AI hardware race… and that Intel’s “AI game-changer” offers a competitive alternative to Nvidia’s popular H100 chip…

That Intel has challenged Nvidia’s industry dominance.

Yeah, right.

If you take Intel’s benchmarks – and press release – at face value, you might be convinced. After all, that’s exactly what the media has done.

But dive with me into the details, and you’ll see that Intel still has miles to go if it wants to get in step with Nvidia… let alone dream of outperforming it.

It’s all in today’s video. Click below to watch… or read on for the transcript my team and I have edited for flow.

Bleeding Edge subscribers, hopefully, guys are doing well.

If you read the headlines yesterday, you might be pretty impressed. Here are just a few…

Intel Unveils Latest AI Chip as Nvidia Competition Heats Up – CNBC

Intel Launches Gaudi 3 With Impressive Gen AI Performance – Forbes

Intel Brings the Fight to Nvidia – TipRanks

These headlines followed when Intel unveiled its Gaudi 3 AI accelerator chip yesterday. And the financial news media did what they do best…

They simply read a press release from Intel and regurgitated it.

It wouldn’t surprise me if some of these articles and headlines were simply plugged into ChatGPT and cranked out for you, the investor, to read.

The truth is, if you believe Intel’s press release at face value the way the financial news sites did, you’d actually believe Intel might have a legitimate competitor to Nvidia’s dominance. But to borrow a phrase coined by a former president, that would be fake news.

Intel claims their Gaudi 3 chip delivers an average of 50% better inference capabilities and 40% better power efficiency than Nvidia’s H100 chip… at a fraction of the cost.

Now, that last part is believable. Nvidia has been printing impressive 80% gross margins ever since the A100 and H100 AI computer chips became hot commodities in the tech world. But is the Gaudi 3 really a bleeding-edge AI computer chip set to take market share from Nvidia?

Trusting Intel’s benchmarks and press release at face value, you might think that’s the case. But diving into the details shows Intel is still more than a step behind Nvidia in terms of performance.

Intel claimed their new Gaudi 3 chip delivers 50% faster time to train on average across Llama2 models with 7 billion and 13 billion parameters. Llama2 is one of the most popular open-source large language models (LLMs). It was developed and trained by Meta, the parent company behind Facebook and Instagram.

However, the fact that Intel singled out the 7 billion and 13 billion-parameter models for training – and not the larger 70 billion-parameter Llama model – tells you everything you need to know.

The fact that Gaudi 3 can cruise through the smaller 7 billion and 13 billion-parameter models better than the Nvidia chip isn’t all that surprising. Similar to how a Honda would perform equally as well as a Ferrari driving around the neighborhood streets at 35 miles per hour, it’s not a surprise Intel could meet or exceed Nvidia’s performance.

The fact that Intel didn’t tout the chip’s performance on the larger parameter for the compute-intensive training task is the same reason Honda doesn’t claim it can outperform a Ferrari on a freeway or a racetrack.

Another aspect of these benchmarks that the financial media just failed to single out is the fact that the Llama2 model is about to become obsolete.

Meta announced today that the company’s latest Llama 3 LLM is expected to be released in the next few weeks. So by the time the Gaudi 3 chip comes to market, most AI companies will be training on a new model by then.

But the idea Intel is ready to bring the fight to Nvidia is even more laughable when you consider Intel’s comparisons with Nvidia’s H100 GPU.

Last month, Nvidia announced the successor to the H100 GPU – the GB200. Nvidia claims the newest GB200 chip is 30 times faster than the H100 for inference and four times faster for training.

Nvidia claims that instead of running LLMs with billions of parameters, the new GB200 chip will be capable of running trillion-parameter models.

The reality is Nvidia has been well out in front of the entire semiconductor industry when it comes to accelerated computing AI data center chips. And it’s not just Intel lagging way behind. If the financial news media had any interest in actually informing you, rather than just regurgitating Intel’s claims, they would tell you this.

But the reality is that Intel’s problems don’t just end with Nvidia. The fact that Intel is at least a step behind Nvidia doesn’t surprise anyone with some knowledge about the semiconductor industry.

The reality is Intel’s financials paint a picture of a semiconductor company in a desperate situation. The company’s data center business – which includes sales of previous-generation Gaudi chips – is down 20% over the past year.

And it’s not just Nvidia eating Intel’s lunch with higher-performance data center GPUs. At one time, Intel was the gold standard for CPU computer chips in PCs and data centers. But for years, Advanced Micro Devices (AMD) has been capturing market share in both of these areas.

But it gets worse for Intel… because further competition is coming. Google’s announcement of its Axion processors didn’t get much attention from the financial press this week. But these Arm-based CPU chips are designed to replace Intel’s X86-based CPU chips in data centers.

Not only is Intel several laps behind Nvidia, but the company is falling behind in its bread-and-butter CPUs business as well.

The bottom line is that over the next couple of weeks, there will be thousands of press releases related to AI. It’s the financial news media’s job to rewrite them with the most clickbait-style and interesting headlines possible.

They’re not really trying to inform you.

But just understand to get a true idea of what is actually happening, you have to dive into the details. Intel is well behind its peers across the board. The company’s best days are now well behind it. That was The Bleeding Edge for today. We’ll be back again later this week. Bye for now.