Bigger Than Anthropic and OpenAI Combined

What do Bitcoin and AI have in common?

On the surface, maybe nothing. But look closer.

The current progress of AI is following a similar track to Bitcoin from years back. And it’s hinting at a breakthrough.

Let me show you what I mean…

Bitcoin was two years old when it hit an important milestone—100,000 mined blocks. Price was also near a milestone at the time—$1 per coin.

The network was maturing from proof of concept to something legitimate. There were still hurdles, of course. There were still no user-friendly exchanges or ways to easily convert newly minted Bitcoin into dollars.

But importantly…

The number of miners active on the Bitcoin network was starting to grow.

Miners are tasked with an important part of maintaining the security of the Bitcoin network. They validate transactions, help distribute new coins into circulation, and help ensure transactions remain permanent and tamper-proof.

They compete to solve a mathematical puzzle, and whoever is first to solve it gets to mine the next block. And the more miners that exist, the more secure the network.

In the early days, specialized hardware like application-specific integrated circuits (ASICs) for mining didn’t exist. Most miners were using their personal computers.

Based on the amount of hash rate that existed in late 2010, we can estimate the number of miners taking part on the network to be around a few hundred.

But then…

Source: Blockchain.com

What the chart shows is that growth of ASICs for mining eventually caused the total amount of mining power, as measured in hash rate, to go parabolic shortly after.

Price followed.

But even in the late 2010s, miners were more focused on ensuring the network was operating as it should. They were not yet focused on ASIC chip farms that could extract as much Bitcoin as possible from the network.

Miners were still focused on making sure the technology was sound. Extracting value efficiently came later.

One corner of the AI arena is at that stage right now, one that’s based upon individuals providing computing power permissionlessly.

AI’s Inflection Point

Large Language Models (LLMs) really took off on the heels of OpenAI’s first model called GPT-1, which was released in 2018.

It essentially acted as a baseline for all future developments with its 117 million parameters used for pre-training using compute from a few hundred Google Cloud TPUs—a compute number that should seem familiar across Bitcoin miners and soon decentralized AI.

We can see how the parameters used for each model exploded in the years that followed.

The largest jump was when the number of parameters grew from just over one billion, to 175 billion. This was GPT-3, which is the moment where LLMs began to demonstrate zero-shot capabilities. Put another way, AI was able to perform tasks it was never specifically trained to do.

It was a major turning point in creating frontier models. It led to the release of the highly successful ChatGPT application in 2022, which reached one million users in five days of release. That’s a feat that took Facebook 10 months, Airbnb 2.5 years, and Netflix 3.5 years.

These models now exceed one trillion parameters in size and are reaching a point where they are sourcing esoteric data just to help grow the knowledge base.

The takeaway is this: AI had a major inflection point from GPT-2 to GPT-3.

Decentralized AI is reaching a similar inflection point.

Decentralized AI’s Inflection Point

Early on, Bitcoin’s network had a few hundred miners participating in securing the network.

For frontier LLMs, we saw models pretrained using a few billion in parameters.

Shortly after, both suddenly exploded in growth. Valuations for projects attached to the technologies went parabolic. And as I alluded to, something similar appears to be happening with decentralized AI.

As a refresher, decentralized AI is where many computers across the globe train a model. It might sound simple on the surface, but it’s a major technological feat.

The technology must coordinate a run across various computers doing separate tasks and then blend them together into one cohesive result. By comparison, OpenAI had it easy. The entire operation took place in one centralized “walled garden.”

Layer in differing hardware and internet speeds, and the ability for decentralized AI to coordinate only grows in complexity.

It’s a challenge…to say the least.

And that’s why the problems being tackled by decentralized AI in the pretraining arena are so interesting.

The Projects

Researchers from a project called Macrocosmos, which operates within the Bittensor network, recently began pre-training a 16-billion-parameter model. It’s known as “Orion-16B.”

What’s important is how the model is training…

It’s spread across three continents with a mix of graphics cards for gaming (RTX 4090 and 5090). These are chips that many individuals have in their home. And what’s great is Macrocosmos can handle a pool of contributors that might fall offline at any moment.

Orion-16B is using 16 billion parameters. It builds on similar work done by Pluralis Research (Node-0) last year where 303 active participants were contributing GPUs that were similar in capacity. Their successful training was for a 7.5B-parameter model, making the work being done for Orion-16B twice the size, with a similar setup.

And both used a few hundred participants, just like the number of Bitcoin miners and OpenAI’s computing cluster for training. It’s a figure that seems to be hit prior to inflection points.

To say it differently, this is bleeding edge work. And the ramifications are not being understood by the market…

These teams are tapping into a global compute pool that is arguably larger than what OpenAI and Anthropic are using for their centralized runs.

Consider that for a moment…

We are discussing potential infrastructure and a network that would have more compute capacity than the largest AI entities on Earth, combined.

This will create new verticals within the AI space for everyday individuals. We’re talking about anybody renting the network for pretraining runs, the ability to create more niche and specialized models, and possibly models that are more ephemeral or dynamic.

And here’s one more thing to consider…

What an Inflection Point Looks Like

Macrocosmos and Pluralis are building on work done by other teams conducting pre-training using a decentralized setup with a few hundred contributors – just like the number of Bitcoin miners on the network before it hit an inflection point.

These projects have pushed the boundary on hardware requirements to tap into what’s available in everyday homes.

We can see the progress of the two models in the top right corner. Their location in this part of the plot means they are using more retail-friendly hardware and using a larger number of participants.

It’s incredibly impressive.

Bitcoin’s network went from hundreds of miners to thousands in a short span. The value of the underlying asset exploded from there.

For the centralized AI labs, the jump from GPT-2 to GPT-3 was a major leap in capacity. OpenAI’s valuation went vertical with this unlock.

We are nearing similar inflections in the decentralized AI space as more participants are tapped for training and parameter sizes begin to grow.

This is the next growth area for digital assets, one that is reminiscent of two of the most transformational technologies of the last two decades.

Let’s track this trend as it hits an imminent inflection point later this year.

Your Pulse on Crypto,

Ben Lilly