From Infrastructure to Intelligence - Navigating the AI Wave

In November 2022, ChatGPT was launched. After the initial awe, startup founders, venture capitalists, and incumbents scrambled to understand where new value from this technological wave would accrue.

The quick consensus in the venture community was that startups building on top of OpenAI and other Large Language Model (LLM) providers were only viable as short-term cash grabs and would be quickly made obsolete. In contrast, model creators, such as Anthropic or Mistral, rapidly raised massive venture capital rounds to get ahead or at least catch up with OpenAI’s GPT suite of models.

In late 2023, a new twist in this technological wave emerged: agents. Both startups and incumbents had realized that the real value in LLMs was not just exposing a generic model in small parts of your application. Creating systems around LLMs that can deeply integrate into existing workflows for a customer would be a real differentiator. These systems not only assist with current job functions but also replace humans in some of them.

The base LLM provider market, although still rapidly evolving, appears challenging. The big tech incumbents like Google and Microsoft create ever more sophisticated models that can compete with and at times surpass those of OpenAI, Anthropic, and Mistral. Meta has even launched an open-source family of models, Llama, that are on track to becoming a state-of-the-art (SOTA) model that can be freely deployed and fine-tuned.

So where should founders put their time and investors their money? It turns out that looking backward and previous technological waves can give us a hint at how this mad maelstrom might settle.

Cloud Computing: Fueling the Digital Economy

We don’t have to look far back to see how a tech wave evolves. In 2006, Amazon launched AWS with its first service, S3.¹ Suddenly, instead of having to buy, set up, and maintain expensive servers, other companies could rent server space online from Amazon. As we will see with other examples later on, this had wide-ranging implications for both the providers and users of the cloud:

1. Amazon had uncovered a cash machine with AWS. It could use its existing server infrastructure to bill customers based on usage. It was easy to start using AWS and your bill grew with the success of your customers.
2. The cloud opened the floodgates for online businesses of all types: now that the cost and time it took to start an internet business were reduced by orders of magnitude, it acted as high-octane fuel for fledgling industries such as Software as a Service (SaaS).

What happened to the cloud market? AWS was the sole leader in this market until 2010 when Microsoft Azure launched, followed a year later by Google Cloud.² Even in 2024, AWS remains the market leader with 31% market share, though chased by Azure at 25%. Other providers with meaningful market share are Google Cloud, Alibaba Cloud, Salesforce, IBM Cloud, Oracle, and Tencent.³

What does this list of major cloud providers have in common? They are all incumbents that had existing large businesses outside of the cloud before entering. Why did none of the new entrants break through?

One obvious explanation is that the infrastructure for hosting a cloud requires high upfront investment. Most of these incumbents already had the necessary infrastructure to host their own web services and kickstart a cloud service. They could also carry the financial investment to extend this infrastructure. Another reason could have been existing distribution channels. Most of the providers on the list had existing business customer relationships that could be leveraged to market their cloud offering.

This is not to say that no new entrant into the cloud space succeeded. Providers like Digital Ocean, Linode, or Scaleway are large businesses but remain niche providers without significant market share. The cloud providers have also gobbled up other competitors during their growth trajectory that have at times led to great returns.

However, the list of companies built on top of the cloud is endless. Obvious examples are the large SaaS, Ecommerce, and Social Media industries. But a giant industry was created also entirely around hosting, monitoring, and maintaining applications in the cloud. An example is the juggernaut Snowflake, a cloud-based data storage and analytics provider, that is sporting a $52 billion market cap as of May 2024.⁴ Snowflake hosts its services on other cloud providers like AWS. Compare that to the market cap of $3 billion for Digital Ocean.⁵

Duopoly in Your Pocket

Another tech wave in recent years has been mobile. Steve Jobs famously unveiled the iPhone in January 2007.⁶ Again, this technology enabled opportunities for both providers of the phone market as well as companies using it for their own services:

1. Apple did not invent mobile phones or even the concept of an App Store. But it managed to package and market both in a way that drove large adoption. It had learned from previous product lines that value could not only be accrued from selling hardware but also monetizing platforms built on top of the hardware itself, like iTunes and the App Store.
2. The mobile wave enabled incumbents to grow their existing business and completely new entrants to enter. Instagram could exist because now your phone had a usable camera. Uber could exist because both driver and passenger had a way to easily share their location and find each other in real time.

The development of this market rings familiar. In 2008, the first Android-equipped phone was launched by HTC. Google had bought Android in 2005 and positioned it as an open ecosystem against Apple’s closed iPhone business.⁷ Today the smartphone operating system market is a duopoly of Android and iOS. The market for smartphone hardware has been heavily commoditized with Apple still leading the pack.⁸

Again, the actual technology providers of this wave were largely incumbents that still dominate the market to this day.

Yet, we can find many adopters of this technology that have moved on to become large independent businesses like Uber or TikTok.

Rhyming with the Past: AI’s Next Moves

So what can we learn from these examples for AI?

The technology that kicked off this revolution had again its origin in one of the large incumbents. Google published a paper “Attention is all you need” in 2017 that described a neural network architecture called Transformers.⁹ This time, however, Google could not leverage their technological lead right away and OpenAI used Transformers to put the large tech companies on the back foot when it launched ChatGPT in 2022.

Yet, we can see similar patterns from previous tech waves emerge. Google has almost caught up with OpenAI and is now providing its own powerful Gemini models to use for other companies. Microsoft, which forged a partnership with OpenAI, hosts its models through Azure. Mistral and Anthropic have also made partnerships with major cloud providers to give access to their own model fleet.

The big elephant in the room is, of course, Meta. With no cloud business of their own but massive GPU resources that Mark Zuckerberg wisely acquired ahead of the AI boom, Meta is setting out to build the de facto SOTA open-source model that can be deployed anywhere. In practice, this means that existing cloud providers will soon all have access to a model that can be integrated into their existing cloud offerings.

I predict that the market for LLM and mixed models will become increasingly commoditized over the next 3-5 years. The big tech companies like Microsoft, Meta, Google, and potentially Apple will dominate this market and gobble up other LLM providers like Anthropic or Mistral. Offering LLM models can be a great adjacent business that scales with usage, but similarly to cloud or mobile, requires large upfront investments and favors incumbents.

Does that mean that companies building LLMs today won’t be successful or their investors won’t see great returns? Absolutely not. These businesses are vital and in the case of OpenAI were the reason that this wave kicked off in the first place. But the real big companies will be built elsewhere, and OpenAI knows it.

Sam Altman has made clear what direction he wants to take OpenAI, as Aaron Levie explained in a recent podcast appearance. Sam envisions ChatGPT as your general assistant for anything. You can ask it to book your next flight or analyze that Excel sheet. It can read, hear, see, and do anything for you. This is no doubt in part driven by Sam’s obsession with the movie Her. Mostly Altman has realized though that the real value in LLMs lies in making existing personal and professional workflows frictionless. This combination of LLMs and existing technology for building web and mobile applications is the emergent wave of agents.

If OpenAI succeeds with its lofty goals of building your one general agent and the surrounding ecosystem remains to be seen. Companies like Apple or Google are in an excellent position to integrate LLM capabilities into their existing ecosystems. I predict that hardware makers like Rabbit or Human will disappear unceremoniously when Siri, Alexa, and Google Assistant update to use the latest LLM models. AirPods and an iPhone are really all you need for most tasks that your assistant might be able to fulfill.

There will be many other verticals that AI will transform over time and where big new players will emerge. Customer and market insights will be one of the first because of the large amount of unstructured data. This transformation is happening right now and we at Arro are one of the first to have built solutions with LLMs in the customer research space. Law and software development is equally ripe for disruption. Just as with mobile or cloud services, we cannot yet anticipate all the industries and use cases AI will transform.

What does that mean for startup founders? You don’t need a team of machine learning PhDs. Nor do you need a $30 million seed round to make your mark. The fundamentals are still the same: solve a large problem for a customer with high willingness to pay. And let LLMs help you find a better solution to that problem.

For investors, this means thinking beyond the “thin wrapper over OpenAI” argument. There are certainly startups out there that will disappear with upcoming product features from OpenAI or Google. But the most valuable startups will likely leverage existing LLM technology to solve specific workflows in a vertical job role or industry.

This technological wave has been faster and more radical than anything before it. It’s time to build and solve real-world problems with it.