Artificial Intelligences, So Far

I wrote this short memo last November, 2024, at the invitation of Wired Mid-East for their year-end issue. I think it still holds up nine months later, and represents where we are on this astounding journey.

There are three points I find helpful when thinking about AIs so far:

The first is that we have to talk about AIs, plural. There is no monolithic singular AI that runs the world. Instead there are already multiple varieties of AI, and each of them have multiple models with their own traits, quirks, and strengths. For instance there are multiple LLM models, trained on slightly different texts, which yield different answers to queries. Then there are non-LLM AI models – like the ones driving cars – that have very different uses besides answering questions. As we continue to develop more advanced models of AI, they will have even more varieties of cognition inside them. Our own brains in fact are a society of different kinds of cognition – such as memory, deduction, pattern recognition – only some of which have been artificially synthesized. Eventually, commercial AIs will be complicated systems consisting of dozens of different types of artificial intelligence modes, and each of them will exhibit its own personality, and be useful for certain chores. Besides these dominant consumer models there will be hundreds of other species of AI, engineered for very specialized tasks, like driving a car, or diagnosing medical issues. We don’t have a monolithic approach to regulating, financing, or overseeing machines. There is no Machine. Rather we manage our machines differently, dealing with airplanes, toasters, x-rays, iphones, rockets with different programs appropriate for each machine. Ditto for AIs.

And none of these species of AI – not one – will think like a human. All of them produce alien intelligences. Even as they approach consciousness, they will be alien, almost as if they are artificial alien beings. They think in a different way, and might come up with solutions a human would never do. The fact that they don’t think like humans is their chief benefit. There are wicked problems in science and business that may require us to first invent a type of AI that, together with humans, can solve problems humans alone cannot solve. In this way AIs can go beyond humans, just like whale intelligence is beyond humans. Intelligence is not a ladder, with steps along one dimension; it is multidimensional, a radiation. The space of possible intelligences is very large, even vast, with human intelligence occupying a tiny spot at the edge of this galaxy of possible minds. Every other possible mind is alien, and we have begun the very long process of populating this space with thousands of other species of possible minds.

The second thing to keep in mind about AIs is that their ability to answer questions is probably the least important thing about them. Getting answers is how we will use them at first, but their real power is in something we call spatial intelligence – their ability to simulate, render, generate, and manage the 3D world. It is a genuine superpower to be able to reason intellectually and to think abstractly – which some AIs are beginning to do – but far more powerful is the ability to act in reality, to get things done and make things happen in the physical world. Most meaningful tasks we want done require multiple steps, and multiple kinds of intelligences to complete. To oversee the multiple modes of action, and different modes of thinking, we have invented agents. An AI agent needs to master common sense to navigate through the real world, to be able to anticipate what will actually happen. It has to know that there is cause and effect, and that things don’t disappear just because you can’t see them, or that two objects can not occupy the same place at the same time, and so on. AIs have to be able to understand a volumetric world in three dimensions. Something similar is needed for augmented reality. The AIs have to be able to render a virtual world digitally to overlay the real world using smart glasses, so that we see both the actual world and a perfect digital twin. To render that merged world in real time as we move around wearing our glasses, the system needs massive amounts of cheap ubiquitous spatial intelligence. Without ubiquitous spatial AI, there is no metaverse.

We have the first glimpses of spatial intelligence in the AIs that can generate video clips from a text prompt or from found images. In laboratories we have the first examples of AIs that can generate volumetric 3D worlds from video input. We are almost at the point that one person can produce a 3D virtual world. Creating a video game or movie now becomes a solo job, one that required thousands of people before.

Just as LLMs were trained on billions of pieces of text and language, some of these new AIs are being trained on billions of data points in physics and chemistry. For instance, the billion hours of video from Tesla cars driving around are training AIs on not just the laws of traffic, but the laws of physics, how moving objects behave. As these spatial models improve, they also learn how forces can cascade, and what is needed to accomplish real tasks. Any kind of humanoid robot will need this kind of spatial intelligence to survive more than a few hours. So in addition to training AI models to get far better at abstract reasoning in the intellectual realm, the frontier AI models are rapidly progressing at improving their spatial intelligence, which will have far more use and far more consequence than answering questions.

The third thing to keep in mind about AIs is that you are not late. You have time; we have time. While the frontier of AI seems to be accelerating fast, adoption is slow. Despite hundreds of billions of dollars invested into AI in the last few years, only the chip maker Nvidia and the data centers are making real profits. Some AI companies have nice revenues, but they are not pricing their service for real costs. It is far more expensive to answer a question with an LLM than the AIs that Google has used for years. As we ask the AIs to do more complicated tasks, the cost will not be free. Most people will certainly pay for most of their AIs, while free versions will be available. This slows adoption.

In addition, organizations can’t simply import AIs as if they were just hiring additional people. Work flows and even the shape of the organizations need to change to fit AIs. Something similar happened as organizations electrified a century ago. One could not introduce electric motors, telegrams, lights, telephones, into a company without changing the architecture of the space as well as the design of the hierarchy. Motors and telephones produced skyscraper offices and corporations. To bring AIs into companies will demand a similar redesign of roles and spaces. We know that AI has penetrated smaller companies first because they are far more agile in morphing their shape. As we introduce AIs into our private lives, this too will necessitate redesign of many of our habits, and all this takes time. Even if there was not a single further advance in AI today, it will take 5 to 10 years to fully incorporate the AIs we already have into our orgs and lives.

There’s a lot of hype about AI these days, and among those who hype AI the most are the doomers – because they promote the most extreme fantasy version of AI. They believe the hype. A lot of the urgency for dealing with AI comes from the doomers who claim 1) that the intelligence of AI can escalate instantly, and 2) we should regulate on harms we can imagine rather than harms that are real. Despite what the doomers proclaim, we have time because there has been no exponential increase in artificial intelligence. The increase in intelligence has been very slow, in part because we don’t have good measurements for human intelligence, and no metrics for extra-human intelligence. But the primary slow rate is due to the fact that the only exponential in AI is in its input – it takes exponentially more training data, and exponentially more compute to make just a modest improvement in reasoning. The artificial intelligences are not compounding anywhere near exponential. We have time.

Lastly, our concern about the rise of AIs should be in proportion to its actual harm vs actual benefits. So far as I have been able to determine, the total number of people who have lost their jobs to AI as of 2024, is just several hundred employees, out of billions. They were mostly language translators and a few (but not all) help-desk operators. This will change in the future, but if we are evidence based, the data so far is that the real harms of AI are almost nil, while the imagined harms are astronomical. If we base our policies for AIs on the reasonable fact that they are varied and heterogenous, and their benefits are more than answering questions, and that so far we have no evidence of massive job displacement, then we have time to accommodate their unprecedented power into our society.

The scientists who invented the current crop of LLMs were trying to make language translation software. They were completely surprised that bits of reasoning also emerged from the translation algorithms. This emergent intelligence was a beautiful unintended byproduct that also scaled up magically. We honestly have no idea what intelligence is, so as we make more of it and more varieties of it, there will inevitably be more surprises like this. But based on the evidence of what we have made so far, this is what we know.