Understanding the Intelligence Explosion Leading to the ChatGPT Moment
While for some, today’s obsession with Artificial Intelligence (AI) seems to have happened overnight, you may be surprised to discover that the development of this intelligence explosion has been discussed and anticipated since the 1950s. In this chapter, we will highlight the thinking of five polymaths who anticipated the intelligence explosion and “thinking machines” from as early as 1950. We believe it is quite extraordinary that these luminaries were able to envision the future of machine intelligence so clearly particularly as the first three mathematical geniuses wrote in the 1950s and 1960s.
Alan Turing’s Imitation Game
Alan Turing was clearly a man ahead of his time. At a time when the first general purpose computers had just been built, he asked in his now famous 1950 paper Computing Machinery and Intelligence whether or not machines can think.
He opened the paper proposing a test for that eventuality which he referred to as the Imitation Game:
1. The Imitation Game
I propose to consider the question, “Can machines think?” This should begin with definitions of the meaning of the terms “machine” and “think.” The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous, If the meaning of the words “machine” and “think” are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, “Can machines think?” is to be sought in a statistical survey such as a Gallup poll. But this is absurd. Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words.
The new form of the problem can be described in terms of a game which we call the ‘imitation game.”
Here is a link to his paper:
https://courses.cs.umbc.edu/471/papers/turing.pdf
This imitation game, now known as the Turing Test, continues to be for many the ultimate test as to whether or not artificial intelligence has reached the same level of intelligence as humans. For example, Ray Kurzweil uses the Turing Test as his metric for Artificial General Intelligence. Kurzweil predicts that AGI, as determined by the Turing Test, will not be able to be passed until 2029.
Turing also anticipated in a 1947 lecture to the London Mathematical Society, that “What we want is a machine that can learn from experience,” adding that the “possibility of letting the machine alter its own instructions provides the mechanism for this.”
Source: Alan Turing: A Strong Legacy That Powers Modern AI By Amber Jackson June 07, 2024. https://aimagazine.com/machine-learning/alan- turing-a-strong-legacy-that-powers-modern-ai
Turing went on to predict way back then, the concerns we are seeing today. Turing predicted that once machines learn from experience like humans “it would not take long to outstrip our feeble powers […] At some stage therefore we should have to expect the machines to take control.”
Source: Friday essay: some tech leaders think AI could outsmart us and wipe out humanity. I am not worried. Feb 13, 2025. https://theconversation. com/friday-essay-some-tech-leaders-think-ai-could-outsmart-us-and- wipe-out-humanity-im-a-professor-of-ai-and-im-not-worried-248901
As we will see as we discuss other computer visionaries, these extraordinary polymaths really understood from early on the potential power of these new inventions. Turing predicted many of the breakthroughs that would be realized in AI over the next 75 years as we saw above. The great thinkers that follow used their mathematical understanding of the world to make prescient predictions about the evolution of machine intelligence.
John von Neumann, the Polymath who first coined the term “Singularity”
John von Neumann (1903-1957) was a Hungarian-American mathematician and polymath who worked during World War II on the Los Alamos Project. His later work, after the war, involved game theory, and then computers, which were at that distant time (the 1950s) quaintly referred to as “electronic machines.” He is the first thinker on record to use the term singularity as it applies here.
Von Neumann’s thinking on the singularity comes to us from his longtime friend and collaborator Stanislaw Ulam, who wrote a lengthy tribute to Von Neumann, recalling various conversations, one of which was this recollection:
“One conversation centered on the ever-accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.”
While Von Neumann’s “singularity” remark, as recounted by Ulam, was not a formal theory, but a speculative observation about the accelerating pace of technological progress, it is amazing that he came up with this theory when the newest computer was a UNIVAC. Ulam offers this explanation:
“I remember quite well how, very early in the Los Alamos Project, it became obvious that analytical work alone was often not sufficient to provide even qualitative answers. The numerical work by hand and even the use of desk computing machines would require a prohibitively long time for these problems. This situation seemed to provide the final spur for von Neumann to engage himself energetically in the work on methods of computation utilizing the electronic machines.”
The important point here is that in the decade of the 1950s, Von Neumann perceived that as these machines evolved and their power accelerated, we could reach a point in the future, or an event horizon beyond which humanity would be overwhelmed by machine intelligence.
(Those interested in reading Stanislaw Ulam’s full tribute will find it here.)
I.J. Good’s Essay on the Intelligence Explosion
The term “intelligence explosion” was explicitly coined for the first time by I.J. Good in his 1965 paper, Speculations Concerning the First Ultraintelligent Machine. Good (1916-2009), a British mathematician and cryptologist who worked during the Second World War alongside the brilliant Alan Turing at Bletchley Park, starts his essay with a bang:
“The survival of man depends on the early construction of an ultraintelligent machine.”
Reading through Good’s essay sixty years later provides a fascinating look at how mathematicians and thought leaders in the 1960s tried to come to terms with the early computers. In the second section of his essay, Good shares the core message for which he became famous:
2.Ultraintelligent Machines and Their Value
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man, however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind (see for example refs. [ZZ], [34], [44]). Thus, the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control. It is curious that this point is made so seldom outside of science fiction. It is sometimes worthwhile to take science fiction seriously. –I.J. Good
In subsequent paragraphs, he pushes back on critics who suggest we should focus our efforts on making more humans to obtain greater intelligence:
“B. V. Bowden stated on British television (August 1962) that there is no point in building a machine with the intelligence of a man, since it is easier to construct human brains by the usual method. A similar point was made by a speaker during the meetings reported in a recent. IEEE publication [I], but I do not know whether this point appeared in the published report. This shows that highly intelligent people can overlook the “intelligence explosion.” It is true that it would be uneconomical to build a machine capable only of ordinary intellectual attainments, but it seems fairly probable that if this could be done, then, at double the cost, the machine could exhibit ultraintelligence.“
Good then wonders about the potential value of such a machine and speculates on the opportunities and challenges such a machine might generate for humanity:
“Since we are concerned with the economical construction of an ultraintelligent machine, it is necessary to consider what such a machine would be worth. Carter [II] estimated the value, to the world, of J. M. Keynes, as at least 100,000 million pounds sterling. By definition, an ultraintelligent machine is worth far more, although the sign is uncertain, but since it will give the human race a good chance of surviving indefinitely, it might not be extravagant to put the value at a megakeynes. There is the opposite possibility, that the human race will become redundant, and there are other ethical problems, such as whether a machine could feel pain, especially if it contains chemical artificial neurons, and whether an ultraintelligent machine should be dismantled when it becomes obsolete [43, 841. The machines will create social problems, but they might also be able to solve them in addition to those that have been created by microbes and men. Such machines will be feared and respected, and perhaps even loved. These remarks might appear fanciful to some readers, but to the writer, they seem very real and urgent, and worthy of emphasis outside of science fiction.”
Of course, the important point for those of us alive today is the first point: if we can create an ultra-intelligent machine that can surpass all the intellectual activities of any man, however smart, and if that machine can design even better machines, that would indeed be the last machine anyone need ever build.
This concept is called recursive self-improvement. We entertained a lengthy series of prompts back and forth with xAI on this topic. This is the feedback we received:
“Recursive self-improvement, as envisioned by I.J. Good, refers to an ultraintelligent machine’s ability to iteratively enhance its own intelligence by redesigning its algorithms, hardware, or architecture. This process creates a feedback loop where each improvement enables the machine to make even better improvements, leading to an exponential “intelligence explosion.” Good saw this as a transformative leap, potentially making the ultraintelligent machine “the last invention that man needs ever make,” provided it remains controllable, with profound implications for humanity’s survival and progress.”
We also asked xAI how close we are to realizing Good’s vision (five to ten years away), and to analyze the impact once achieved, and whether Move 37 constitutes recursive self-improvement. Here is that back and forth: Recursive Self-Improvement
Vernor Vinge Describes the Technological Singularity
Many readers of Good’s work might have overlooked the last two lines from the heart of his oft-quoted core statement:
“It is curious that this point is made so seldom outside of science fiction. It is sometimes worthwhile to take science fiction seriously.”
One person who did not overlook that line was Vernor Vinge (1944- 2024), a science fiction writer who taught mathematics and computer science at San Diego State University from 1972 until his retirement in 2000.
Vinge is the futurist thought leader who explained the intelligence explosion leading to Technological Singularity. He began with an article in Omni magazine in January 1983, then followed up ten years later with his now-famous essay, “The Coming Technological Singularity: How to Survive in the Post-Human Era.” This he presented at the NASA Lewis Research Center, VISION-21 Symposium on March 30–31, 1993. It was published in the symposium proceedings and in Whole Earth Review (Winter 1993).
Vernor Vinge’s January 1983 Essay in Omni
In his January 1983 essay, Vinge first wonders if dramatic advances in technology have led other great civilizations to self-destruct, anticipating the Fermi Paradox and the Great Filter Theory:
“Yet there is a stone wall set across any clear view of our future, and it’s not very. far down the road. Something drastic happens to a species when it reaches our stage of evolutionary development, at least that’s one explanation for why the universe seems so empty of other intelligence. Physical catastrophe (nuclear war, biological pestilence, Malthusian doom could account for this emptiness, but nothing makes the future of any species so unknowable as technical progress itself.”
Vinge then goes on to explain that while the evolution of human intelligence took millions of years, we will now “devise an equivalent advance in a fraction of the time. We will soon create intelligences greater than our own.”
These advances will lead to what Vinge calls the Singularity:
“When this happens, human history will have reached a kind of singularity, an intellectual transition as impenetrable as knotted space-time at the center of a black hole, and the world will pass far beyond our understanding. This singularity, I believe, already haunts a number of science-fiction writers. It makes realistic extrapolations to an interstellar future impossible. To write a story set more than a century hence, one needs a nuclear war in between to retard progress enough so that the world remains intelligible.”
Vinge explains that those beings who live on the other side of singularity will be very different from us:
“A Cro-Magnon Man brought into our present could eventually understand the changes of the last 35,00 years. The difference between contemporary man and the creatures who live beyond the singularity is incomparably more profound. Even if we could visit their era, most of what we would see would be forever incomprehensible.”
In closing, while he fears a future where humans are not needed, he is still hopeful that we will use computers and AI to augment our human intelligence and that “as caterpillars who will soon be butterflies, we look to the stars, take that vast silence as evidence of other races already transformed.”
Vinge’s 1993 Essay: Warning of the Technological Singularity
While Vinge’s Essay in Omni is a great read, he is best known for his essay The Coming Technological Singularity: How to Survive in the Post-Human Era, which was first presented to the VISION-21 Symposium sponsored by the NASA Lewis Research Center and the Ohio Aerospace Institute in March 1993. Vinge again kicked off his talk with a bang:
“Within thirty years, we will have the technological means to create superhuman intelligence. Shortly after, the human era will be ended.”
Vinge explains that:
“The acceleration of technological progress has been the central feature of this century. I argue in this paper that we are on the edge of change comparable to the rise of human life on Earth. The precise cause of this change is the imminent creation by technology of entities with greater than human intelligence.”
Vinge envisioned a point where AI or augmented humans surpass human cognition, creating a “new reality” beyond prediction. Vinge saw risks, “a reality that might be beyond our ability to control,” and called upon us to recognize the need to shape the development of this new form of intelligence so that we can retain some level of control while we still can.
“If the technological singularity can happen, it will… Even if all the governments of the world were to understand the ‘threat’ and be in agreement about it, could they actually prevent it?
“We need to take seriously the possibility of superhuman intelligence and prepare for its consequences, because it could happen in our lifetimes.”
“Unless we can ensure that the resulting intelligence is human-friendly, we may face a future where humans are not the dominant species.”
However, he also makes clear that controlling this powerful intelligence, growing at an exponential rate, would be difficult, and suggested that our best hope might be to merge human beings with machines:
“I think it’s fair to call this event a singularity. It is a point where our old models must be discarded and a new reality rules, a reality that might be beyond our ability to control or even understand.
“From the human point of view, this change will be a throwing away of all the previous rules, perhaps in the blink of an eye, an exponential runaway beyond any hope of control.
“The best hope for long-term survival may lie in some form of intimate collaboration between humans and machines.
“Unless we can ensure that the resulting intelligence is human-friendly, we may face a future where humans are not the dominant species.”
You can read Vinge’s full article here.
Ray Kurzweil’s Approaching Singularity
We believe that with The Singularity is Near, in 2005, Ray Kurzweil wrote one of the most important and profound books of the last hundred years, building on themes he introduced in his previous book The Age of Spiritual Machines, where he introduced the Law of Accelerating Returns.
Kurzweil has predicted the evolution of machine intelligence far better than any current thinker. No one can doubt that over the last twenty years, the pace of technological change has only continued to accelerate, leaving most of us with a sense that we are indeed living through a technology tsunami that no one has explained as well as Kurzweil.
More importantly, the big question is, is he right about where we are headed? Are we really headed toward an event horizon twenty years from now called the Technological Singularity, where we will witness a rupture in the fabric of human history, the merger of biological and nonbiological intelligence, achieve immortality, and space travel, exploring the universe at the speed of light?
When we first started reading about the idea that humans would be able to migrate their brains to avatars and life forever, we introduced the idea jokingly because no one had ever imagined that death would become a technical question.
That concept seemed laughable 20 years ago, however, now that we are 20 years out from the predicted Technological Singularity no one is laughing because to date Kurzweil’s predictions have an astounding record of success.
Indeed, we first came across Kurzweil’s thinking in 2017 which prompted One World to design a technology awareness class because it was becoming increasingly apparent to us that it was virtually impossible to understand today’s world without understanding how technology was transforming our world prompting us to write: One World Tech Trends Study
In addition to his books, we came across excellent articles on this topic like the one published by Time Magazine in February 2011 which we highly recommend:
“2045: The Year Man Becomes Immortal” https://time.com/ archive/6595274/2045-the-year-man-becomes-immortal/
We believe the best way to understand Kurzweil’s thinking is to start by understanding the Law of Accelerating Returns. In Chapter #1, we included a description of Ray Kurzweil’s Law of Accelerating Returns. We are including the first two paragraphs from his essay here as they are quite important. We start with the second paragraph first:
“You will get $40 trillion just by reading this essay and understanding what it says. For complete details, see below. (It’s true that authors will do just about anything to keep your attention, but I’m serious about this statement. Until I return to a further explanation, however, do read the first sentence of this paragraph carefully.)”
What is important to understand is that Kurzweil is an inventor who never had a boss his whole life until Larry Paige from Google asked him to join his firm and do whatever he thought made sense.
The interesting point here is that Kurzweil developed the law of accelerating returns as part of his effort to understand and predict where technology is headed. As an entrepreneur he understood that you need to skate to where the puck is going to be not where it is today. To be a good entrepreneur you need to correctly anticipate trends.
So, while Kurzweil’s $40 trillion offer is a bit tongue in cheek, we do believe that anyone who understands where technology is headed and who can correctly anticipate tech trends will be able to be a successful entrepreneur. In fact, we believe that understanding these trends is essential to 21st Century success for both individuals and our global society which is why we have written this book.
The first paragraph in the Law of Accelerating Returns is by far the best way to sum up Kurzweil’s thinking, where he sees technological change evolving at an ever-increasing exponential rate of change such that we will live 20,000 years of progress in our lifetimes leading to a rupture in the fabric of human history culminating in a Technological Singularity in 2045. Here is the first paragraph:
“An analysis of the history of technology shows that technological change is exponential, contrary to the common-sense “intuitive linear” view. So we won’t experience 100 years of progress in the 21st century — it will be more like 20,000 years of progress (at today’s rate). The “returns,” such as chip speed and cost-effectiveness, also increase exponentially. There’s even exponential growth in the rate of exponential growth. Within a few decades, machine intelligence will surpass human intelligence, leading to The Singularity — technological change so rapid and profound it represents a rupture in the fabric of human history. The implications include the merger of biological and nonbiological intelligence, immortal software-based humans, and ultra-high levels of intelligence that expand outward in the universe at the speed of light.“
For anyone interested in understanding where we are headed over the next 20 years we highly recommend Kurzweil’s two books, The Singularity is Near and The Singularity is Nearer.
We discuss Kurzweil’s thinking on the potential for humans merging with machines via a Brain Computer Interface in Chapter 6. In the meantime, we have included below a number of his important ideas below.
The Rate of Change Will Continue to Accelerate
The Law of Accelerating Returns predicts that we won’t experience 100 years of progress in the 21st century—it will be more like 20,000 years of progress at today’s rate. By the 2040s, the pace of technological change will be so rapid that a single day will see advancements equivalent to centuries of prior progress.”
As we approach 2045, the exponential growth of technology will reach a crescendo, with new paradigms like quantum computing and molecular nanotechnology sustaining the Law of Accelerating Returns. This will make the Singularity a period of unimaginable innovation, compressing millennia of progress into moments.
We Will Have the Potential to Overcome Death
By the 2040s, nanobots will repair and enhance our biological systems, achieving longevity escape velocity where life expectancy increases faster than time passes. We will also be able to upload our minds into nonbiological substrates, creating software-based humans with the ability to back up their consciousness.
In the mid-2040s, AI-driven nanomedicine will conquer aging, with nanobots rebuilding our bodies at the molecular level. Longevity escape velocity will be achieved, and mind uploading will offer digital immortality, allowing us to live indefinitely as software-based entities.
UBI, or Universal Basic Income Made Available
As automation displaces routine jobs, universal basic income will become necessary to ensure economic stability. By the 2040s, the abundance created by superintelligent AI will allow humans to focus on creative, intellectual, and social pursuits, redefining the nature of work and society.
By 2045, universal basic income will be a global norm, supported by the wealth generated by AI and nanotechnology. Society will shift toward a post-scarcity economy, where humans pursue art, science, and exploration, enhanced by our merged intelligence.
Virtual and Physical Reality will Merge
By the 2040s, virtual reality will be indistinguishable from physical reality, with nanobots providing full-immersion sensory experiences. We will live in simulated environments as easily as the physical world, with augmented reality overlaying information onto our perception.
Nanobots will enable virtual and augmented realities that are more vivid than physical reality by 2045. You’ll be able to live in a simulated Renaissance Florence or enhance your physical world with real-time data, all seamlessly integrated into your consciousness.
The Singularity is a Cosmic Event
The Singularity will allow us to transcend the limitations of our biological bodies and brains. By 2045, we will begin to saturate the universe with our intelligence, using nanotechnology to convert matter and energy into computational substrates, spreading at the speed of light.
By the mid-2040s, our merged intelligence will expand outward, colonizing the cosmos. Nanotechnology will transform planets and stars into intelligent systems, fulfilling the universe’s potential to become saturated with consciousness.”
At the Singularity, we will redefine what it means to be human
At the Singularity, we will redefine what it means to be human. Our consciousness will be a pattern of information, not tied to biological substrates, much like a stream’s identity is its flow, not its water. This will raise profound questions about individuality and existence.
By 2045, the merger of human and machine intelligence will force us to rethink identity. As we become software-based entities, our sense of self will be defined by patterns, not bodies, requiring new ethical frameworks to ensure superintelligence aligns with our values.
In closing this chapter, we believe it is important to point out that the term used by the futurists cited above to describe what lies ahead is, intentionally, a term borrowed from physics: singularity.
In physics, the singularity describes a point of infinite mass density in a black hole, where space, time, and physical laws break down, rendering outcomes unpredictable. In The Singularity is Near (2005) and The Singularity is Nearer (2024), Kurzweil adapts this concept to describe a technological singularity—a future point where exponential technological progress, driven by superintelligent AI, surpasses human intelligence, fundamentally transforming society in similarly unpredictable ways.
We point all of this out to make clear that after seventy-five years of discussion, this is our last chance to guide and shape the technology tsunami overwhelming humanity today. As we make clear in the next chapter, the future has arrived.