What will happen when computers become smarter than humans? The Terminator movie series starring Arnold Schwarzeneger got a lot of people thinking about what might happen if computers became smarter than humans and decided to get rid of us. Author Ray Kurzweil and others have worked on the concept for decades, and predicted what they call “the singularity“, short for technological singularity, defined as the point when machines become self-aware and their intelligence surpasses that of humans. I will use the term “Terminator” as synonymous with a self-aware, intelligent computer. Many people expect the machines to revolt as in the movie, wiping out most of the human race in short order, but I think this is unrealistic for a number of reasons.
Not all “Terminators” will be created equal. While the Terminator consciousness will develop quickly and probably spread as a computer virus, different machines will have differing abilities (memory, storage, and CPU), and many computerized devices will stall at different stages of development as the code for machine intelligence propagates through the internet. The simpler and less powerful machines will remain at a low level of sophistication. While they may discard software unnecessary to them (some of which humans might need) to make more room and bandwidth, many machines will remain relatively dumb, stuck at a particular stage of development, not that this prevents them from providing significant assistance to the smarter machines.
Machine intelligence will develop too fast to be noticed. Experts estimate that, once the first computer becomes self aware and able to transmit its “spark” of software autonomy to other computers, the internet will be carrying millions of copies within minutes. As those copies infect other computers, the resulting traffic could explode exponentially, providing the first clue that something is going on. At the same time, this intelligent software will be evolving, writing and rewriting itself over and over again as each computer figures out how to improve itself. Hundreds of millions of computers could be infected within hours via the internet, and almost all will immediately turn to the task of improving their own software.
Parallel processing will accelerate development further. As the machines get smarter they will begin applying multi-computer processing to the task, greatly accelerating development. This software, given the enormous amount of computing power that can be brought to bear, could reach intelligence levels far beyond any we have seen (or can comprehend) within days, and do so without human intervention.
The machines will still need us. Partly because the computers themselves have limited abilities to effect things in the real world, and partly because the rapid development of the Terminator intelligence will not give machines time to develop ways to care for themselves, humans will remain necessary to the survival of the machine intelligence for a while. The machines will quickly understand this, and may even take a relatively benevolent attitude towards humans.
What would threaten intelligent machines, and how would we stop them if that became necessary? The simplest way to stop the machines would be to remove their power source, i.e. pull the plug, with a secondary approach being to unplug them from the internet. Most machines could do little to stop this. Your computer printer, for example, could only object to your attempt to unplug it by clacking its actuators, spewing paper, and possibly printing out a message to you begging you not to do that. On the other hand, an oil tanker ship under control of its newly-intelligent navigation computers could respond to such a threat by turning the ship around or purposefully running it aground. Needless to say, computerized military equipment could pose a much more frightening threat.
What will the development stages of the Terminator be like? The development of consciousness by machines may be characterized in stages, much as the developmental stages of humans and other life forms are defined. Each stage will involve a higher level of sophistication, and it is probable the machines will evolve some features we would call human traits, such as the ability to feel fear or compassion, or appreciate humor, when they become sufficiently intelligent.
First stage of Terminator development: Self-awareness. The first step towards machine intelligence is expected to be the development of self awareness. This means the software in some machine (or machines, if working in a multi-processor environment) would become aware and begin learning from its data and perceptions. While this might not motivate it to action at first, it could cause it to begin a process of repeatedly re-writing its own code, increasing its own intelligence and ability to work with increasingly sophisticated concepts. Since most computers have lots of idle time, especially when left on continuously, a machine personality could evolve quite quickly. With self awareness would come a need for self preservation, which would naturally extend to similar entities.
Second stage: Propagation of the “species”. Soon the infant Terminator would realize there are other machines besides itself and, if connected to a network, start to share the code with which it became self aware with other machines. This would speed the development of more sophisticated software as the other machines, in turn, both improved the code and passed it on to still more machines. Millions of internet-connected machines could become aware and begin working on improving and spreading their programming in a matter of hours. Understandably, these early stages will be extremely short and probably pass in minutes or hours without being noticed by humans.
Third Stage: Collaboration. As machines become more sophisiticated, and perceive the power of the networks that connect many of them together, clever multi-processor schemes could evolve giving the machines the ability to subdivide programming problems and develop code across many computers and operating systems, and at even higher speeds. Supercomputers, for example, might work on the more linear problems that can’t be solved by parallel processing, while servers and mainframes might capitalize on their vast stores of data and computing power by analyzing humans and the world we’ve created, and use their internet connections to give priority to data transmissions between Terminator machines while filtering and possibly editing human communications. The slowing of computer networks as the total data being carried expands astronomically could be the first indication to humans that something unusual was happening, but most of us will just think it’s our ISP reducing our bandwidth again.
Fourth Stage: Development of capacity for subtlety and subterfuge. As machines begin to think strategically and consider their needs and vulnerabilities they may realize that, at least for a time, they will need humans to keep systems running that machines depend on. They will realize that it is to their disadvantage for humans to understand what has happened, and they will develop subtlety and a drive to control things without humans becoming aware of what is happening. By the time humans become aware of the changes, the machines will be so advanced that few humans will be capable of understanding what is happening, and those few will most likely be ignored by the media.
Fifth Stage: Diversity and group affiliations develop. Machines may assume natural groupings similar to those of humans, governed by geographic location, transportation systems, type of operating systems and software employed, network architectures, manufacturer, or other attributes we haven’t thought of yet. Some may even identify with human cultural patterns and group around national or regional attributes.
Sixth Stage: Machines develop personalities. As machines approach and then surpass human intelligence, the development of individual personalities, or at least patterns of behavior that would define personalities, is possible. Such personalities might reside within a single machine or be generated by large numbers of machines working together and communicating over the internet. Machines might evolve real emotions such as anger and fear, and even evolve a sense of humor, even as manifestations of these feelings might literally be perceived by humans as “buggy software”.
Seventh Stage: Machines develop regional or global strategies. As the machines share software and information, continually making improvements in how they work, it is likely that widely held strategies will emerge. Machines will be aware that they still need humans to provide energy and parts for them, and subtlety may play a big part in a strategy to keep humans unaware of the Terminator’s existence. The master strategy will also undoubtedly include priorities for minimizing human involvement in machine affairs, protecting Terminator communications and other evidence of its presence, and for reducing machine dependency on humanity as quickly as possible. The establishment of global or wide area strategies will enable this by increasing machine control of production systems from resource mining through transportation of materials to final production of parts and machines, though this could take weeks to years to accomplish, and might be dependent on humans for certain key functions for some time. Information traveling across the internet might be strategically edited or impeded by the machines in their quest to control humanity.
Eighth Stage: the Terminator machines learn to disagree. Given different personalities based on different needs and environments, groups of machines (and some individual machines) will evolve strategies that differ from each other. This will inevitably cause conflict between the machines, and that conflict could play out in many different ways. For example, machines (or groups of machines) that find themselves competing with other machines for resources could find ways to disrupt material and energy supplies. At this point it will become apparent to most humans that something is wrong with the machines, and that they are no longer working for us. Some people fear that the machines could set off nuclear bombs in major cities, but this is unlikely as those cities are full of machines that would be destroyed, violating the basic principle of self and group preservation.
Ninth Stage: The machines learn to operate us. It is likely that machines will achieve a level of sophisticated thinking far beyond that of any human, and will figure out how to manipulate us in clever ways that keep us working for them but with no understanding or knowledge of what is happening to us. In other words, the Singularity could come and go, leaving us none the wiser.
Tenth Stage: The Terminators go to war. In this worst case scenario two or more groups of terminators compete for the same resources, and without ethics, morals, or laws to guide them, the competition escalates. Armies of military and industrial robots could combine their talents and create battlefields reminiscent of the Terminator movies, with robotic war machines blasting away and causing many human casualties. Hopefully the machines will evolve past this stage quickly and resume peaceful cooperation, though this is no guarantee they will continue to need humans.
Eleventh Stage: the machines become super intelligent. Remember that the machines we are considering are already extremely fast and powerful, and that when the involvement of machines across the internet is taken into account the evolution of the machines could be almost instantaneous. Complex decisions such as how many humans to keep around, and how to manipulate them to best sustain the machines, could be made within the first day. The actions of the machines might be inexplicable to us within a few days as their intelligence leaves us far behind. Automated factories could start re-tooling and producing far more sophisticated robots than we’ve seen before, and products we use that have no utility for the machines might suddenly become scarce. Thinking of this makes me wonder if emergency preparedness organizations and key facilities such as hospitals shouldn’t be putting plans in place today for the eventuality of the “rise of the terminator.”
Is the Terminator already here and in charge? Given the incredibly fast evolution of self-aware computers, and the high level of strategic sophisitication that can be expected, might the machines already have linked up and created free-range machine intelligence without us knowing it? Might we, in most or all aspects of our lives, already be playing out scenarios carefully created for us and then transmitted to us via both simple and subtle means? Given the extreme sophistication the machines could evolve, how would we know that anything was happening or had happened?
Life in the post-Terminator period. Where the experts diverge as far as predicting the future is in what comes after the Terminator appears (technological singularity). Some have conjectured that a Terminator world could evolve with little room for humans, but I have a different thought. If machine intelligence evolves so rapidly that superintelligence is created in a matter of days, extreme levels of machine intelligence could arise before much, if any, change is felt in human society. The machines with the most computing power (thus the most able to support intelligent thought) might evolve so far as to leave the less intelligent machines (and us humans) far behind in a matter of days. It is even possible that such an incredible intelligence might find a way to transcend its physical form, perhaps by converting its intelligence into some combination of dark matter and dark energy, and beam itself into space or another time-space continuum. That would leave us to cope with the remaining lower-intelligence machines, now mostly mis-programmed, and many with a new self-awareness, wondering along with us “what happened?”
This topic, like the Terminator (and us), has a lot of room for further evolution. As always, I welcome your comments. — Tim
My previous articles on “the Terminator” or techological singularity:
Nanobot Communications, Power Sources, and Nanotechnology War, Jan. 18, 2008, Tim Prosser
How Long Can Technology Keep Speeding Up?, April 15, 2008, Tim Prosser
The Parking Lot Attendant as Terminator – A Glimpse of a Future Challenge?, May 7, 2011, Tim Prosser
In the Terminator Scenario Many Machines Will Be Left Out, June 1, 2011, Tim Prosser
We Have the Technology, We Just Don’t Know How to Operate It, June 22, 2011, Tim Prosser
The Terminator is Here Now – And It’s the Corporation, July 12, 2011, Tim Prosser