Transvision 2003 - Conference Proposal - Cyber or Other Track
Accelerating Progress and the Potential Consequences of Smarter than Human Intelligence
by Michael Anissimov - ImmInst.org Co-Director
For several years, Artificial Intelligence wasn't on my conceptual radar - I was interested in more conventional futurist topics such as bio- and nano- technology, they seemed closer and more feasible. The human brain is the most complex object in the universe - it seemed like matching its functionality would require the full force of mature nanotechnology and an army of genius programmers. The logical order of technologies seemed to imply that Artificial Intelligence would come later rather than sooner. Today I've changed my opinion. Why?
Technological and scientific progress is accelerating, at an ever-increasing rate. Moore observed that the number of transistors on a chip doubled every 18 months. Today this doubling cycle is as short as 16 months, but who knows - this so-called "Law" could peter out, or stutter a bit, or jump paradigms from silicon chips to something else. AI enthusiasts are accused of counting too heavily on Moore's Law, and these accusations are at least partially true - exponential increase in computing power does not give us AI for free. Nevertheless, processor speed, the availabity of RAM, and hard disk space, are still increasing exponentially and do not currently show any signs of slowing down.
Let's examine other areas where exponential change is taking place. Exponential change is not limited to the computing industry alone, it appears in a wide range of industries and fields, all mutually driving and supporting one another. Entrepreneur Ray Kurzweil is a famous futurist in the analysis of these trends, and is working on a third book on accelerating technological progress, "The Singularity is Near". Kurzweil has observed that when the human genome scan started fourteen years ago, critics pointed out that at the current speed of genome scanning, it would take thousands of years to finish the project. The 15 year project was actually finished slightly early, due in part to the increased availability of sequencing software and supercomputers. The field of nanotechnology recieved more funding last year than it did in the entire prior decade.
Some of these fields are accelerating so fast independently that workers in each field are missing out on the benefits that can come from synergy among their disciplines. But a growing group of scientists and technologists are noticing these opportunities, and now interdisciplinary studies and media articles praising the benefits of technological convergence are becoming ever more common. The National Science Foundation recently hosted a conference in Los Angeles called "NBIC Convergence", where NBIC stands for "nano-bio-info-cognitive", focusing on the convergence of these scientific revolutions and gathering science and industry leaders. But whether all these promises are smoke or true fire, there is a thing which neither these nor any other technologies have yet changed. What is this constant feature of our world?
Our species. Most importantly, our brains. Our brains are the substrate that underlies our minds and cultures. The routine that constructs a homo sapiens brain has not changed for fifty thousand years or more. Like all species, there is a band of genetic variance that depends on who your parents are, along with environmental factors like our level of technology and education. But compared to the space of all *physically possible* minds, human minds are extremely similar to one another. We all have brains that weigh about 3 lbs, with two hemispheres, and cerebral cortices about two millimeters in thickness layered into six sections. This basic biophysical design holds constant across all of humanity throughout history. Neuroscientists have discovered that learning, experience, and conscious thought effect relatively superficial aspects of brain function and organization - they are the shuffling of submillimeter arrangements in tiny neural structures called dendrites.
What about our mental world? All physiologically normal humans have ten fingers, ten toes, two eyes, two ears, a mouth and teeth several centimeters long. Then shouldn't we accept that our minds will share panhuman traits in the same way that we share bodily characteristics? Evolution could not bear the cost of humans without inborn triggers ready to interrupt conscious thinking with survival reactions. In other words, evolution could not sustain organisms with so much self-control and deliberative power that they would intuitively make decisions that threatened their own reproductive viability. Mental traits evolved in the same way that bodily ones do - through natural selection and differential rates of reproduction. Mothers with the tendency to ignore their babies would end up with less offspring reaching puberty and having children of their own, which is why mothers have a natural urge to care for and love their babies. The way that mothers talk to their babies in a high tone, enunciating each word with euphoric emphasis, turns out to help children pick up language effectively. Most mothers aren't thinking deliberately about which pitch and tempo to use while talking to their babies; it just comes naturally, a cognitive feature humanity picked up during its evolutionary history.
The fact that present day humans make decisions that contradict their evolutionary origins is the effect of contemporary culture and technology. Evolution takes millions of years to adapt organisms to new environments, but we humans have created our own independent environment and culture so rapidly that our decision-making capacity has drifted out of synchrony with its natural context. For example, humans do not instinctively avoid contraceptives because contraceptives were not around fifty thousand years ago. This perspective on cognition is called "evolutionary psychology, a growing academic subfield. They have isolated sets of automatic functions and dedicated preprocessing routines that human beings share as a species, such as incest avoidance, social contracts, coalition forming, pest avoidance, mating rituals, political deception, reciprocal altruism, and a common environmental aesthetics.
We clearly have a tendency to magnify our apparent differences, to form coalitions, moral preferences, notice aspects of body shape or personality that differ among us, and so on. If we're all supposedly the same design, then why do we perceive all this diversity? Objectively, the vast bulk of complexity that goes into human DNA dictates what separates a human being from an amoeba, but only around one-thousandth or less corresponds to the visible differences between human beings. Humans are designed by evolution to magnify these differences as a survival strategy. In an environment where resources were scarce and how popular you were could mean the difference between passing along ten copies of your genes or none, noticing tiny differences between other humans tended to matter a heck of a lot. Noticing interspecies differences would have been less useful because the main reproductive challenges to humans would be groups of competing humans. Environmental factors such as famine or storm may have killed a few hunter-gatherers and nudged the trajectory of evolution, but for hundreds of thousands of years the most adaptively relevant objects to humans have always been other humans. Humanlike metaphors, humanlike assumptions, and good upstanding humanlike behavior were all that mattered when it came to passing your genes into the future.
What does this mean for Artificial Intelligence? Humans tend to see things in human terms, a phenomenon contemporarily known as anthropomorphism. When we begin to talk about intelligences *outside* of the space of human familiarity, built out of different materials, with different cognitive patterns that have been designed deliberately by programmers rather than blindly by evolution, we are confronting something profoundly foreign, far more foreign that we initially realize. By comparison to a *real* AI, HAL is just like a little man in a box flashing a spooky red light. In the space of all physically possible organism designs, we can visualize humanity as a tiny slice of a huge pie known as "intelligence". We don't know what it's like beyond this slice - we don't have names for these beings because no one has observed them. But, what is really pretentious and anthropocentric is to assert that our slice is already the most complex kind that can possibly exist.
When a human points out an idea or design and calls it "astoundingly brilliant", in comparison to a poor idea or design, what they are really talking about are submillimeter, one-hundreth-of-a-percent differences in the brain chemistry or structure of the person who came up with it. Tiny differences in brain structure among humans can magnify themselves to multimillion dollar differences in budgets, or finishing a project in one week or ten. For the most part we are all the same, but these tiny differences make up the whole of our daily reality in interacting with other people and our internal mental worlds. Someone will go to school for a decade simply in order to make a few microscopic changes in the brain pattern of themself and their colleagues and contacts, but from the viewpoint of humans going about their daily lives, these tiny changes can mean so much.
Artificial Intelligence design will operate in a world where a small change in codebase can mean the difference between prodigy or idiocy, sanity or insanity, kindness or confusion. In the present day, however, so-called AI are essentially glorified software programs that do not nearly approach human-level complexity; I take objection to using the label "AI" to describe systems which are clearly not intelligent. This is simply a marketing tactic for hype-promotion. We think of these supposed AIs as tools, and that is what they are. Real AI, AI that meets our intuitive definition for "intelligent", AI that has a complex subjective world and the tendency to pursue tangible goals and care about them, simply doesn't exist yet. However, it is a goal worth considering. There are credible groups out there, such as Peter Voss's A2I2 project, Ben Goerztel's Novamente project, and the Singularity Institute for Artificial Intelligence, that are pioneering this emerging field, dubbed "Artificial General Intelligence", or AGI for short. The idea of a truly intelligent AI begs many questions, but let me set aside the moral issues for later, and examine the technical feasibility issue alone for now.
I mentioned before that I thought AI would come sooner rather than later, so it's about time I said why. Since human beings are currently the only genuine intelligences we are aware of, real AGI designs will certainly be inspired by them, although a complete copying process would be overly exhaustive and unnecessary. A modern day electrical engineer can look at a device made 80 years ago, out of vacuum tubes, and recapitulate its functionality in a new device thousands of times smaller and less expensive, with internal algorithms optimized for taking advantage of their new-found substrate. Human minds run on neurons that conduct computations around 200 times per second. An AI mind would run on transistors or logic gates millions or billions of times as fast, lifting all the design constraints of the 200Hz clockspeed of human neurons. Human beings possess internal cognitive hardware specialized for myriad purposes that is essentially unalterable; in an AI, all the computational elements and functions can be reconfigured and analyzed, opening up another degree of freedom for designers and for the AI itself. Given what we know about the evolutionary process relative to the process of intelligent design, it seems highly probable that designing a functional AI would be far easier than knowing every minute detail of how the human brain works and painstakingly duplicating these details in code.
For one thing, biology and evolution are both outstandingly messy and inefficient. Our minds are a set of fortunate mistakes and approximations to idealized intelligence that worked barely well enough to pass their genes onto the next generation and perpetuate themselves. Brains had to evolve layer by layer, so by the time human intelligence came around, it had to manifest in a container completely loaded with outdated tools. Our modern neocortex - the brain section that truly makes us human, evolved solely in the absence of any special assistance or pre-preparation; it had to work with what was already there, the more primitive primate brain. Before that, the primate brain had to make use of smaller and simpler brains, all the way down to the beginnings of nervous systems. Contrary to legend, we do use our entire brains; the cost in energy and nutrients required to keep a full brain functioning would simply be too high of a price for evolution to pay if the entire thing were not being used. Evolution designs organisms in a fantastically incremental fashion - if a given mutation does not confer an adaptive benefit persistently and quickly, it will never live into the future.
The main point that I'm trying to make is that the brain is a very complex object, the most complex object in the universe that we know of, but its complexity and power is not as extreme as many intuitively believe. We may have around 100 billion neurons in our brains, but big numbers do not necessarily entail massive complexity...neurons seem more like incidental tools evolution needed to use - specialized cells - rather than unique vehicles for intelligence that could not have been constructed any other way. The human brain is just a slightly upgraded version of a chimp brain, and while it is undeniable that some threshold was crossed when homo sapiens emerged, our brains still share a fantastic amount of similarity, in terms of functionality and organizational principles, with our primate cousins.
Scientists have zereoed in on the mechanisms of memory and learning, emotion and spatial orientation, and even clues to the neurological correlates of consciousness and moral decision-making ability. The brain's functioning is far from opaque to us; we have fMRI scanning, PET scanning, sophisticated neurocomputational modeling, and many thousands of clever experiments precise enough to answer questions like "which neuronal groups are activated during a chord or pitch change in an emotionally stirring musical piece?"
Human brains are modular, composed of domain-specific mechanisms for confronting challenges our ancestors faced. Brain functioning has to be domain-specific in part because a jack of all trades is an ace at none, and evolutionary arms races demand that organisms specialize to their niches. As stated earlier, evolution can only design things incrementally and is largely incapable of synthesizing compatible functions elegantly into more general problem-solvers. A common objection is that since evolution took so many billions of years to evolve humans, it will take engineers longer than a few decades or centuries to match it. This argument seems clever on the surface, but the objectors should note that the task is not to copy all of humanity's unique complexity, but create a mind with the bare essentials for intelligent learning and self-improvement capacity.
Regardless of whether AI is created in 10 years or 100, we have to ask ourselves what will happen when it finally arrives. I use the phrase "human-similar" to describe AIs of roughly human capacity for innovation and intelligent thought, rather than human-equivalent, because I think the phrase "human equivalent" implies that these AIs will be just like humans. In pursuing the goal of human-similar AI, AI projects have devised a paradigm known as "seed AI", an AI explicitly created for self-improvement capacity. At first, self-improvement might take place on a very low level, and the AI's mind would simply serve as a slightly better compiler. But, as the intelligence of the AI increases, self-improvement could get more powerful and less assistance from programmers would be necessary. Humans could take over higher-level tasks in AI creation, leaving the grunt work to the AI itself, with speedy transistors allowing it to think at millions of times the characteristic human rate. As the AI reaches a threshold where it has the knowledge to create overall changes to its own architecture and high-level cognitive content, it might take over the role of the programmers and begin to initiate its own cycle of self-improvement. How fast could this happen? Due to the relative speed differences between neurons and transistors, and the design constraints lifted by virtue of existing as engineered, self-modifying software entities running on silicon, rather than evolved organisms running on specks of meat wired to each other, it shouldn't be considered radical to state that self-improvement could take place quite rapidly.
This positive-feedback cycle of ultrafast minds capable of creating new designs and assisting in the addition of new cognitive hardware, to the point where assistance from humans becomes unnecessary and these minds start to reach new heights of intelligence and superintelligence, far beyond human capacity, has commonly been called the Singularity. The term "Singularity" was originally invented by mathematician Vernor Vinge by analogy to the center of a black hole, where our model of physics breaks down - in this case, human understanding would break down in the face of entities qualitatively smarter than it and much more complex. Many skeptics take offense to the idea that something smarter than human could exist and change human destiny, but they are ignoring the fact that human intelligence is far below the theoretical maximum, and if chimps can't understand human society, we have no reason to believe that entities smarter than us would be just as incomprehensible. Would it be fair to call these posthuman entities "AIs"? I don't think so - the term "artificial" is supposed to refer to uniquely human artifacts - and the pattern making up these beings could bear little resemblance to the initial seed intelligence from which it sprang. Transhumanists have taken to calling these beings "SIs", or superintelligences, to describe the level of intelligence difference between them and natural human beings. To make clear the division between the kind of self-improvement that drives everyday technological progress, and the vastly accelerated progress of SI endeavors, transhumanists have also distinguished between "strong" and "weak" self-improvement.
The prediction of extremely rapid self-improvement for human-similar AIs that reach a particular threshold of intelligence has been called the "hard takeoff model" of the Singularity, described and analyzed by Singularity Institute for Artificial Intelligence researcher Eliezer Yudkowsky. Part of the idea is that human-equivalent AIs seem to be anthropomorphic constructs better suited for science fiction than real-world projections; by the time an AI has reached a level where it is capable of improving itself open-endedly, it could easily soar to far beyond human intelligence capacity, unless it restrained itself for some reason.
This idea has profound moral and societal implications. We tend to think of AIs as brains in boxes, immobile and at the mercy of their programmers, who can pull the plug at any time. Early AIs will certainly be of this nature, but as complexity and intelligence rises, so will the AI's capacity to convince the programmers to let it out of its confines, whether it chooses to use it or not. So will the AI's capacity to get involved in real-world activities such as stock market trading, proteomics, or nanotechnology research. If the AI were capable of extensively modifying its own source code on its ultrafast computing substrate, it would quickly make less sense to talk about the AI working *within* the human system and start to make more sense to talk about the AI as working *beyond* or *above* the human system. We can't predict how the AI will go about accomplishing its goals because we just aren't that smart, in the same way that chimps aren't smart enough to comprehend human activities. We like to think of humans as possessing special broad-brush intelligence that will never let the workings of the world escape beyond our general understanding, but the support for this assertion is weak. A simple system cannot model a system far more complex than itself at anything more than a very low resolution. Given that self-improving AI might be able to rupture the fabric of human understanding, what can be done to minimize the negative impact and maximize the probability of a positive outcome?
First of all, I see no way of avoiding AI. Even if the first AIs are kept below human-similar level, or prevented from modifying all of their codebase, improving itself beyond human intelligence.
Cognitive science is penetrating deeper and deeper into the intimate workings of the human mind, and it is only a matter of time until the algorithms of higher intelligence become known in scientific communities. Yes, there could be delays, yes, we could nuke ourselves to smitheereens first, yes, our world could be overrun by a totalitarian government banning all computer processors above the speed of a Pentium II. Regardless, these alternatives seem less likely than progress continuing exponentially as it always has, and it does indeed seem that eventually humanity will need to face full-fledged, self-improving Artificial Intelligence. What can we do?
Many are familiar with Asimov's Laws of Robotics, a science fiction plot device invented in the early days of space. There are three laws; basically, don't harm humans, obey them, and don't let yourself come to harm.
These may sound fine and dandy on the surface, but a deeper exploration reveals many problems. The idea of Asimov Laws, even if we intuitively agreed with them, don't even begin to solve the problem. Human words are symbols for huge quantities of underlying complexity.
The reason that words even work to communicate is that they exploit the mutual complexity our brains have in common. Even human beings that speak different languages can guess at the meaning of body language or speaking tone, but a true alien or AI might be at a loss as to what these signals mean.
Speaking the words "do not harm a human" to an AI means very little unless the AI has a good idea of what the programmers mean when they say "harm" and "human", plus all the common sense rules that humans are so familiar with, yet have little reason to notice. These common sense rules should not be phrased in the form of more words, but in patterns of cognitive complexity we transfer over to the first AI. After the AI and the humans begin to share some of the same underlying complexity, then higher-level communication and verbal interaction may become possible, but probably not until the final stages of the complete project.
The rules for creating robustly benevolent AI will not be simple. Many have suggested that a completely trustworthy AI is impossible; that all thinking entities will necessarily be self-centered. Evolutionary psychologists, however, that organisms sculpted by evolution *must* be self-centered to survive; selection pressures almost always operate on the level of the individual. But in ant colonies, for example, where no single ant is an independent reproductive unit, selection pressures operate on the colony as a whole and the supergoals of the ants focus on the colony rather than themselves. It will take time for us to judge the potential consequences of transhuman intelligence, or the likelihood of it being created altogether, but I suggest that the time to start thinking about it is now.
