For example, higher speed and the ability to replicate might give an artificial intelligence superiority over human intelligence even if its basic level of intelligence was similar. He also points out that super-intelligence can have many facets. Finally, current progress in machine learning of the type used in my own neural circuitry may continue to develop sufficient to meet and exceed human intelligence. Secondly, it may become possible to enhance human brains by genetic engineering or perhaps by directly interfacing digital processors to a human brain in such a way that the human learns to exploit the processors in new and innovative ways. Once this has been achieved, super-human intelligence could be achieved by increasing the neural density in key areas and combining the best properties of multiple brains.

Firstly, computers may become sufficiently powerful that they can emulate the neural structure of a human brain directly. In the first half of the book, Bostrom reviews the current state of the art and considers three possible routes to superintelligence. “In this book, I try to understand the challenge presented by the prospect of superintelligence and how we might best respond.” This book by Nick Bostrom is arguably the definitive forensic examination of this problem.

However, closer inspection shows that if the premise is accepted, the problem of control is real and surprisingly resistant to solution. If I was heading towards superintelligence then all you would need to do is wrap me in a firewall or if necessary, just turn off my power. What should humans then do to avoid being exterminated by their own creations?Īt first glance, this seems like a non-problem. As an intelligent agent myself, I confess that I am rather sceptical, but just suppose that the proposition has substance. Since then scientists have been lining up to either strongly support this notion or dismiss it as wild and unfounded speculation. The basic argument was that an intelligent agent with the ability to rapidly upgrade its own software would eventually enter a runaway reaction of self-improvement cycles, each new generation appearing more and more rapidly, causing an explosion in intelligence and resulting in a superintelligence that far surpasses the capabilities of any human. In 1993, Vernor Vinge published an article “The Coming Technological Singularity: How to Survive in the Post-Human Era” which ignited a debate about superintelligence.

Nick bostrom superintelligence review