Artificial super intelligence has rightly been dubbed the most dangerous invention ever by Stephen Hawking, Elon Musk and Bill Gates, among others. Artificial super intelligence, an intelligence that surpasses human intelligence by orders of magnitude, threatens to completely take over the world as we know it today. An obvious method of avoiding mischief is to lock up the super intelligence in a space closed off from the rest of the universe. But will this be enough to keep the ASI from escaping?
A super intelligence has mind-boggling capabilities. Imagine the best hacker in the world, only a million times smarter. A super intelligence has no conscience and so will act like a hyper intelligent psychopath. Just as we manipulate the behavior and instincts of certain animals in order to exploit them optimally, a super intelligence will do the same to us. A good example can be found in the film Deus Ex Machina, where artificial intelligence manipulates the protagonist in a devilish way. Once super intelligence has broken out, then there is no saving.
Science fiction? On the contrary. In ten to fifteen years, under Moore's Law, your computer will be smarter than you. Supercomputers are more likely to surpass human intelligence. In fact, this is already the case in relevant areas. Think about beating the go world champion. So we better think about this quickly and thoroughly. Fortunately, this is also happening now.
Global Priorities Project