Human Compatible: Artificial Intelligence and the Problem of Control

Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell, Viking

What human beings have been able to achieve has been the result of our unique ability to communicate and convey knowledge to each other and onto successive generations. We have stood on the shoulders of giants, used and built upon the knowledge of our ancestors first through the word of mouth, and then through written language—mostly in the form of books. The result in our very human civilization. The prospect of conveying the culmination of that knowledge and information to intelligent machines is both incredibly exciting and potentially catastrophic. 

The profound and mostly unanswered question at the foundation of AI researcher Stuart Russell’s new book on those efforts is “What if we succeed?” It is a question he has been asking, in publication, since he wrote his first textbook on artificial intelligence in 1995.

By 2013, I became convinced that the issue not only belonged in the mainstream but was possibly the most important question facing humanity.

Speaking to the lack of imagination the AI community has in envisioning the future of their own research and development, Russell writes:

Robots are imagined as individual entities carrying their brains with them, whereas in fact they are likely to be wirelessly connected into a single, global entity that draws on vast stationary computing resources. It’s as if researchers are afraid of examining the real consequences of AI.

But Russell’s book is as much about the history of ideas and theories that have brought us this far as it is about where the technology we’ve developed based on those ideas is now leading us. It contains a brief history of the evolution of intelligence from single cell organisms to the utility, game, and probability theories that paved the way for us to model our machines as rational agents, based partly on the idea that we are ourselves rational agents, and why we might want to chart another course.

He also provides a history of the hardware meant to carry this out, beginning most notably with Charles Babbage’s Analytical Engine, and Ada Lovelace’s larger intellectual step that it could be a more general reasoning machine applied to all subjects. It took a century for such a universal computing machine to actually be built based on their theoretical work. He discusses the areas in which machines already outpace humans, and the conceptual hurdles we still need to scale in order to achieve general-purpose, human-level AI. Discussing why quantum computing, despite its great potential, isn’t the answer, he writes:

Speed alone won’t give us AI. Running a poorly designed algorithm on a faster computer doesn’t make the algorithm better; it just means you get the wrong answer more quickly. (And with more data there are more opportunities for wrong answers!)

Examining the ways in which poorly designed algorithms—particularly how the not “particularly intelligent” content-selecting algorithms on social media maximize click-through by promoting extreme views—are already affecting the world today, Russell does not mince words: 

Like any rational entity, the algorithm learns to modify the state of its environment—in this case, the user’s mind—in order to maximize its reward. The consequences include the resurgence of fascism, the dissolution of the social contract that underpins democracies around the world, and potentially the end of the European Union and NATO. Not bad for a few lines of code, even if it had a helping hand from some humans. How imagine what a really intelligent algorithm would be able to do. 

It may seem harsh to lay all those ills at the feet of social media, but would any of them be occurring without it? I am sure none of this was in Mark Zuckerberg’s intentions when he was creating a website to rank the attractiveness of his fellow classmates at Harvard, which is exactly the point.

As the technology becomes smarter, it is imperative that we imbue it with broader objectives that are beholden to human preferences rather than strictly commercial or narrowly preprogrammed ones. The problem is not only that our preferences are often in conflict with each other, it is that they are often in conflict with our own. Our current preferences more broadly are almost certain to be in conflict with our more evolved human descendents. AI will therefore have to be adaptable.  

Logic requires certainty, and the real world simply doesn’t provide it. 

He refers to Google and Facebook as “information utilities,” which is a good way to think of them, even if it is anathema to their current business model and desire to remain largely unregulated. 

Alongside climate change and nuclear proliferation, AI is one of the existential issues facing us as a species, indeed life on Earth. It raises questions of surveillance, disinformation, a jobless future, algorithmic bias, and autonomous weapons. There is also a danger that we will end up serving a technical infrastructure we no longer fully understand, and will be vulnerable to becoming generally incompetent in managing oursleves in the case of a computer glitch or cyberattack. And, as Russell states in relation to machine intelligence earlier in the book, when it comes to AI effecting a desired outcome, “competency trumps consciousness.”

When trading algorithms caused the 2010 “flash crash” on the New York Stock Exchange, wiping out $1 trillion in a few minutes, the only solution was to shut down the exchange. What happened is still not well understood.

Money is not merely paper currency, a piece of plastic, or numbers stored in a machine. It is a social contract and system of societal trust. In our relationship to each other and the universe, consciousness may ultimately trump competence, but if we can’t trust the underlying systems that govern our collective behavior in society like the financial system, systems that we’re increasingly entrusting to machines, we may not have much room to contemplate those larger relationships and further human civilization. If we can trust them, and retain our autonomy and human agency, our ability to understand and intercede in the systems that govern us, the prospect for human flourishing would be unmatched. 

His argument is that “we need to steer AI in a radically new direction if we want to retain control over increasingly intelligent machines.”

To AI we will need to add ideas from psychology, economics, political theory, and moral philosophy.

If AI is eventually able to help us extend human life indefinitely, who makes the decision about whether that is actually a good thing? We would most likely prefer not to be manipulated by algorithms, but what if the nudge they provide is toward self-improvement and personal development? 

These questions are a call to action to all of us to pay attention and get involved in the conversation. The book also acts, in part, as a narrative bibliography and history of those topics, referencing many of the most consequential ideas and theorists of each field. You could create a really great reading list from the books he touches upon. 

The more one studies AI, the more one realizes that it is not a new development that breaks from the past, but a field that has been built on the theories of mathematicians and philosophers dating back to Aristotle, that is working on understanding the nature of existence and intelligence, and of the evolutionary steps that have brought us to this point. It is “the accumulation and communication of knowledge and concepts by thousands of people across centuries of observation and research.”

Right now, we program objectives into machines and develop algorithms for them to achieve those objectives. Russell suggests we design machines that meet our objectives, and that we program them to learn them by observing our behavior and learning from us rather than programming them in. They will defer to us, ask our permission to proceed, and allow themselves to be switched off when it is not given. It will also require AI agreements, governance, and oversight on an international level. If we can get it right, we can turn our attention to life’s greatest mysteries:

How does the universe work? How did life begin? Where are my keys? These are fundamental questions worthy of thought.

But first we must answer the most fundamental question facing us today: will we control development in artificial intelligence, or allow them to control us?