The science of Westworld
A summary of recent artificial intelligence research
By Mike Hearn
NB: Westworld is a TV show. This article contains spoilers.
Artificial intelligence made enormous strides in 2016, so it is fitting that one of the year’s hit TV shows was an exploration of what it means for machines to gain consciousness. But how close are we to building the brains of Westworld’s hosts for real? I’m going to look at some recent AI research papers and show that the hosts aren’t quite as futuristic as you might think.
“Ah yes, your mysterious backstory. It’s the reason for my visit. Do you know why it is a mystery, Teddy? Because we never actually bothered to give you one, just a formless guilt you will never atone for. But perhaps it is time you had a worthy story of origin.”
Story comprehension
The robots of Westworld are not programmed solely by software developers. The bulk of the work is done by professional writers, who give each character a unique backstory. These stories give them the memories and depth they need to seem real to the park guests. When asked who they are, what they’ve done or why they feel a certain way, they can consult their backstory to find out the answer.
Being able to answer questions about stories is a fundamental requirement for being able to pass the Turing test, which the show tells us started to happen “after the first year of building the park”. But Turing proposed his test as a kind of thought experiment, not as a useful yardstick for measuring progress in AI. A machine either passes or fails and that’s not very useful for figuring out how close we are.
To fix this, in 2015 Facebook’s AI lab introduced the bAbI tests in a paper called “Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks”. Quoting from the paper’s abstract:
To measure progress towards [building an intelligent dialogue agent], we argue for the usefulness of a set of proxy tasks that evaluate reading comprehension via question answering. Our tasks measure understanding in several ways: whether a system is able to answer questions via chaining facts, simple induction, deduction and many more. The tasks are designed to be prerequisites for any system that aims to be capable of conversing with a human.
In other words, before you can hope to pass the Turing test you must learn to pass bAbI.
The test is a large, auto-generated series of simple stories and questions, that test 20 different kinds of mental skill. Here’s one that checks the machine isn’t distracted by irrelevant facts:
Mary went to the bathroom. John moved to the hallway. Mary travelled to the office.
Where is Mary? Answer: office
Here’s a harder one that tests basic logical induction:
Lily is a swan. Lily is white. Bernhard is green. Greg is a swan.
What color is Greg? Answer: white
The bAbI tests check come in English, Hindi and a scrambled form where the English words are randomly shuffled so the tasks can no longer be understood by humans. To pass the test a machine should get equivalent results on all three: the idea is to learn everything, including the language itself, simply by reading. Programs specifically designed to handle bAbI can obtain near-perfect scores, but what about general AIs that are given only the words and nothing else?
The best result yet is from Facebook AI Research. The results are reported in their December 2016 paper, “Tracking the world state with recurrent entity networks” and their AI can solve all 20 tasks.
You can of course train a neural network both on these tasks and on large question/answer databases too, which yields an AI that can talk about the story using learned real-world knowledge:
Fred went to the kitchen. Fred picked up the milk. Fred travelled to the office.
Where is the milk? A: office
Where does milk come from? A: milk come from cow
What is a cow a type of? A: cow be female of cattle
Where are cattle found? A: cattle farm become widespread in brazil What does milk taste like? A: milk taste like milk
What does milk go well with? A: milk go with coffee
Where was Fred before the office ? A: kitchen
Similar algorithms have proven able to read — I kid you not — the Daily Mail, which turns out to be ideal for AI research because the stories come with bullet point summaries of the text (see the DeepMind paper, “Teaching Machines to Read and Comprehend”).
In this task an anonymised news story is presented and the goal is to correctly fill in the X. The answer is “ent23”. The heat map shows which parts of the text the neural network gave the most attention to figure out the answer. The names are randomised to stop AIs from answering questions like “can fish oils cure X?” as “X = cancer” without even reading the document, simply by knowing that cancer is a very commonly cured thing in the Daily Mail.
Remember, this kind of learning works even when the questions are written in a randomised language. It’s real understanding derived from nothing at all except studying raw text.
That’s important because a machine that can learn to answer questions given nothing but words can eventually — if it scales up — learn about the world, and about humanity, by reading books. That’s the next goal for DeepMind, a British AI lab owned by Google that has also done research into story comprehension. And once it’s read the entire contents of Google Books it can go ahead and read a book you wrote just for it: the book that creates its character.
What’s important to understand is that there’s no reason a neural network trained by reading books and backstories would know it is a robot. When it queries its memory with a question like “what am I?” it would retrieve whatever it was trained on. And as books are typically written from the perspective of a human, rather than a robot, that’s the perspective it would access.
Clementine lost in “reveries”, fragments of memories that were supposed to be overwritten but are still accessible, thanks to Arnold.
Memory
Two key plot points in Westworld are about memories:
Hosts begin to access memories that were supposedly erased.
Hosts have photographic memories and cannot tell the difference between recall and reality.
How realistic is this? Perhaps surprisingly the answers are, respectively, “very” and “not at all”.
Let’s tackle the topic of erasing memories first.
Most current progress in AI is coming from advances in the field of neural networks, which are data structures inspired by the brain. If you’ve recently noticed big improvements in the quality of your phone’s speech recognition, or in the quality of Google Translate, you’ve noticed neural networks in action. Don’t take the analogy too literally: neural networks are kind of like brains in the same way that your computer’s files and folders are kind of like the paper things found in an office … the comparison is helpful but not meant to imply exact simulation.
The networks used in speech and image recognition work on something a bit like instinct. After being trained, they’re presented with some data and immediately spit out their best guess at the answer, which is synthesised from the entire contents of the network. There isn’t much that can be called structured reasoning. That limits their performance on many important tasks, if only because as the networks get larger their performance drops. So researchers have started adding an additional component to them: memory.
The memory available to a neural network is very different to regular computer storage, even though the contents are stored in ordinary files. For one, it’s “content addressable”: memories are accessed by querying with something similar to what is wanted. For another, neural memory is not neatly partitioned into human-meaningful files and directories. It’s simply a large set of numbers and the neural network itself decides how to use and combine them. From the DeepMind paper, “Neural Turing Machines”:
We achieved this by defining ‘blurry’ read and write operations that interact to a greater or lesser degree with all the elements in memory (rather than addressing a single element, as in a normal Turing machine or digital computer). The degree of blurriness is determined by an attentional “focus” mechanism that constrains each read and write operation to interact with a small portion of the memory, while ignoring the rest.
So defining where something is stored in a neural memory turns out to be hard: a particular memory could be spread over many locations with some contributing more than others. This poses obvious problems for the task of erasing specific memories whilst leaving others intact. You of course always have the option of a “rollback”, as Dr Ford puts it. In a rollback the entire memory contents are replaced with an earlier snapshot. That’s guaranteed to work. But doing that means the AI also forgets everything else it has learned, including things that may be useful to keep:
“It’s the tiny things that make them seem real, that make the guests fall in love with them.”
— Bernard
Dr Ford and Bernard are faced with a difficult task: they want to erase the host’s memories of their previous time around the narrative loop along with the memories of being shot, raped and kidnapped by the guests. But they want to preserve the memories of new words and phrases, the improved horse riding skills and so on … all the small experiences that lead to improvement in the machine’s realism.
Given the way AI technology is evolving, erasing specific memories won’t be easy, because in a neural network — just like in a real brain — the memories are all linked together in ways that aren’t easily understood by an observer. Believing you’ve successfully erased specific memories and then your AI finding ways to access them anyway is thus an entirely believable plot point.
What about the second idea in the show: that hosts can’t tell the difference between memories and what’s real? That seems a lot less likely. In another 2016 DeepMind paper, “Towards conceptual compression”, the authors introduce an NN based algorithm that works just like our memories do: by throwing out fine details but retaining the concepts. This image compares multiple image compression algorithms: the top row is the original image, the grey row is ordinary JPEG as used on the web (grey/missing because it can’t make files compressed this much), then JPEG2000, and the last two rows are using neural network based compression (in different modes). Each algorithm is given the same amount of space to encode the original image.
As can easily be seen in the fifth column the neural network was able to retain the concept of a bird in front of water even when the advanced JPEG2000 algorithm created a blurry mess, and ordinary JPEG as used on the web gave up entirely. The man looking at the elephant was likewise retained as a series of brush strokes, like what a painter might create. The detail is gone but the important parts remain … just like how our memories fade and details go missing, but the basics remain.
Given the “blurry” nature of neural memory and the universal desire engineers have to utilise resources most effectively, it’s hard to imagine robotic memories retained with so much detail that they are impossible to separate from the reality being captured by the machine’s sensors. Even though we are used to thinking of computers as perfect devices that never lose information, the reality is they do throw away data all the time in order to improve their performance in other areas.
Control
The final part of Westworld’s story worth comparing to real research is about control.
Robots that rebel against their makers are a sci-fi staple and Westworld is no different. Maeve learns to override the “big red button” that’s supposed to stop her in her tracks. But in real life we have robots all over the place: making things in factories, vacuuming homes. None of them ever rebel, nor are they even capable of it. It is hard to see how this concept might become realistic.
Hard … but not impossible.
The neural networks in production today, like the one in your phone that recognises your speech, are all pattern matching networks. They are presented with a single piece of data and provide their best guess as to what it is. Presented with a picture of a cat, they yield “cat”, and that’s it. Even though we find it hard to understand exactly how these networks arrive at their answer, they’re still perfectly safe because they have no agency: no ability to interact with the world or make plans.
However, researchers have recently been putting great effort into developing AIs that can interact with the world and make plans. These new AIs play video games and work very differently to the very simple, entirely script driven programs that populate game worlds today. They can learn to play a new game presented only with the pixels on the screen as input and the controls as outputs. Because playing games designed for humans may involve complex planning and execution, they’re much more similar to the kind of AI that the hosts would need. Indeed, Westworld is in the end nothing but a giant game run for the entertainment of the guests.
Unfortunately, these types of AI have already shown in several incidents that they can go wrong in unexpected ways. It starts with reward gaming. To learn to play a game requires some notion of whether you’re doing well or not. In a video game this usually means maximising the score. But scores are often imperfect proxies for what we really want the AI to do.
Here’s a neural network from the OpenAI project that learned to play CoastRunners, a boat racing game. The AI’s designers wanted it to finish the course so programmed it to maximise its score by collecting powerups and coins from around the track. What could possibly go wrong? Watch:
The problem becomes apparent within seconds:
We assumed the score the player earned would reflect the informal goal of finishing the race. The RL agent finds an isolated lagoon where it can turn in a large circle and repeatedly knock over three targets, timing its movement so as to always knock over the targets just as they repopulate. Despite repeatedly catching on fire, crashing into other boats, and going the wrong way on the track, our agent manages to achieve a higher score using this strategy than is possible by completing the course in the normal way. Our agent achieves a score on average 20 percent higher than that achieved by human players.
In another example a game-playing AI learned that it could get better scores on Tetris by pausing the game indefinitely when it was about to lose. Neither behaviour was desired, but even in the very limited environment of a video game agents can do unexpected and harmful things.
The situation gets more volatile when you realise that games, like all programs, can contain security bugs. Here’s an example of hacking an ordinary copy of Super Mario Bros only via pushing buttons on the gamepad. The running code is replaced with that of a new game:
This sort of “escape hack” isn’t possible for human players because you have to do too many precise actions too quickly, so in the video it’s performed by a separate computer wired up to the gameport. But it’s the sort of thing that theoretically a Super Mario playing AI could stumble upon over millions of training games. If an AI can learn to hack the software controlling its environment, even if that’s very unlikely, we can easily imagine how a strange kind of score-obsessed robot rebellion could follow.
Real AI researchers are taking the issue seriously. In the paper “Safely interruptible agents”, DeepMind and University of Oxford researchers write:
If an agent is operating in real-time under human supervision, now and then it may be necessary for a human operator to press the big red button to prevent the agent from continuing a harmful sequence of actions — harmful either for the agent or for the environment — and lead the agent into a safer situation. However, if the learning agent expects to receive rewards from this sequence, it may learn in the long run to avoid such interruptions, for example by disabling the red button — which is an undesirable outcome.
Put another way, there’s a risk of AIs learning to achieve their assigned task better by preventing humans from shutting them down. This problem is also explored in two papers, “The off switch game” and “Concrete problems in AI safety”.
Westworld doesn’t have much to say on this issue, at least not in season one. In fact the hosts don’t seem to be set any particular goals beyond following their narrative over and over again, which leads to the question of what a suddenly free-willed host would try to do — take over the world? Entertain it? Exact vengeance? Do nothing? Is even asking the question missing the point? Perhaps season two will tell us. But Maeve’s neural networks finding a bug in the regular non-neural software running her brain and then “hacking herself” to disable the shutdown commands is … not entirely implausible.
Conclusion
Westworld has received rave reviews for its intriguing story and intelligent treatment of AI. Whilst it’s in essence a philosophical take on robotics and not really about technology, it’s still interesting to compare the fictional take on robots with what can really be built today.
As I was reading research papers for this article what really struck me was not the capabilities of the latest tech, but the incredible speed with which it’s suddenly advancing. Less than two years passed between the bAbI tests being set and a general AI successfully completing them. The next challenge being tackled is called the Children’s Book Test. How many more years are needed before we’re training neural networks on the contents of entire libraries? Two? Three? It looks plausible that I’ll witness a machine definitively pass the Turing Test within my lifetime.
Machines that manipulate humans by making them think they’re real already exist in primitive forms.
Now might be a good time to read “How To Tell If You’re Talking to a Bot”.
Source: https://blog.plan99.net/the-science-of-westworld-ec624585e47#.5tgzbf85d
The post The science of Westworld – A summary of recent artificial intelligence research appeared first on Integrated Multi-Channel Marketing Services | Multi Award Winners.