Shutterstock / Sean D
Videogames, in general, have one thing in common across genres: They’re supposed to be fun. (Duh.)
But sometimes games aren’t fun, for reasons that are hard to suss out during the development process, leading to an industry averse to creative risks. Gripe all you want about the incremental updates to franchises like Madden or Call of Duty, but by now, EA and Activision know how to zero in on what’s fun for sports and shooter fans.
An engineering experiment currently underway at Stanford University, though, aims to use a videogame controller to automatically determine what the player is feeling — including excitement, boredom, happiness, sadness or fear.
Stanford University (top 2); Eric Johnson From top to bottom: The back and front of Stanford’s experimental controller, and the front of a standard Xbox 360 controller.
In the lab of Stanford electrical engineering professor Gregory Kovacs and Ph.D candidate Corey McCall, sensors in a modified Xbox 360 controller can determine a game player’s pulse or breathing rate. In an interview with Re/code, McCall said a small red light on the controller’s bottom, the pulse oximeter, can both read the heart rate and estimate blood pressure.
“It’s not an actual blood pressure measurement, but it has some components of the blood pressure,” McCall said. “When you get excited, your arteries dilate, and when you get scared they [constrict] a little bit.”
For context, it may help to consider that modern story-driven videogames are often choreographed with exacting detail: In order to get from point A to point C, the player must pass through point B and fight enemies X, Y and Z. The player is free to decide things like timing and technique, but at some point during the game’s development, someone planned that freedom carefully.
Knowing that a player is getting bored at point B, and thus might never reach point C, could be a hugely valuable data point during game testing. Or, if the technology ultimately finds its way into consumers’ hands, it could let game companies design a game that changes itself on the fly if the player isn’t feeling it.
“We imagine the end goal being a game like Left 4 Dead,” McCall said, referring to the Valve zombie game that uses artificial intelligence to create tension and adjust the difficulty level.
“We’re also thinking about some other important applications,” he said. “Anything that’s like this [controller], the sensors basically can be ported to any handheld device, so we’re thinking about operating machinery. Someone’s driving, it’s pretty easy to detect if they’ve fallen asleep at the wheel, or maybe they get road rage.”
Kovacs and McCall are hardly the only ones sniffing around “biofeedback” in games. Valve itself employs an experimental psychologist, Mike Ambinder, who has experimented with measuring players’ sweat.
“It’s not how much you’re sweating, it’s actually the pH content of your sweat, but it’s highly correlated with arousal,” Ambinder said at last year’s NeuroGaming Conference. “We created a mod of a game where you have a time limit to shoot 100 enemies, but if you stay calm, you have plenty of time. If you start getting aroused, the time starts ticking down quicker and quicker.”
Sony also reportedly experimented with a PlayStation 4 controller that measured a player’s sweat with what’s called galvanic skin response. In layman’s terms, that means how easy it is for an electrical current to travel from one point on the skin to another; the sweatier that patch of skin is, the faster the current will travel.
Microsoft already has a biofeedback-ready device in the wild: The Kinect for Xbox One, which can measure a player’s pulse by looking at his face. So far, though, that feature has only been used for fitness and not zombie-killing.
Nintendo has repeatedly tried to make a go of biofeedback, too, first with a Japan-only sensor that adjusted the speed of a Tetris game based on the player’s heart rate. More recently, it announced — and then, after years of delays, cancelled — the Wii Vitality Sensor, which (stop me if you’ve heard this one before) read the player’s heart rate.
“We did various experiments to see what is possible when it was combined with a video game,” Nintendo president Satoru Iwata said at the time. “But as a result, we have not been able to launch it as a commercial product because we could not get it to work as expected and it was a narrower application than we had originally thought.”
(Stay tuned for more on that, though. As part of its bounceback from the troubled launch of Wii U, Nintendo has vaguely promised a new health-oriented device that it insists is not a wearable activity tracker).
There’s still a long road ahead before the Stanford controller tech could see a wide consumer release. In the gaming hardware world, comfort and looks are key, but the experimental controller swaps sleekness for sensors and requires a custom extra-large battery pack to hold the circuit board.
Plus, there’s a lot of testing still to be done — something McCall indicated he’d be glad to get to after Stanford announced his research to the world and threw him into contact with the press earlier this month. He has just recently received the green light to run human trials, stacking data collected by the controller up against traditionally measured brain waves.
“We hope that it goes well,” he said. “If nothing else, no one’s collected a data set as elaborate as we are working on. We’ll be able to tell exactly how the body’s changing when you’re playing videogames.”