What Only Artists Can Teach Us About Technology, Data, and Surveillance
by Jen Graves
Not long ago, during an exclusive reception for donors to the University of Washington's College of Arts & Sciences, a handful of student artists were invited to present their works. When it came time for a piece by Robert Twomey, a PhD candidate in the university's emerging-technology department called DXARTS, one donor responded viciously, openly declaring his art "an abomination."
The art was a machine that makes drawings. The donor was a woman trained in painting and printmaking, a die-hard believer in the importance of the hand of the artist.
"But there's no one behind it!" she exclaimed.
"But I am behind it," said Twomey.
She gave him her name and number and the impression that she'd like to cure him of an affliction. Twomey kept trying to explain that he was a painter at heart, that they were on the same side.
The tension reminds me of the story of a young William Henry Fox Talbot. As legend has it, Talbot was on his honeymoon at Lake Como, in Italy in 1833, when he became frustrated with a camera lucida he was trying to use to draw the landscape. A camera lucida is a stick with a prism on its end, and the prism projects the scene in front of you onto the paper to aid you in sketching it. Talbot got his camera lucida from a friend who had no trouble making great drawings with it, but even using the assistance of a gadget, Talbot's drawings were a "melancholy" sight (his word). He was no artist. That is, he was no artist if you define an artist as someone who can draw. But Talbot has gone down in history as an artist, an important artist, despite his hand-eye coordination issues, because his frustration with the camera lucida inspired him to develop the next wave of tools for making art: tools that fixed images to paper permanently. He developed the technology that became the photographic negative.
Photography came to us out of the "failure" of human hands. After developing the technology, Talbot made the first commercially published book ever to be illustrated with photographs, and titled it The Pencil of Nature, as if photography were even more "natural" than drawing because it was divorced from the intervention of that most human of tools, the hand.
I thought of Talbot the day I first encountered Twomey's drawing contraption, at the Burke Museum, at a public event called The Big Draw. In front of some display cases of bones and fossils, there were three drawing stations. The first drawing station had pencils and paper. The second drawing station was equipped with NeoLucidas, new versions of that optical tool that so annoyed William Henry Fox Talbot. The third drawing station was Twomey's drawing machine, where not only the artist's hand but the whole artist was absent.
People gathered around, full of questions, as the sputtering machine made its low racket. It jumped around like a pet trying to say something that would never quite come across. A mechanized pen was suspended on a rickety pulley system, attached to an easel. A laptop computer on a table near the easel was connected to a camera with a 180-degree circular fish-eye lens, taking one picture of the same view every few minutes. People liked to insert themselves into a picture, but they would appear in the drawing only as streaks, if at all, because the end product was a composite based on many images. On a computer monitor, the code scrolling down the screen was translating digital information from the camera into automated instructions for the pen. On the outside it looked like a Charlie Chaplin–era device, but its innards were 2014: a hacked CNC plotter and custom computer-vision software.
At the end of the day, the drawing—a layering of as much information as the machine could translate and draw from each successive photograph—would prove to be far from informational. It looked glutted but starved, this parallel world filtered through hybrid intelligences. There were areas of eerily unfamiliar precision, the kind that a camera will see but that feel strange to a human eye, while other parts of the drawing showed storms of Etch-A-Sketchy blips, part-captured but ultimately lost details. I had the odd impression while I watched the machine work that people who knew this room at the Burke Museum well, who already had layered memories of it, would recognize it in the drawing, but people like me who'd never been here before would not see much we could identify.
This was not Twomey's drawing machine's first outing. For three months beginning at the end of 2013, it was displayed in a public storefront of Amazon, in South Lake Union, where it drew what it saw—the neighborhood, life going by. A day's worth of images, fed in by a connected camera that took occasional shots, would pile up into a single, webby, staticky time-lapse drawing. To me, the drawing machine's loveliest moments were at night, when the lights inside the window stayed on but the outside world went dark, and the machine could see only its own reflection. At night, the machine drew itself.
Robert Twomey didn't start out behind machines. As a kid, he could draw and paint realistically. He majored in painting at Yale and double-majored in biomedical engineering, but maintained a separation of the two after graduation, working days at a neuroimaging research lab and slinging paint at night in his studio.
Then he heard about an artist developing robotic dogs that could sniff environmental toxins. He contacted her. Natalie Jeremijenko let him help her on her project Feral Robot Dogs, and then funded Twomey as her researcher for his first year of art grad school at the University of San Diego.
It was 2006, and Twomey's grandmother had just been diagnosed with Alzheimer's. He was living far from her, so they would have disjointed, distant conversations. Twomey knew that inventors had created chatbots designed to offer people digital company, and he wondered how interacting with one of those might be analogous to the conversations he was able to have with his grandmother. The piece he made was called Megahal Grandmommy, and for several months, he chatted with "her" every night in private. MegaHAL was an early computer conversation simulator, invented in 1998. It learns words as you talk to it. Twomey talked to Megahal Grandmommy about Alzheimer's, about family members, about his life and his grandmother's past. Later, he published the transcripts of these conversations, and he installed the chatbot in galleries so people could talk with her themselves. The conversations are frustrating to read, erratic and cryptic but with moments of near-connection—hope followed by disappointment. It is clear in them that Twomey doesn't want to give up on either his aging family member or this demented, ambitious technology.
The conversations with the chatbot are frustrating to read, erratic and cryptic but with moments of near connection.
MegaHAL was one more step toward the holy grail of perfect artificial intelligence—that aspiration also represented by the sadistic computer Hal in the 1968 movie 2001, and the seductive operating system in the bittersweet 2013 movie Her. In Her, the love affair between a lonely man and an intelligent software program may have been doomed from the start, but nobody questioned why the company of a computer was better than the torture of loneliness. Technology comes from single-minded inventors, profit-hungry corporations, paternalistic and warring governments, and deep places inside human beings.
Art is supposed to combat loneliness, too. The key question about tech-based art is still the one Roy Ascott asked in 1989: "Is there love in the telematic embrace?" When we say of a work of art, "I didn't get it," we don't mean only that we didn't understand it but literally that we didn't receive it. It didn't cross over to where we are. It left us just as it found us, on our own. All art has to cross this distance, but tech art comes up against the additional bias of our "deep-seated fears of the machine coming to dominate the human will and of a technological formalism erasing human content and values," Ascott wrote. Our technophobia. It's a bias I know I have, founded in my insecurity about my life essentially depending on technological structures that I barely understand. Twomey had a feeling deep down that computer art "was bunk" because he didn't believe in its ability to love.
It took Twomey a long time to think about art as "not just painting and gesture and the handmade." He hasn't come so very far. He still insists on making every part of his works himself—which is a hallmark of DXARTS, his program at UW, a hallmark that would probably surprise the woman who saw Twomey's work as an "abomination" because it didn't directly show the marks of his human hand.
DXARTS is something of a mystery. It's an entire department at the University of Washington, but it seems virtually no one has heard of it. Among people who have heard of it, very few know what goes on there. It's an art school, but it's relatively new, and it's nothing like the UW School of Art + Art History + Design, which is located right next to DXARTS on campus, even though the two have separate faculty, leadership, facilities, and philosophies. DXARTS isn't just a theory school, and it isn't just a studio school. It's not even strictly for visual artists; it's also for musicians.
The machines behind the locked doors at DXARTS, like the machines at Willy Wonka's chocolate factory, make delirious magic. Ultrasonic speakers emit beams of sound that are smart enough to follow listeners around. Other gadgets bestow upon you the ability to touch things that are not there, to feel them with your fingertips even though they're absolutely virtual. There's a fleet of the most powerful computers Apple has ever made, which resemble bitty silver trash cans, plus motion-sensing devices, rapid 3-D printers, and virtual-reality portals. More toys arrive all the time to equip this colony of interdisciplinary supergeeks. Which is why it comes as a surprise to learn that DXARTS is designed to be the most old-fashioned high-tech art school around.
To understand what that means, we need to travel back in time for a minute, to the era of such novelties as the modern flush toilet, the Declaration of Independence, and the widespread acceptance of the scientific method. "What we use is what I call the Artistic Method, following the Scientific Method," says DXARTS founder Richard Karpen, echoing 18th-century neoclassicist painter Sir Joshua Reynolds.
On December 10, 1776, Reynolds announced to assembled students at the British Royal Academy, where he was president: "As our art is not a divine gift, so neither is it a mechanical trade. Its foundations are laid in solid science."
Reynolds's words were published as a rule book, and his ideas came to dominate art education and reception until the modern era. According to him, there was a science to art, a way of producing and quantifying greatness. Paintings and sculptures were assigned value hierarchically, and there were certain ways to make art and other ways not to. This was stifling, and so, eventually, was the bohemian free-for-all that arose in rebellion against it. How should an artist make art in 2014? Few people within the creative realm will venture a black-and-white answer to that question, which vexes some people and reassures others.
But Karpen will. He elaborates on his philosophy at a cafe downtown. On this particular fall afternoon, his next appointment is with Seattle Symphony music director Ludovic Morlot. Symphonic music: talk about a discipline desperately seeking its right place in contemporary culture. That's the world Karpen comes from. Visual-art professors, including now-retired Paul Berger, joined Karpen in the founding of DXARTS, but Karpen was the strongest single force. He's now chair of UW's School of Music and downplays his involvement in DXARTS, but students and professors say he's still a major presence.
"I have my students measuring paintings," Karpen says.
As in, with rulers?
Yes, they're looking for patterns and forms and all manner of detectable phenomena, he says, describing rhythms he's identified that are embedded in time-tested works across disciplines, from Andrei Tarkovsky's poetic films to Willem de Kooning's abstract paintings to Allen Ginsberg's Beat poems. "Over time, it becomes intuition," he says. "You absorb the model. To work as an artist, you're accountable to, say, Beethoven in the same way that a physicist is accountable to Newton. In my classes, we look at five examples each week and break them down."
From the outside, to an artist like Jason Salavon, who's based in Chicago and has had a successful career making computer-generated art since the 1990s, Karpen's quantitative approach has its allure.
"I have sympathy for trying to look at things we've assessed qualitatively in quantitative terms, to make them more than 'I know what I like when I see it,'" Salavon says.
Then again, scientism, as Karpen's approach might be called, feels uncomfortably like an attempt to legitimize art using the standards of another field, Salavon elaborates. But then he adds a complicating thought. Consider this: Using fMRI machines, brain researchers at Oxford University showed that pleasure centers fire when people untrained in art are told that a painting is an authentic rather than a fake Rembrandt. It's not only that people can't distinguish—of course they can't. Lifelong art historians famously disagree about the authenticity of certain Rembrandts. No, what's really fascinating is that the visual parts of those untrained brains respond to the paintings they're told are fake and real with the same activity, meaning their brains process the visuals as equal, but their brains' pleasure and reward centers fire only with the additional outside reassurance that they're looking at an original, rare, expensive Rembrandt. They don't like what they see better, they like what they're told better, and this adds up to an aesthetic preference.
"I'm like, 'Oh my god, the bullshit that surrounds something is real!'" Salavon says. "The marketing and all that? It actually does something to your perceptual apparatus."
What Salavon is saying is that since people can't help but be influenced, it's at least as good to dress up art in scientism as in filthy lucre or centuries of external validation.
Twomey, who's getting his PhD in the department Karpen founded, says, "I absolutely do not believe there's an articulable formula or an exhaustive description of why something is a great piece, but I think trying to wrestle with that is a really awesome project." He takes Karpen's dogma with a grain of salt. Assigned to "take apart" a great work, he chose Goya's late "Black Paintings" on the walls of Goya's home, about which little is known, with some scholars speculating Goya wasn't even the artist. "I looked at the facts: the biography, the size, the technique, the material, their narrative, the question of are they a set, and if anything, it left me charged up with the inscrutability of images," Twomey says.
One of Karpen's quips is that art schools want nothing to do with what's before 1900, and music schools want nothing to do with what's after it. Art schools are stale-bohemian, all about loosey-goosey personal expression and ego, and music schools are stale-Reynolds, all about tradition and convention, he says. He points to the fact that his music colleagues see him as far-out and his contemporary-art counterparts "think I'm a reactionary." Closer to the truth may be that people outside his brainchild, DXARTS, just don't know what to make of it yet.
PhD programs in studio art are mostly unheard of; the MFA is the standard terminal degree. DXARTS has 15 students, all PhD candidates since no lower degree is offered. UW's School of Art, by comparison, has 600 majors in art, art history, and design at the bachelor's and master's levels. To Karpen's way of thinking, artists need just as much training and research time as scientists. It's a matter of professional pride.
"The premise of DXARTS is to admit that art is hard," Karpen says, "and to require the university to admit that."
The state allocates money to UW every two years. In the 2013–2015 budget, DXARTS got $1.96 million. To put that in context, the School of Art got $9.64 million, and the Computer Science & Engineering department got $28.99 million. I tried to talk to somebody at the university about whether DXARTS is succeeding, whether it's achieving what it's supposed to with its money. Robert Stacey, dean of the College of Arts & Sciences at UW, declined multiple requests to be interviewed about DXARTS. You get the feeling that DXARTS is so new—it began in 2004—and esoteric that administrators don't know how to assess it.
But there's every indication that they're pleased.
So is Twomey, even while not swallowing too much of Karpen's Kool-Aid. At DXARTS, Twomey has developed unquestionable expertise, but he puts it in the service of error. He's gone from a kid talented at drawing correctly to a systems whiz who sees the greatest possibilities in "imperfections in the systems." As he explains it: "What's the messy humanity left over on the edges? What does Google know about me, and what will Google never know? What can never be captured by the systems?" His questions ring of surveillance states and leaky agents, of Facebook and Edward Snowden and civil liberties.
The University of Washington isn't the only local institution experimenting with art made with emerging technologies. Microsoft has a whole sector called Microsoft Research, and within Microsoft Research are a bunch of art-minded techies who recently created an artist-in-residence program, which brings actual working artists into Microsoft Research to play around.
The current artist in residence is Jason Salavon, whom I mentioned earlier and who wants to bug Microsoft Research. Like, spy on it.
"I'm gonna have to talk them into it, but I'm making headway," Salavon says over coffee near Paul Allen's Living Computer Museum, where there's a 1984 Macintosh SE on display—the first computer Salavon made art on. Microsoft Research is paying Salavon to be here for seven months, to conduct artistic research of his own alongside the rest of the technical and conceptual research the company sponsors. "The reason this came about is there was never any conversation about specific goals," Salavon says. "I never felt any performance requirements or obligations. So far, I feel incredibly comfortable in the environment."
Microsoft Research is on Microsoft's Redmond campus, but not quite of the Redmond campus. The way one artist put it to me, if you think of Microsoft as a city, Microsoft Research is the university. Like DXARTS, it's an idealistic playground. Microsoft Research is the group of people who brought you Kinect, the camera you can set up in your living room that allows you to control your activity in a video game "naturally," not with a controller but through your own movements and words.
Now the Kinect is being hacked for all kinds of alternative uses. James George, the artist in residence at Microsoft Research before Salavon, used Kinect technology to simulate the sci-fi vision that was supposed to be installed inside New York City's subway systems in the aftermath of the London train bombings of 2005. In response to those attacks, New York paid Lockheed Martin more than $200 million to install a surveillance network intelligent enough to sift through the mass of human activity on public transit and pick up and report potential threats. But the network never worked properly, and when George heard that Lockheed Martin's cameras were sitting idle, he decided to simulate what it would have looked like to see "through the eyes of that machine." George attached a digital photo camera to a Kinect and hooked it up to a battery and a laptop in a backpack. He walked around New York's subway system taking both color and depth images of commuters, then used software to combine them into fragmented 3-D candid snapshots. The final, distorted images remind me of a smart horror film's bad-guy-cam sections—scary and spellbinding. But this particular way of seeing, this obviously alien visual intelligence, was originally meant in this case to make us safer, to be a friend in a world of suicide bombers and would-be criminals.
Only one artist comes to Microsoft Research at a time. And unlike regular Microsoft employees, Microsoft Research employees don't have product quotas. Some don't develop systems or prototypes at all. They write arcane papers on subjects like "Deep Learning of Knowledge Graph Embeddings for Semantic Parsing of Twitter Dialogs." That paper is due any day, along with another Microsoft Research group's paper on "Touchless Interaction in Surgery." Then there are all kinds of experimental prototypes that may yet become products, like the EmotoCouch, a couch that expresses—through lights, patterns, and vibrations—happiness if you put flowers in front of it, excitement during a party, or anger when kids are fighting on it.
Two months into his seven-month residency, Salavon's plan to bug Microsoft Research was partly under way. "We figured out how to kind of grab the whole meeting-room system" for recording, Salavon says. These first surreptitious recordings are only of meetings between people already supportive of Microsoft Research's visiting artists program, but they also don't know they're being recorded. Eventually, Salavon may have to put signs that say something like "Research in Progress" near his recording devices, spread around the building, in the cafeteria, up high in administration. The idea is that he'll hide microphones all over, then take the "bag of words" he collects and send it through a filtering program that will sift and play back the material according to whatever the most sophisticated technology in the world thinks it can tell you about what you—you individually or you collectively—have just said.
"There's a burgeoning area of research called 'sentiment analysis,'" Salavon says, explaining that programs detecting word "mood" and frequency are learning to assign sets of words attributes like "positive" and "negative." Sentiment analysis "has creepy edges," says Salavon. "I tend to get really interested in things I have ambivalent feelings about."
His grand interest, the fascination sustaining him across his career, is the everlasting human desire to know the truth by seeing everything, achieving a God's-eye view. His art, most commonly taking the form of photographic prints, is generated by computer programs he writes that mine masses of data for meaning by crunching and revisualizing them. They survey superhuman—divine? Demonic?—quantities of information, just as Twomey's drawing machine is capable of drawing for days on end without rest. Big Data, the computed amassing and processing of a quantity of information so huge, it was simply impossible before the 21st century—that's Salavon's medium.
"I have always thought it was super-fun to look at a gajillion things in some way, to consider from an artistic point of view a gajillion things," he says. "We're finally capable as a civilization of looking at a gajillion things. That's a 21st-century thing. What are you going to do with that?"
What people tend to do is seek reflections of themselves, or of an order, maybe a governing intelligence. Another project Salavon is considering as part of his Microsoft Research residency involves developing a program capable of scanning millions of photographs of nonhuman subjects, say stains or trees, to find pictures of Jesus or pictures of the person who's doing the looking. There's something goofy there, and narcissistic. "Throughout history, humans will go, 'I think we got it all figured out now, we got all the information we need, and we're going to put it in this big book, or in this encyclopedia,'" says Salavon. "Scientists keep going, over and over again, 'We're only three years away from the whole thing!' It's obviously not going to happen. As organisms, just as ants aren't capable of everything, we're not capable of everything. But the quest is beautiful, and it leaves a lot of beauty and destruction in its wake."
Unlike many artists working in code, Salavon produces fine-art objects that can be displayed and sold through traditional routes. His aggregate images from the early 2000s were especially popular. In 2002, he made Every Playboy Centerfold, The Decades (normalized) by essentially boiling down four decades of Playboy magazine's centerfolds into four single composite images, one for each decade. The final figures are hovering wraiths, blurry averages, but placed next to each other in chronological order—one composite for the 1960s, one for the 1970s, one for the 1980s, one for the 1990s—they also present a clear portrait of what seems like a disturbing trend: The naked ladies get thinner and whiter from 1960 to 2000. Where does that lead us? Is it an average that lies or tells the truth?
Here's another visual set Salavon "averaged": portraits by revered 17th-century painter Frans Hals. Salavon was inspired to find the "hidden norm" in Hals's portraits after visiting the Dutch Golden Age galleries at the Metropolitan Museum of Art in New York. The Met now owns Salavon's 2009 picture Portrait (Hals), and even in its absurdly blurry generality, you might find yourself searching for—and finding?—a nugget of what makes Old Master paintings so loved.
Salavon uses the tools of new digital culture, but his art is based in the ancient fantasy of omniscience, omni meaning "all things" and the root of science being "to know." He grew up in Fort Worth, Texas, the son of a nurse mother and a painter father who worked as a museum guard. Their house was full of "books and books and more books of art history," he says. Today, Salavon, 44, operates in the fine-art world, not in the specifically tech-art world where DXARTS artists exist. "I'm anachronistic," he says. "I like making pictures."
Despite having seen the struggle of the life of a painter up close—his father's existence was "not romantic"—Salavon had to admit in college that while he liked computer science a lot, he loved art. He pursued both. During his master's program in art and technology at the School of the Art Institute of Chicago in the early 1990s, "all the profs had literally zero knowledge of what was going on in the contemporary art world or any emphasis in art history," he says. They knew technology, and tech art. The people he went to grad school with went into industry, and now they're working at Pixar or doing concert graphics for Beyoncé. During school, Salavon also worked 30 hours a week for Viacom's video-game company, specifically on the Beavis and Butt-Head game, and was faced with a dilemma, a question he kept asking himself: "Are you gonna work full-time in video games?"
"I grew up lower-middle-class, and it was more money than I'd ever seen and people thought it was so cool," says Salavon. "I got way more love at a bar saying I worked in video games than saying I was getting my MFA. It wasn't even close. But I was like, I like making this weird art shit more. I decided to double down on thingyness, on making stuff, to focus on things that would be in a space, in gallery spaces, and for me, that turned a corner."
He got a New York gallery, won a prestigious grant, and quit another video-game job, finally becoming, officially, an artist at the turn of the millennium, just before digital art seemed to be blowing up at major museums across the country. Now Salavon is both a successful studio artist (he oversees a studio and employs assistants) and a teacher at the University of Chicago, where he has a double appointment in the Department of Visual Arts and the Computational Institute, an important place in the broader world of cloud- and supercomputing.
"Back in the day, I used to have professors talk about the tech-art ghetto," he says. "Now I'm seeing so many art-physics, art–computer science, art-economics majors. Is there still a hipster bias against technology in the fine-art world? I equate that with video art in the '70s. I go, just give it time, because obviously our civilization is not rejecting this stuff, so the art world won't eventually."
But the dominance of technology does not necessarily mean exponentially more artists working with it. Students like Salavon's—with an aesthetic sense and some tech horsepower—have absolutely no trouble finding jobs outside art. It may be more true in tech than in any other medium that the only tech artists are the ones who really have to do it, who are motivated by something deep down inside themselves. All the external forces encourage them to leave art behind and live the comfortable lives that are easily available.
Salavon is one of those has-to artists. His mind overflows with the news of technological breakthroughs that might lead to works of art, like the announcement in late November that researchers at Stanford and Google have created image-recognition software that can caption photographs. The software's captions are deemed better than their human-written equivalents 25 percent of the time—by humans.
Research into that same image-recognition software is happening at Microsoft Research, Salavon says, and maybe he'll tap into it. He really is precisely in his happy place among the scientists and researchers chasing the outer limits of possibility, which always brings with it a touch of grandiose absurdity. One afternoon, I'm standing in the Microsoft Research office of Asta Roseway, who is one of the founders of the visiting artists program and who happened to be part of the EmotoCouch team. She creates cockamamie, brilliant things in her tiny, dim office, where we're at her standing desk looking at a slide show. The images are of past artworks featured at Studio 99 since 2013. Studio 99 is the name for the makeshift exhibition space of Microsoft Research's lobby, which the visiting artist's team commandeers to put art on the walls, climbing the columns, filling the empty spaces. (Studio 99 is not open to the public, unfortunately; you need an employee key card to enter. They're looking to add a more public space, Roseway says.)
As Roseway talks, I'm distracted by a humanoid robot sliding by outside her door. I interrupt her with a "Wha?" and she explains that Oh, yes, him, he rides around the offices. He is one of several "telepresence robots" at Microsoft Research, and you can use him to have a face-to-face meeting with a researcher in California. His name is John in a Box.
I have no special affection for special effects. I am irritated when, in the art exhibitions at Seattle's annual Bumbershoot festival, something high-tech inevitably happens to me in the name of neat-o, like maybe my shadow is made to stick to a wall, or my presence changes the colors of a room. I still carry with me the unpleasant memory from 2010 of being saddled into a harness sculpture that made me trigger lights that were supposed to simulate water so that I could "connect, melt, and finally expand [myself] through aural dissipation," according to the statement by the artist, who earned her PhD at DXARTS. "May you fathom the ocean," the statement encouraged. I felt like a dog in a three-piece suit.
Being raised as a consumer of surfaces has made me hungry to understand the systems behind them. To my 14-year-old self, the news that the singers of Milli Vanilli were not the ones singing Milli Vanilli's songs did not anger me, it set me free. Their image broke from its source like a hole sliced in the wall revealing the plumbing, and from there, I could crawl all over the place to see what was what. Yeah, I thought, how did radio music get made and beamed to me, and how come I never considered it before?
We talk about tech art in terms of newness, of flashing and blinking, but another way to look at it is that it's about systems. It's art made using systems that just happen to be the ones that we generally know the least about in proportion to the extent that they govern us. Like James George's subway system portraits using Kinect. George grew up in Moscow, Idaho, now lives in Brooklyn, and got connected to Microsoft Research through two people from the separate but related worlds of new-media and mainstream art: his mentor Golan Levin, a leading artist and associate professor of electronic time-based art at Carnegie Mellon University, and Leah Dickerman, a preeminent curator at the Museum of Modern Art, whose brother-in-law heads up product management at Bing, Microsoft's search portal. These are small worlds, really.
As a teenager in Idaho making skateboarding videos and internet animations, George somehow came across the video art and interactive sculptures of Gary Hill, who has lived in Seattle for many years. George decided to attend UW, and he was aiming for DXARTS, but he found DXARTS too rigid and academic—he was, frankly, a little too punk for it—so George switched to computer science, studying art and making it on the side. He's now part of a community of what he'd probably call artists but who might prefer to call themselves code craftspeople. They share everything they make on an open source platform available to anyone. "I'm not part of the Chelsea circle at all," George says, referring to Manhattan's mainstream art galleries.
At Microsoft Research, George worked directly with Bing's image-search team. He'd send them word requests like "black square" or "red circle," and they'd send him back the first several million image results, more than any individual would be able to click through manually. George developed a program that picked up all the images, filtered them, and arranged them into huge prints he hung in the Studio 99 lobby gallery like wallpaper—so you could see at a glance what the world currently thinks various colors and shapes mean. The results were far from literal. What you were seeing on the walls was how culture translates words to pictures. Of course the results would be different today, as the image search database of the internet is ever-changing.
If researchers figure out what technology can do, artists can reveal how and ask what it's for. As an example of this, George describes a project of his involving Bing image search and porn.
"It used to be that if you searched for numbers, you got pictures of numbers, and then when you searched for 18, you just got tons of porn," George says. "They [at Bing] saw this as wrong, as inconsistent with the rest of the numbers"—even though it was dictated by their own algorithm, which is complex but largely driven by what people click on. The system takes the pulse of the crowd the way Salavon's art crunches and revisualizes data. The people had spoken, and "18" meant porn.
"But on the back end, they prefixed it, they added the number," George says. They made it so that when you type "18," you get images of the number 18, like you do with all the other numbers. "It's not out of censorship. They don't censor things. If you search for porn, you get porn. They're looking for consistency in patterns. I was arguing playfully that the actual right answer was porn—because it's a cultural survey."
Later, I follow up with Bing's Andrew Shuman in his Bellevue office. As corporate vice president of Digital Work + Life at Microsoft, and until earlier this year corporate vice president of Bing Development, Shuman is an incredibly high-up guy whom everyone tells you is impossible to actually interview. When you google (sorry, Bing) him, you find his name attached to a whole mess of patents as well as his writings for Slate from the late '90s under the superhero tag "boy developer." Once you meet him, you realize handlers probably protect him because otherwise he'd never get anything done, since he's so much fun to talk to.
I ask about 18.
Shuman says he doesn't remember that precise case, but that Bing makes calls like that all the time, because it's constantly tuning. Really: constantly. An army of around 1,000 human testers employed by Bing is working at all times to judge the validity of image results, to teach the machines to think more like a mass of humans. The judges rate results on a scale of good, neutral, and bad. Those aggregated results are fed into the existing algorithms used to measure relevance, based on factors like text associated with imagery, duplication across sites, or "image understanding" (where the program can visually ID, say, a car or a tree). This ongoing process is called "machine learning," and it's like Twomey trying to train his Grandmommy chatbot, except far more sophisticated—yet still completely imperfect.
As for 18, Shuman guesses the change from showing porn to showing numerals was a response to customer complaints. There's a feedback button on every Bing image search page, through which hundreds of thousands of responses come, from a volunteer pool of judges. Some of the options for complaints are: "Adult or offensive," "Bad relevance," "Lack of freshness." You can click on those. An internal menu just for Bing employees offers more choices.
In other words, what goes on is more fascinating and more frightening than censorship.
"We monetize according to use, not anything else, so we're trying very hard to think about what everyone wants," Shuman explains to me. "We're constantly surprised."
On his great flat-screen computer, mounted to the wall of his office, Shuman pulls up a surprise he's recently discovered: comparing what Bing returns as the most relevant pictures for the search terms "American," "European," "African," and "Russian." On a Bing image search, "American" means pictures of flags and eagles. "European" means flags and maps. "African" means exotic animals. And "Russian" means sexy women. (This is all on the default "Moderate" setting; the other options are "Strict," i.e., kid-friendly, and "Off," i.e., no-holds-barred.) Those results in themselves could yield a doctoral thesis, but throw in other search engines and you get even more information, and confusion. On Google image search, "American" and "European" are roughly the same, but "African" is mostly African people instead of animals, and "Russian" is a mix of sexy women and military men. Now try Pinterest. "American" is flags again, "European" is tourist tips (places to stay, places to eat), "African" is patterned fabrics, and "Russian" is, overwhelmingly, food.
If you accept the truism that the technology we make makes us in return, then we are at the moment a very confused complex organism. Andrew Shuman is the guy you'd go to for answers. The source. He's the one who knows more than anybody about this stuff. He's boy developer!
"What people forget is that it is completely a black box, even to us," Shuman tells me. And it hits me—as I'm sitting in an office that wasn't easy to get into, that I had to pass through two levels of security and a PR staffer to get to, that's located above an outrageously upscale mall where the exclusive shops are Jimmy Choo and Louis Vuitton and Gucci—that image search, which always seemed accessible and basic to me, is yet another kind of closed system I don't comprehend. I am a new kind of scared to discover that Microsoft knows only incrementally more than I do about the self-governing wilderness of increasingly intelligent systems like image search. Who's in charge in technology?
"A big part of what artists are talking about now is that we're in a rapidly changing technological environment where social norms aren't set," George explains, calming me down, making me think artists are on my side. "We can make things visible that are invisible. We can talk about privacy in a post-Snowden world."
Even if artists don't have the answers, "companies want these technologies to appear seamless, and we can peel back the covers."
At an international festival in Berlin called Transmediale at the start of 2014, two artists named Julian Oliver and Danja Vasiliev created an installation simply simulating what the National Security Agency does. They set up in the gallery the same technology the NSA uses to hack into everybody's phones. After their system sent unsolicited text messages to more than 700 devices in the area—messages that said "Welcome to your new NSA partner network IMSI"—the German authorities stepped in and immediately shut down the show.
In tech art, George says, "maybe you're not making something timeless, because sometimes timeliness is more important. It challenges art's long-standing bid for immortality, like performance did. But maybe you have the ability to shape the future in a way, or help people make decisions. The artist can be a guide or a translator to this world."
As to the obvious question—why would Microsoft quietly sponsor meddling artists?—the truth is that tech companies since the 1960s, from Bell Labs to Intel, have funded collaborations with artists. They're great while they last. Who knows how long Microsoft Research will continue sponsoring artists like George and Salavon; the program is fueled by that small team of sincere internal enthusiasts, all of whom I interviewed, and I hope they can keep it up.
Public exhibitions of art made with emerging technology are still rare in Seattle, although there's one piece that's been on display for more than a year. It's next to the entrance of the Henry Art Gallery, at the edge of the University of Washington campus, and when you approach it, insectile cameras capture you. They grab your facial features and scan you against a database of 50,000 other faces to try to guess your age and your gender. In real time, you're projected on a wall of video monitors, and a "matching" caption soon scrolls beneath your face. The caption is a story from the actual Facebook feed of a person whose demographics match yours. This is a surveillance artwork sarcastically called Sanctum; it does not, in fact, provide sanctum, unless you think of sanctum as being singled out, sized up, and told who you are.
On my first visit, Sanctum pegs me as a mom driving kids around. Not bad. I've been recognized and reduced, a familiar unsettling sensation encompassing pressing questions about who I am and how people can tell given the systems we currently have for seeing and being seen. Surveillance 1, privacy 0. Art wins.
But on my second and third visits, Sanctum seems to be under the wildly mistaken impression that I am a black male teenage basketball player. I wait several minutes, more than a non-critic probably would, and finally, the baller disappears and I'm assigned a new identity. My new life caption begins, "My first act of art criticism..." A shiver passing through me quickly dissolves as I realize this is just sheer chance. There is no combination of age and gender that adds up to "art critic." It's as if a stranger happened to guess my full name, which would be eerie but meaningless. In context, it's absurd. Imagine that every mom who drives her kids around is also a male hoops enthusiast and an art critic. Sanctum has gone haywire, in my experience with it. It was designed by the artists James Coupe and Juan Pampin to mimic the sophisticated algorithms that Big Information is gathering every day. Presumably it ought to sharpen my awareness, or at least give me the opportunity to rehearse my emotional reactions to real-life surveillance systems, to provide some useful catharsis. Instead, it has become that amusing sidekick, the malfunctioning machine, a toothless cartoon of the precision-fanged forces I know are really out there target-marketing to me and monitoring my cell-phone calls. Sanctum is a work of art that may have enormous potential—but not a work of art that's worked for me just yet.
Sanctum happens to be the most visible evidence of DXARTS's existence. UW's School of Art graduates usually emerge into the fabric of the city somewhere, in public or commercial galleries, at local contemporary art talks and openings, in scrappy shows at apartments, tattoo parlors, or thrift shops. But the art by DXARTS artists is harder to find. With the exception of the ongoing public installation of Sanctum—made by Pampin, the DXARTS chair, and Coupe, the first DXARTS PhD awarded, in 2009, now a professor in the program—where's all the art by the DXARTS artists?
The fact is, DXARTS artists, and many artists working with emerging technologies, operate primarily outside Seattle. Though based in Seattle, these artists tend to send their art far away to show it. That world does not boomerang back to Seattle, which is a problem for several reasons. One, it makes you wonder, since there is hardly any way to know: Is the tech art coming out of Seattle and DXARTS worth a damn? Two, should we care about it? More importantly, these devices and systems are not just any artistic materials—these are the technologies shaping our lives, yet about which we know dangerously little. Could this missing art, if we had the chance to interact with it, guide us to pull back corporate and government technocratic veils and know a thing or two? Is it too much to wonder whether this art could actually make a tech-happy city also a tech-savvy one?
Granted, DXARTS is just beginning to come out of its shell. It has the go-ahead to hire a new faculty artist this spring. Today it has three full faculty members, four research scientists, and a visiting professor and a visiting artist, in addition to the student body of 15 PhD candidates. Almost all the faculty are men, though many of the students are women; the male chairs say their next hire needs to be a woman.
Technology is perennially sexy to funders. It's hard-edged and flashy. UW biophysicist Eberhard Fetz told me he likens artists to scientists in that "they're both explorers and experimenters." He co-taught a cross-disciplinary DXARTS class last year called Art and the Brain. It resulted in a series of student proposals that were so compelling that DXARTS won UW's new Bergstrom Award for Art and Science, for the purpose of creating an ongoing Art and the Brain Lab. DXARTS asked for $9,000. They got $20,000. Tech sells, even when it's not trying.
But DXARTS could just as easily have not happened at all. If it weren't for the go-go economic moment at the turn of the millennium, Karpen's salesmanship, and the idiosyncratic enthusiasm of a UW administrator named Debra Friedman, who green-lit the first iteration of DXARTS and who died earlier this year, new-media art at UW might be the purview of one professor at the School of Art, rather than a full-blown, stocked-to-the-nines independent doctoral program. (They're in the process of developing a master's program, too.) "But if we asked now," Karpen says with a smile on his face, eyebrows raised, "the answer would be no."
DXARTS involves some very accomplished people and fascinating ideas. It also skates on luck, perception, and expensive machinery. Yet given Seattle's tech infrastructure, it's pretty stunning that the city has no place devoted to showing this art, or more support for a scene in general. The gentrification of the central urban core—led by Amazon—has killed off experimental arts nonprofits like 911 Media Arts. Every place that's a tech-art hub has its reasons. There's an incredible electronic culture center, for instance, in Linz, Austria, a town with a desire not to be known for what it's already known for being: Hitler's hometown. In booming Asian cities like Seoul, proud native companies like Samsung throw money at new-media art. "There are hundreds of places [in Seoul]—museums, galleries on floors in towers, it's amazing," says Pampin, the chair of DXARTS, with great longing.
All three main art museums in Seattle show contemporary art, but if they can't handle new-media art, they can't really be called "contemporary," insists DXARTS professor James Coupe, his indignation escaping the surface of his dry, calm British cadence. Digital arts, he believes, shouldn't have to be segregated. He has a point. The Henry owns Coupe and Pampin's installation Sanctum, but the Henry doesn't know how to fix it if it breaks, or update it if the systems it uses morph technically or otherwise—if, say, Facebook shuts down and is replaced by other social-media equivalents. The only maintenance and conservation plan for Sanctum, Pampin says, is "call us."
Sales and conservation are both major problems for the fleeting products of these artists. Seattle artist Claude Zervas shows at Seattle's leading mainstream contemporary gallery, Greg Kucera, and he doesn't necessarily advertise the fact that he used to work as a computer programmer and knows code, but he's a natural ally for DXARTS—yet he's perplexed by some of the ways the department goes about tech art. In general in high-tech art, he believes there's too much geekery, that "the equipment gets in the way, becomes more of the focus than making something good." Zervas says he finds UW's whole idea of a separate digital-arts department "kind of strange," though he's supportive of many of its artists and recently gave a talk at the DXARTS lab. He spoke of being forced to stop using high-tech parts in his sculptures because he was constantly being asked to replace them himself. Museums and galleries and collectors just aren't equipped, he says, echoing Coupe. Now Zervas is making paintings on paper, using a mechanical arm he built and programmed that he follows along with his real arm.
With Zervas's dilemma and the death-by-neglect of Sanctum on my mind, I met with Julia Fryett, a young, ambitious, and serious art presenter who recently returned to Seattle with a mean network of art contacts, thanks to working for a prominent New York collecting family.
Fryett is originally from Pullman, Washington, and in Seattle, she says, "I have no interest in starting a traditional gallery." She firmly believes that technology is the most important medium artists are using, and she has plans in the next year to open a space for it in Seattle. Otherwise, she says simply, "people won't know it ever existed here."
Separate spaces for separate mediums is a reflection of the failure of existing institutions to keep up with the times. Scholars debate whether art made with new media is a category shift of a kind we've never seen before, or simply where photography or video once were, marginalized outsiders that would eventually be absorbed. "But I've been saying that for 20 years," waiting for the mainstream acceptance that hasn't happened, Ed Shanken tells me. Shanken wrote the textbook Art and Electronic Media, and is a fount of knowledge and connections, and inside DXARTS, where he has taught history and theory since last year, he is a one-man army fighting to bridge the worlds of new media and mainstream contemporary art. In my conversation with DXARTS founder Karpen, he came right out and says he's "skeptical" of theorists teaching artists—which is exactly what Shanken is. But it would be a shame to lose Shanken at DXARTS. He opens up the place. And while Karpen, Pampin, and Coupe say PhD students enter DXARTS full of background, Shanken says he's been stunned to discover classes of 500-level DXARTS students with little knowledge of the art of the last century, either mainstream or the stuff in Art and Electronic Media.
Meghan Trainor is another DXARTS PhD student, a formidable maker and thinker. Before coming to DXARTS, she got her master's from the Interactive Telecommunications program in New York University's art school. "I want to go down to the 10,000-foot depths," she says, and that's what she's doing. Her desk at the DXARTS lab is strewn with robotic parts and delicate 3-D-printed lattices, and these all are experimental prototypes for her sculptures—which are controlled by her thoughts through a neuroprosthetic device she trains. The technology is imperfect, but so is her mind. Her thoughts have turned out to be the hardest variable to control, and the sculptures jerk and twist in responsive loop-de-loops of trial and error.
"We are not joking," Trainor says, describing what distinguishes DXARTS artists. "We are not joking around."
There is a grandness to the mind-set at DXARTS that may not be so good for the art itself, that may keep it in hothouse environments. I don't want to single out Trainor—she's accessible and active in Seattle as a speaker on social justice and technology, even if we don't get to see very much of her work. Is this grandiose attitude because a PhD program is naturally more cloistered? There's a tradition in art where movements get seen first plugging away in scrappy places like tattoo parlors and cafes and lofts. That's not what DXARTS students are doing. DXARTS students are the equivalent of musicians working in super-deluxe recording studios. No matter your surroundings, it's healthy to play in some dive bars, too. Some love is generated there.
Roy Ascott's answer to his own question "Is there love in the telematic embrace?" was, to paraphrase: It depends on the interface. Interface is a fancy word, but in the case of a city, interface also means where you show your work, how you show it, who you show it to, and what's your attitude about putting it out there. Museums ought to be showing the best of this work, but artists—please get out there. Put yourselves in places where you risk being misunderstood, where someone might say, as the woman did to Robert Twomey about his drawing machine, "No one is behind this," so you can stand up and say, "I am behind it."
Art gets recognized as art by people standing up and saying, as many times as they have to, I made this, and it matters.
According to Twomey, who gave me a tour around DXARTS's warehouse lab in Ballard, they're considering turning one part of the warehouse into an exhibition venue. Here's what I saw happening in that room recently: An artist was sitting at a computer station, typing code that controlled, in real time, an installation by another DXARTS artist on exhibit at that moment. In Hong Kong.
[ Comment on this story ]
[ Subscribe to the comments on this story ]