On January 11, 2013, James Heilman, an emergency-room physician and one of Wikipedia’s most prolific medical editors, was standing watch over the online encyclopedia’s entry for a back procedure called a kyphoplasty. The page originally suggested that the procedure’s effectiveness was “controversial,” and an unidentified Wikipedia user had proposed changing the text to “well documented and studied”—a characterization that Heilman thought wasn’t supported by existing research. He rejected the change.
Kyphoplasty, along with vertebroplasty, the procedure it shares a Wikipedia page with, is a common treatment when someone’s spine breaks—a frequent occurrence in people with osteoporosis, which makes bones brittle—and then doesn’t heal naturally. The procedure is meant to reduce the pain of a fracture, even though it sounds unpleasant: It consists of inflating a tiny plastic balloon near the fracture, removing the balloon, and then injecting a toothpaste-like plastic cement into the resulting crevice and letting it harden.
The procedure grew popular in the ‘90s, despite the fact that its effectiveness wasn’t backed up by definitively convincing research. By the time two studies published in 2009 found that vertebroplasty—and, by extension, kyphoplasty, which is similar but has not been tested in controlled experiments—was no more effective than a placebo treatment, at least 100,000 of the two procedures were being performed every year. (It’s hard to say an exact number, as the procedures are not recorded in any national database.) In 2011, Medicare paid out around $1 billion for vertebroplasties and kyphoplasties, and the number of the procedures performed each year is not estimated to have decreased significantly since then.
Some are concerned about the money being spent on a procedure that’s controversial and sometimes risky. “To my mind, [kyphoplasty] is an unproven modality and based upon current evidence would have to say it works as well as vertebroplasty, which is to say likely to work as well as a placebo,” says Rachelle Buchbinder, a professor of epidemiology and preventive medicine at Australia’s Monash University, as well as a co-author of a recent vertebroplasty review published by the Cochrane Collaboration, a network of independent researchers. She notes that in Australia, where she lives, public funding for the procedures was withdrawn after the two 2009 studies were published. “From my perspective there is no longer any dispute,” she says.
There are experts who disagree. Sean Tutton is a professor of radiology at the Medical College of Wisconsin, and spoke to me on behalf of the Society of Interventional Radiology, which put out a position paper with other medical societies that called vertebroplasties safe and effective under the right circumstances. “If my mother had a vertebral-compression fracture and after several weeks of conservative management with bed rest, plus or minus bracing, and appropriate pain management, if she still was having ongoing pain and disability, I would treat her,” he says. “I wouldn't even think twice.”
As James Heilman thought more about the attempted edit to the page for vertebroplasty and kyphoplasty, he grew curious about who might be trying to write over the controversy of the procedures, so he Googled the would-be editor’s Wikipedia username. Sifting through the results, he saw that a man named Kim Schelble had an email address that contained the same nickname. Schelble, Heilman found, was employed by Medtronic, a company that sells medical devices used for, among other things, kyphoplasties.
There is little evidence to suggest that kyphoplasties are any better than vertebroplasties, but to a medical-device manufacturer, there’s an important distinction: A kyphoplasty kit sells for thousands of dollars more than a vertebroplasty kit, which generally costs a few hundred dollars. Medtronic doesn’t supply the latter, but it spent nearly $4 billion in 2007 to purchase a company that makes some of the products included in a kyphoplasty kit.
“Their concern is that those at Medicare might read the Wikipedia article,” Heilman says. “If I go to Google, and I put in ‘percutaneous vertebroplasty,’ the first page that comes up is Wikipedia.” Is it really a concern that some high-level decision maker at Medicare or a hospital system might be making billion-dollar decisions based on information from Wikipedia? “Yes,” Heilman insisted. “Definitely.” Indeed, Schelble at one point complained in an email to Heilman, “This site and the content on here is scaring prospective patients and insurance companies are not wanting to cover these procedures.”
(Schelble, who no longer works for Medtronic, didn’t respond to multiple requests to comment for this article, and Medtronic would not comment on the actions of a specific employee. However, communications between Heilman and Schelble did later confirm that it was Schelble who tried to make the edit.)
Schelble’s concern that a Wikipedia article was hurting his company’s business is a common one—the site has enormous reach, and the information it contains makes its way to nearly everyone, from consumers to policymakers to people Googling innocuous questions on their phones. Even minor changes in wording have the potential to influence public perception and, naturally, how millions of dollars are spent. What this means for marketers is that Wikipedia is yet another place to establish an online presence. But what this means for Wikipedia is much more complicated: How can a site run by volunteers inoculate itself against well-funded PR efforts? And how can those volunteers distinguish between information that’s trustworthy and information that’s suspect?
Soon, Heilman found himself rejecting other changes to the page for vertebroplasty and kyphoplasty. After the word “controversial,” a user who Heilman says was most likely Schelble, tried to add “among some but not among the actual physicians who perform these procedures.” The site’s Talk page for the procedures, where proposed edits are discussed, took on a bitter tone. “[Heilman] clearly cares more for his cyber reputation than producing reputable content … Patients who find their way to this page will not be shown the best and current findings on the topic,” wrote the user Clay1500. That handle belongs to Clay Schwabe, another Medtronic employee. “After...seeing how grossly selective Mr. Heilman was being in what data he chose to editorialize on the page, I did write that post,” Schwabe told me, but he insists that he did not contribute to the Talk page in his capacity as a Medtronic employee.
Shortly after these proposed edits and comments came in, Heilman received an email from Douglas Beall, a radiologist in Oklahoma, encouraging him to reconsider the importance the Wikipedia page placed on those two 2009 studies. This might have seemed like a good-spirited scientific discussion between two M.D.s if it weren’t for two things.
One was that Beall has been consulting for Medtronic since 2005 (among several other companies), and disclosures indicate that he has received $150,000 or more from the company between 2012 and 2014. The other thing was that Beall had cc’d about 30 others, a group that he referred to informally as “the North American experts on Vertebral Augmentation.” Heilman said he felt intimidated; included on the list of recipients were one of his medical-school professors and a Vancouver doctor Heilman had in the past referred patients to. By the time the email thread had finally gone quiet, almost 300 addresses had been cc’d.
To be clear, there is no reason to believe that Beall was coordinating with any Medtronic employees, and his numerous consulting gigs with other companies suggest that he is not some sort of kyphoplasty huckster in Medtronic’s thrall. Still, says Sohail Mirza, a spine surgeon and the former chair of orthopaedics at Dartmouth’s Geisel School of Medicine, doctors can grow to prefer the devices of certain companies over others as a result of such consulting arrangements. In the operating room, the manufacturer’s presence can be more than some abstract notion: “Often the [sales] reps are in the operating room for all of these procedures … guiding the surgical team: ‘This is next’; ‘Here’s how you mix this,’” says Mirza.
Mirza, who is himself not convinced of the procedures’ effectiveness, adds that another plausible reason physicians might prefer vertebroplasties and kyphoplasties is that they believe they’re better at selecting patients, and thus think that they can personally beat the results of randomized, controlled trials. “I think radiologists and surgeons generally believe that their patients do better. That's the only way I can imagine them rationalizing it,” he says.
Beall, for his part, acknowledges that he has “financial ties and research ties to many medical-device and pharmaceutical companies,” but points out that the overwhelming majority of high-quality medical literature on vertebroplasty has been funded by medical-device manufacturers. An economic reality of modern medical research is that rigorous studies can cost millions of dollars, and large companies are often the only ones willing to foot the bill.
A doctor named Lawrence Epstein responded to the email thread lamenting this fact. “The problem we have [is] that the majority of the studies that support Kyphoplasty (even when they are well done) are authored by investigators who receive compensation from, or the studies are funded by, Medtronic,” he wrote. “Many of those who read these … are sensitive to this and take even the most convincing data with a grain of salt. What we need are non-industry funded, completely independent studies.” To which Beall responded: “I believe in critiquing the study itself rather than focusing primarily on the sponsor.” He lamented that this sponsor was typically the medical-device industry, but concluded, “Until unbiased and independent funding becomes more available this is the situation that we will all have to live with.”
Through all this, Heilman kept most of the Wikipedia page intact, but it is easy to imagine a different result if the page’s custodian had been less dogged or less allergic to corporations. By now, Heilman doesn’t find companies' keenness to learn more about Wikipedia unexpected. He has received emails on behalf of companies such as IMS Health, GlaxoSmithKline, and Alexion Pharmaceuticals requesting more information from him about the editing process for Wikipedia’s medical content. “I do not consider the goals of the pharmaceutical companies to be educating people about pharmaceuticals,” he says. Still, he says, “Medtronic has been the worst company that I have encountered with respect to aggressively editing Wikipedia to promote their products.”
“The effectiveness of vertebroplasty,” Wikipedia currently reads, “is disputed.”
* * *
In 2006, Jimmy Wales, Wikimedia’s most public-facing board member, reportedly said that undisclosed paid editing—trying to alter the content of Wikipedia without revealing a financial conflict of interest—is “antithetical” to the site’s aims. The practice continued at a low hum over the rest of the decade, but a few years ago Wikimedia started hearing from its volunteer editorial corps that weeding out undisclosed paid edits was distracting from more substantive work. "They were spending a tremendous amount of their time patrolling articles, particularly articles about celebrities or individuals or companies for PR-type editing,” says Katherine Maher, a spokesperson for Wikimedia. The issue took on a sense of urgency in the fall of 2013, when a firm called Wiki-PR was banned from the site for using hundreds of dummy accounts to fabricate widespread support for pages that flattered its clientele.
To combat activity like this, Wikimedia amended its terms of use last summer to ban any undisclosed paid editing that might carry a conflict of interest. (To clarify, the scenario of a political-science professor, who’s paid to think about political science, forgoing disclosure when she edits a page about Japanese elections doesn’t present a conflict of interest and is thus kosher.)
Maher says the terms-of-use change has been received well by those in the Wikipedia community devoted to stamping out paid editing. Perhaps it has—several large PR firms pledged not to violate Wikipedia’s rules—but the practice hasn’t disappeared entirely. Two months ago, an investigation revealed that even after the rule change, employees of Sunshine Sachs, a public-relations firm, had still been editing the Wikipedia pages of their clients without disclosing their affiliation. One email sent by the company boasted, “Sunshine Sachs has a number of experienced editors on staff that have established profiles on Wikipedia. The changes we make to existing pages are rarely challenged.” Sunshine Sachs is reported to have scrubbed, among other things, Wikipedia’s references to Naomi Campbell’s critically-panned R&B album and Mia Farrow’s Ecuadorian activist efforts.
Many people who work within companies’ public-relations departments are inexperienced in the ways of Wikipedia, and some firms look outside of their ranks for editing help. Priceline, for example, once hired a third party to edit Wikipedia on its behalf; Viacom reportedly did the same. Free online guides for outsmarting Wikipedia’s gatekeepers proliferate, but those can only get a novice so far.
"Wikipedia writing is like no other writing,” says Mike Wood, a freelancer who makes a living editing Wikipedia pages for clients, referring to the site’s tireless pursuit of a neutral tone. Wood has set up his own website, and scores of other Wikipedia editors-for-hire await on freelance websites such as Elance. He says he works with highly visible people and companies, who pay him anywhere from $400 to $1,000 per article, but he won’t name names, for fear that someone might seek out and dismantle the Wikipedia pages of his clients. “You could turn on either Fox or CNN right now, and within one half hour you will … see a commercial for [a company or an interview with someone whose] page that I’ve created, or I’ve edited,” he claims. He says that he’s worked with “one of the world’s fifth largest banks” and members of “numerous presidents’ administrations.”
Wood started editing Wikipedia pages about seven years ago. “Wikipedia actually becomes addicting after a while,” he says. “You’ll see people on there all day long. It’s kind of like anyone who wants to play Warcraft or Candy Crush.” The novelty faded for him, though, and he spent a while seeking out freelance writing opportunities. But as he looked around, he noticed a growing number of ads looking for people to edit Wikipedia pages. “It was good money,” he says. In 2010, he returned to contributing regularly to the site, this time as a paid editor.
During his hiatus, Wood says, the tenor of the site had changed. Veteran editors used to patiently help out new ones. “Now, if you’re brand new to the site, and you make a mistake, you’re going to get jumped on by editors very quickly,” he says.
What changed in his absence, Wood says, is that employees of public-relations firms began to understand the value of a Wikipedia page, and tried going in to make edits themselves, with little regard for the site’s standards. The result was that the burden of proof became even heavier on newcomers, and, Wood says, even valid information was getting rejected out of hand by seasoned editors. Those PR companies are now some of Wood’s clients. “They contracted me for their Wikipedia work because some of their writers are so in tune to writing PR pieces that they can't handle writing for Wikipedia,” he says.
“We’ve had some good editors who’ve done really good work begin to offer their services for sale because this is a decent way of making a living,” says James Heilman. After looking into what’s on offer at Elance, he’s concluded that lots of money changes hands there over Wikipedia edits. Heilman says he’s even come across an Elance posting by a Wikipedia editor with the title of “administrator,” an upper-echelon status that comes with exclusive powers on the site and currently belongs only to about 600 active users. (Heilman says Elance has taken down one paid editor’s posting at his request, on the grounds that this editor is violating Wikimedia’s terms of use, but many postings remain.)
The task falls to Wikipedia’s volunteer editors to detect and reject changes made by editors with undisclosed conflicts of interest. The site has several tens of thousands of volunteer editors who update the site regularly each month, and this would seem like enough to head off any biased edits.
But many active editors are there to help with pages about subjects that they’re passionate about, not to spend their time parsing and eliminating PR-speak. And on top of that, the ranks of volunteer editors are dwindling, leaving fewer and fewer people to maintain a growing site. The authors of a study published in American Behavioral Scientist in 2012 concluded that the number of active Wikipedia contributors has been declining because the site’s community isn’t welcoming enough to new editors. “For somebody who’s making their first edit,” acknowledges Katherine Maher, “we could do more to make it clearer as to what constitutes an edit that is in good faith, that is not a conflict-of-interest edit.”
Heilman, too, has come to a similar conclusion after conducting a study of his own, which was published this spring in the Journal of Medical Internet Research. He and his co-author, Andrew West, found that between 2008 and 2013, the number of Wikipedia editors who focused on medical topics decreased by 40 percent. Undisclosed paid edits, Heilman says, “often distract the core community of editors away from more important topics.”
There is also a fear that editing will wane as a larger and larger percentage of Wikipedia’s users access the site from phones and tablets. The site, Andrew Lih, a professor of journalism at American University, wrote in The New York Times, “has always depended on contributors hunched over keyboards searching references, discussing changes and writing articles using a special markup code.” Mobile devices simply aren’t conducive to those activities.
Mike Wood also has doubts about the Wikipedia community’s resources for fighting paid editing. Since the terms of use were updated last summer, he’s seen more Wikipedia editing jobs posted on Elance. For this reason, he likes the policy—in that it has created business for him. “There are so many rules, so many guidelines, that it's made it near impossible to edit Wikipedia without having issues,” Wood says. Last summer’s policy change was yet another rule for new users to wrap their minds around, and newbies, even those acting in good faith, have trouble gaining the blessing of a Wikipedia veteran. So they give up. “The next search they do is ‘help editing Wikipedia,’” Wood says. “And guess what comes up? My website.”
But Wood himself, who makes his living editing Wikipedia articles on behalf of companies and individuals, doesn’t adhere to the policy—he won’t disclose his conflict of interest when he edits pages for clients. Gregory Kohs, whose own Wikipedia-editing business was reportedly denounced by Jimmy Wales in 2006, also declines to acknowledge when he’s writing for pay.
Wood and Kohs have determined themselves exempt for the same reason: They don’t think Wikimedia follows its own rules. “As soon as Jimmy Wales adheres to Wikipedia guidelines, I will adhere to Wikipedia guidelines,” Wood says. Wood is referring to many alleged hypocritical acts, but the most notable is when Wales was reported to have edited his own Wikipedia page, designating himself “the founder of Wikipedia” and attempting to erase the academic Larry Sanger’s role in the development of the site. Kohs refers to this event as “when Jimmy Wales reneged on our public compromise.”
* * *
All of this is troubling only if one sincerely believes that the information on Wikipedia is read at face value. No high school teacher would (knowingly) accept it as a source, and Wikipedians are fond of saying that research can start on Wikipedia, but it should never end there.
But the way people answer their everyday questions today means that a lot of research does end on Wikipedia. The site’s pages are regularly among the top links that search engines turn up—among the general public, the site’s medical articles are estimated to have a larger readership than WebMD. Google has even started embedding excerpts from Wikipedia pages alongside its search results. Wikipedia isn’t just the final destination of typical denizens of the Internet; sometimes it’s where professional researchers end up as well. Fifty to 70 percent of physicians have been found to consult it as a source of medical information—a testament to its reliability.
In fact, the site’s content can make its way into even trusted academic texts, as a recent case of plagiarism demonstrates.
In October of last year, James Heilman was paging through a copy of The Oxford Textbook of Zoonoses, put out by Oxford University Press. Because western Africa’s public-health crisis had been in the news, Heilman was focusing on chapter 31, “Marburg and Ebola viruses,” written by Graham Lloyd, who works at the British government-research facility Porton Down, when he noticed that some text looked familiar.
He brought up the Wikipedia page for “Ebola virus disease,” and grew troubled. Some of the text on Wikipedia looked eerily similar to the text in the book—a lazy Wikipedia editor had copied from the Oxford textbook, he guessed.
He pulled up the page’s revision history to identify the offending editor. He saw that the section was co-written in 2006 by two users, Rhys and ChyranandChloe, and that it was updated in 2010. He jumped back over to the textbook and saw that it was published in 2011. It turns out Heilman had it backwards: It was someone on Oxford’s side, perhaps Lloyd, who had taken from Wikipedia. (Lloyd did not respond to multiple requests for comment.)
The plagiarism was barely concealed. For reasons that remain unclear, some of the original citations from the Wikipedia page dropped out, and were replaced by pointers to other articles—articles that didn’t appear to support the claims made in the text, according to Heilman. In that sense, the Oxford textbook did not simply contain plagiarized text from Wikipedia; it appeared to make it less reliable.
Medical Textbook vs. Wikipedia: A Side-by-Side Comparison
Christian Purdy, a spokesperson for Oxford University Press, acknowledged that some text was copied, but says this didn’t qualify as plagiarism. Instead, he called it an “inadvertent omission of an appropriate attribution.”
“That our content was able to pass … review at OUP is another indication of our quality,” says Heilman. “I think the fact that world experts feel okay with ‘copy and pasting’ from Wikipedia and claiming it as their own is a statement of just how good some parts of Wikipedia have become.”
Heilman had hoped that Oxford University Press would release its textbook with updated attribution into the Creative Commons, as is permitted under Wikipedia’s license, but the press has decided instead to rewrite the relevant section itself. Whatever the outcome, the provenance of the textbook’s information does not bode well for the sourcing of the average AP English essay.
* * *
Because an undisclosed paid edit that goes through is undetectable, it is hard to empirically assess the effectiveness of Wikipedia’s responses to conflict-of-interest editing over the years. Nowadays, the estimated prevalence of paid editing changes depending on whom you ask. “The site itself is so massive that when you talk about problems, they actually tend to be quite small compared to the overall body of work,” Maher says. She points out that Wiki-PR, the furthest-reaching paid-editing operation yet discovered, only made a few thousand edits. Still, undisclosed paid editing is enough of a fly in the ointment to prompt the Wikimedia Foundation to say it “affects the neutrality and reliability of Wikipedia.”
Joseph Reagle, a professor of communication studies at Northeastern University and the author of Good Faith Collaboration: The Culture of Wikipedia, tends to agree with Maher’s assessment. “If we were to enumerate the list of the things that would cause people to be skeptical of the quality of Wikipedia … I suspect paid contribution would be relatively low,” he says. "It's not an issue where there's none, or there's a lot. It's just one of those things where, probably, everyone speeds a little bit.”
Those who consult on editing Wikipedia frame things a little differently. “Undisclosed paid editing, especially on the part of the largest PR firms, is rampant on Wikipedia,” says Patrick Taylor, the head of communications for Wiki-PR, which has, after being banned, refashioned itself as a Wikipedia consultancy. However, Taylor is convinced that most of these edits actually improve the site, and that conflicts of interest are rooted out fairly efficiently.
In a way, undisclosed paid edits are just a smaller instance of a much more foundational problem for a site that strives for unalloyed “neutrality.” All Wikipedia editors, whether volunteer or paid, come to their keyboards with some kind of bias. The presence of money in this equation is never a reliable indication that some information is untrustworthy, since it’s frequently the case that the people who feel they have the biggest stake in promoting their views on Wikipedia are often the best informed. Douglas Beall might receive money from medical-device manufacturers, but part of the reason they’re paying him in the first place is because he’s an expert on certain medical devices. Moreover, plenty of people hold views for which they receive no compensation that would nevertheless render them inadequate editors. For example, a volunteer Greenpeace activist might not be the most impartial steward of a page about the coal industry. Money is but one limited signifier of information’s quality.
These questions are as salient as ever now that Wikipedia has become not a place to go for information, but the place to go. "Many people do not consider other people to be intelligent enough to use Wikipedia with a grain of salt, yet they consider themselves to be intelligent enough to use Wikipedia properly,” Heilman observes. To rely on Wikipedia without any skepticism is to act as though every editor is as relentless, principled, and stubborn as he is.
This article was originally published at http://www.theatlantic.com/business/archive/2015/08/wikipedia-editors-for-pay/393926/