What does it take to keep online communities going? With over 550,000 public subreddits, many of which are active, the communities on the site rely on ongoing effort by a large number of volunteer moderators. In my research, I’ve made the case that caring for the communities we’re part of is an important kind of digital citizenship. For that reason, I’m excited to learn more from redditors about how they see the work of moderation, why they do it, and what is/isn’t their job.
About this Research Project
Ethics: Who’s This For, What am I Recording, and What am I Sharing?
Why Do Research With Redditors?
How Am I Going About This Research?
This spring, I’ve been reading extensively about digital labor and citizenship online, including the story of over 30,000 AOL community leaders who facilitated online communities in the 90s. With Reddit pushing for profitability and promising new policies on online harassment, I thought that potential tensions arising this summer might offer an important lens into the work of moderators, at a time when listening to mods and recognizing their work would be especially important. “The summer is likely to include substantial discussion and introspection on the nature and boundaries of moderation work on Reddit,” I wrote in my proposal mid-May.
Despite this hunch, I didn’t anticipate that Reddit would ban a set of subreddits and mods in their attempt to carry out their new policies, or that some redditors would vigorously oppose this move. The controversy has convinced me that this research could be especially valuable right now. Press coverage is likely to focus primarily on the controversy, while I can carry out a summer-long project, in conversation with a wider sample of redditors than just those associated with this controversy.
In this post (which I will be sharing with redditors when I ask permission to speak with them) I outline my research to understand how Reddit’s moderators see and define what they do. This blog post includes details of the research, the promises I make to redditors, and the wider reasons for this project.
About This Research Project
I’m a PhD student at the MIT Media Lab / Center for Civic Media and a fellow at Harvard’s Berkman Center for Internet and Society where I research civic life online. As a PhD intern at Microsoft Research, I get to be supported this summer by amazing researchers including Tarleton Gillespie, Mary Gray, and Nancy Baym, who are advising this project.To learn more about my work you can read my MIT blog or check out my portfolio.
Over the next few months, I’ll be:
Hanging out in moderator subreddits like needamod, modhelp, and others to learn more about how mods find opportunities, learn the ropes, and discuss their work
Posting questions to some subreddits, after seeking permission from the mods, asking questions or getting feedback on my working understandings
Collecting basic summary statistics across Reddit, from public information, to understand, on average, how many mods there are (like the above chart) and what kinds of rules different subreddits have.
(potentially) interviewing reddit mods
(potentially) trying my hand as a moderator
Ethics: Who’s This For, What am I Recording, and What am I Sharing?
My summer project is being done at Microsoft Research’s Social Media Collective, where I am a PhD intern. At MSR, I have the intellectual freedom to ask questions that are widely important to society and scholarship. I also expect to make my research widely accessible. Microsoft open-sourced my code when I was an intern in 2013, and Microsoft Research has an open access policy for its research.
Although I am a fellow at the DERP Institute and can, in theory, start a conversation with Reddit employees, I have not discussed this project with Reddit at all, have never received compensation from Reddit, nor am I working for them in any way. While it is possible that I may be asked in the future to share my results with the company, I will not share any of my notes or data with Reddit beyond the findings that I publish in research papers, public talks, blog posts, or open source materials.
This isn’t the first time I’ve done research about the work of moderation from the outside a powerful company. Last month, my colleagues and I published a report on Reporting, Reviewing, and Responding to Harassment on Twitter, including a section on the work of moderating alleged harassment. In that study, we treated everyone in our study with respect, including alleged harassers. Our research team did not share data with the company, we were writing independently of Twitter, and we had full editorial control over our report, even from the commissioning organization WAM!. Likewise, in my 2013 summer research at Microsoft on local community blogging, we either summarized or anonymized/modified all quotes and photos before publishing our results.
In this project, I promise that:
Anyone can opt out of this research at any time by contacting me at /user/natematias. If you opt out, I will avoid quoting or mentioning you in any way in the published results.
By default, I will anonymize any information I collect before publishing
If a user requests that I use their username to give them appropriate credit for their work, I’ll weigh the risk/benefits and try to do right by the user
I will keep all my notes and data secured, with secure backups that I access through encrypted connections.
Why Do Research with Redditors?
Reddit is one of the few major public platforms on the English-language web that allows/expects its users to establish and maintain their own communities, without thousands of paid content moderators and algorithms behind the scenes deciding what to keep or delete. In contrast, the Huffington Post pre-moderates 450,000 comments per day, paying between $0.005 and $0.25 for every comment that comes in. Yet Reddit mods do so much more than just delete spam. They do a huge amount of important work to create new communities, recruit participants, post content, manage subreddit settings & style, recruit new moderators, set rules for their subreddit, and monitor/manage submissions and comments. Moderators also tend to play a large role in debating and establishing wider community norms like Rediquette.
Last week, I used the Reddit API to collect data on the number of moderators who keep subreddit’s conversations going. A random sample of 100,615 subreddits (roughly 1/6 of all public subreddits) had 91,563 user accounts as moderators. While not all of these subreddits are active, each of them represents a moment of interest to try on the role. Among the 46% of subreddits with more than one subscriber, 30% of these subreddits have two or more moderators.
Communities of redditors and mods shaped some of my earliest impressions of the site six years ago, when a work colleague in invited me to join Reddit London meetups, telling me stories about their weekend and after-work gatherings. It was clear that participation meant more to many redditors than just links and comments. Later on, when I took two years to facilitate @1book140, The Atlantic’s Twitter book club, with around 140,000 subscribers, I came to learn how challenging and rewarding it can be to support a large discussion group online.
How Am I Going to Go About This Research?
Computer scientists, economists, and designers often want to ask if offering the right upvoting system or the right set of badges will filter content effectively or motivate people to contribute the greatest amount of appropriate effort to a web platform. This focus on productivity often interprets the activity of users in the language of company priorities rather than community ones. Stuart Geiger and I discussed this idea of productivity last fall at HCOMP Citizen-X, arguing that online that we need to understand user’s values beyond just the “productivity” of a group.
Although I often explore questions through design and data analysis, I’m taking an alternative approach to better understand how redditors see their own participation. My first semester at MIT taught me how important it can be to participate and observe a community rather than just measure it. Rather than spend the whole summer data-mining the Reddit API, I’m participating in subreddits and speaking to redditors. In “The Logic and Aims of Qualitative Research,” a chapter in a larger collection on communications research methods, Christians and Carey say that when researchers ask questions about human life, “we are examining a creative process whereby people produce and maintain forms of life and society and and systems of meaning and value.” They argue that qualitative research sets out to “better understand the meanings that people use to guide their activities” (358-9).
As a student in MIT’s Technologies for Creative Learning class, I was curious about how young people learning to code thought about “bugs” in the stories, art, and games they made with Scratch. In a corporate environment, where there’s a goal for everyone’s work, it’s possible to define software errors. But does the same language apply to a ten-year-old child who’s creating a story after school? Most scholarly discussion of “bugs” applied this corporate term to young people, defining strict goals for students and measuring “errors” when they diverged from pre-defined projects. When I visited schools, observed student projects, and talked to students, I saw that diverging from the teacher’s plan could be a highly creative act. Far from an error, a “glitch” could prompt new creative directions, and an “unexpected surprise” often opened learners to new understandings about code.
Code and artwork from one of my first projects on Scratch.
If I had relied entirely on the definitions and data coming from teachers or the Scratch platform, I might have been able to test statistical hypotheses about “bugs,” and I might even have developed ways to limit the number of errors per student. I would never have noticed how important these unexpected surprises were to young people’s creativity, and at worst, I might even have reduced the chances of students to experience them. By participating with students and spending time in their learning environment, I was able find new language, like “glitch,” that might move conversations beyond “errors” or “bugs.”
For my Reddit study this summer, I want to hear directly from mods about how they see their work; questions that go well beyond what can be measured. Many thanks in advance to those who welcome me into your subreddits this summer and take time to talk with me.