The Internet has been abuzz with a recently released study from the Harvard Business School that found that black hosts on Airbnb were charging 12% less than white hosts for similar listings. So is there discrimination on Airbnb? Do whites really make more than blacks on comparable listings?
The study authors, Benjamin Edelman and Michael Luca, looked at listings in New York City as of July 12, 2012. By using one set of paid sub-contracted workers to rate each listing’s photos on a scale of 1-7 (7 being “this is an extremely nice apartment, I would stay here even if it was more expensive than a hotel room” and 1 being “this is a terrible apartment I would never stay here”), and another set of workers to gauge race.
The duo then collected the specific listing information – including price and qualities like number of bedrooms, previous guests’ star ratings on things like location, communication, accuracy and cleanliness – to control for the quality of listing, and thus how the differences panned out between white and black hosts on comparable listings.
The study compared listings through statistical analysis based on number of bedrooms, social accounts linked, photo quality, communication rating, cleanliness rating, and location rating, among others. Picture quality ratings were perhaps the most subjective, and were not submitted by actual guests who stayed with hosts.
They concluded:
We find that non-black hosts are able to charge approximately 12% more than black hosts, holding location, rental characteristics, and quality constant. Moreover, black hosts receive a larger price penalty for having a poor location score relative to non-black hosts. These differences highlight the risk of discrimination in online marketplaces, suggesting an important unintended consequence of a seemingly-routine mechanism for building trust.
One only has to look at the comments on sites like ValleyWag that show how contentious this paper was – and how inherent racism bubbles up throughout online forums. This is clearly a vital discussion in the context of how technology facilitates inherent human bias – and how technology might be able to mitigate, rather than fuel – some of society’s less desirable traits.
This study brings out some very compelling issues when it comes to mechanisms for building trust in online P2P marketplaces, especially when it comes to trustworthiness gauged by profiles with visible photos, social networks and other identifying information.
What market dynamics are actually in play here?
It’s also important to point out that the market dynamics aren’t simply “black and white.”
Hosts are incentivized by very different reasons for the prices they set on their listings – of course, perceived interest or value due to race or other demographic (female, LGBT, etc.) is one. Another is the desired occupancy rate.
Some hosts (such as the author) prefer to keep a lower price than nearby listings in order to have a higher occupancy rate. Higher prices mean fewer bookings, and this results in lower income overall – it’s volume-based pricing that makes up for the lower nightly rate.
One would never know these realities without directly interviewing each black host that had a lower price than a comparable listing by a white host.
There’s also the issue of pricing intelligence. If Airbnb doesn’t educate hosts about what comparable listings are going for, hosts may not accurately price their listings – or at least they may unknowingly price the listing lower than comparables.
Tnooz spoke with a black host in Seattle, Washington, a 27-year-old named Chaz. Chaz has a shared room on his listing, where the guest sleeps on a sofa bed in the living room. This is a different style of offering than a full-room rental, as the listing states the setup is “meant for the budget traveler and experience akin to Hostel Traveling,” and thus has a naturally lower price point.
Chaz, who also has a video profile where he speaks to potential guests, wasn’t convinced that the study was as cut-and-dry as its authors suggested, saying that:
a true test of this would say if some one who wasn’t black also had a “Crash Pad of Awesome” and seen if he got more bookings than I did with a higher price point, and if we started at relatively the same time. There are too many mitigating factors than just race to be the end all be all of more money versus less money.
This is a key realization, as hosts have a myriad factors to consider when setting price – of which incoming bookings is one. Here’s how this particular host sets his rates:
[The rates are] pretty much seasonal rates plus under cutting. When I started, I lowered my price than competitors (other Airbnbs and hostels) so I could get reviews. Then I fluctuated around the $33~$38 dollar range to see what I could actually get for it. Now I set my prices based on the experience over the year and a half I’ve been doing it.
It’s $31 in the Winter, $35 in the Spring and Fall (Although thinking about changing it to $38 cause of this weird thing our minds do where if an end number is stronger it’ll seem “cheaper”, like Wal-Mart) and $41 in the Summer. The extra $1 is there so I get even numbers when the money is sent to my account.
This pricing component is perhaps the most important market dynamic here. Rather than simply pricing a room 12% lower because people wouldn’t book at a higher rate with a black host, the sweet spot in pricing versus occupancy rate comes into focus.
I think that Airbnb would non-shockingly follow the price rate of apartments based on how many miles are outside of Downtown. Taking Seattle for example, not many blacks live in Belltown which is a prime location and I think factors heavily on what I can charge and why people stay. There is another black host that lives right next to Pike Place Market and he offers his Futon and Room for guests and is charging $45 dollars a night. I’ve seen even some of my guests stay with him as well! I think that his overall bookings are lower than mine because of the higher price point even though his place offers more privacy.
So I feel the old adage of “Location, Location, Location” is the biggest factor and the social economical aspect that blacks are more likely to live in “bad” neighborhoods that are farther away from a city’s central district is most likely at play here.
The study took this into consideration, by comparing guest-generated ratings such as location in the way the listings were compared.
Of course, there’s no way to ask someone who looked at one listing from a black host and ended up choosing a white host that charges more – that data is not forthcoming (Airbnb declined to offer any actual booking data to the study authors), so it’s difficult to identify what part of the market dynamic is most important in each booking.
As Airbnb host Chaz notes:
Discrimination is just preference in another light. We’d have to do what all other brands do and work on the quality of the experience and hopefully that’ll over shadow the presuppositions that come with race along side other things that are out of and the actions taken there of.
When asked about discrimination, Chaz is emphatic: “No! Never actually. It doesn’t even come up in conversation.”
Anecdotally, when searching for “shared rooms” in the Belltown neighborhood in Seattle, only one other listing appears – and it costs twice as much for a futon in a living room than Chaz’s listing.
While this is a significant price difference, one only has to look at the review count: over 66 for Chaz and 0 for the other listing. Clearly, lower pricing increases volume which is desirable for hosts looking to increase occupancy rates.
Chaz, the Airbnb host, very clearly feels that the playing field is not stacked against him on the P2P accommodation platform.
I’ve never felt at a disadvantage cause I was black. I’ve always tried to use my business savvy to stay ahead of the market and price accordingly to maximize my bookings. I think my American is going to be showing when I say in a free market society the quality and demand are going to be the largest mitigating factors in how much one can charge for their product.
It could be that other black hosts like me are woefully ignorant to various techniques they can use to maximize their profits comparative to their non-black counter parts. With Airbnb opening up groups where best practices can be shared and explored, that ignorance will soon fade. Knowledge is power folks, knowledge is power.
This particular Airbnb host sees the sort of free market dynamics that these P2P marketplace encourage as the key market dynamic, rather than using race exclusively.
However, even the statement that “discrimination is preference in another light” shows just how challenging the demographic issue of providers on P2P marketplaces can be – if a guest can see that a host is black/gay/female/insert-minority-here, and chooses not to stay, is it discrimination or personal preference? This line is blurry, thin and constantly shifting.
The study authors respond to Airbnb
Airbnb released a blanket statement to Re/code:
We are committed to making Airbnb the most open, trusted, diverse, transparent community in the world and our Terms of Service prohibit content that discriminates. The data in this report is nearly two years old and is from only one of the more than 35,000 cities where Airbnb hosts welcome guests into their homes. Additionally, the authors made a number of subjective or inaccurate determinations when compiling their findings.
Airbnb’s PR representatives did not respond to Tnooz’s request for comment, specifically related to what “subjective determinations” the study authors made, what procedures the company has to address any discrimination on-site, and finally what technologies might be in the works to help hosts most accurately price their listings.
The study authors responded to questions from Tnooz, with the most important point being that the authors had the same information available to them as guests. While the study is imperfect without actual demand data, the study does reflect how Airbnb users make accommodation decisions.
We don’t know what Airbnb has in mind with the conclusory allegation of “subjective or inaccurate findings”.
When Airbnb has made more specific charges, we’ve often found their allegations to be incorrect. For example, Airbnb told some reporters that we didn’t consider whether a listing offered an entire property or just a single room. That’s demonstrably incorrect. In Table 3 of our paper, see the row that reads “Whole Apartment”.
Airbnb told other reporters that classifying photos is inherently subjective. But we hired independent (Mechanical Turk) staff to rate photos; they didn’t know the subject matter of the study; they could only see the property photos, not host photos or anything else about the properties. This is a proper and robust approach.
The question relating to the subjectivity of photos is important – however, is there ever an objective way to rate photos? Potential guests most certainly are not, so it’s hard to imagine a better way – except for finding actual people seeking to book a spot in New York City and observing how they go through the process.
Regarding the data being outdated, the authors think this is baseless.
The data was collected during summer 2012. There’s no particular reason to think things have changed since then. Indeed, Airbnb’s property listing page has stayed substantially the same, including the same layout and format. We have no reason to think this affected our results, nor has Airbnb offered any reason to think so.
Finally, the authors believe they have revealed an inherent issue at odds with the familiar trope of the “democratization of the Internet:”
The Internet is broadly expected to help prevent discriminatory outcomes by limiting the distribution of unnecessary information and information that facilitates discrimination. And indeed sometimes it does – a seller on eBay or Amazon Marketplace surely suffers no penalty due to a disfavored race, for those platforms don’t make seller’s race (or photo) knowable to buyers.
But Airbnb portends a different vision of the future – one in which photos and minority status become more prominent rather than less. We think that’s unnecessary and probably for many purposes undesirable. As more businesses shift towards the person-to-person model, this change could grow further – unless there becomes a norm that, in fact, such information need not and ought not be granted such prominence.
The study authors are currently looking at expanding the study to include more details and geographic locations, in addition to more listings and more qualitative interviews with hosts of color. Hopefully looking at other minorities – such as LGBT hosts or other demographic qualities that can be gleaned from photos or profile information – will also be included, to see how this all pans out more holistically across the ecosystem.
Market design problem
The key takeaway from the study is that the authors see potential discrimination as a market design problem: by showing photos, names and other social networks, the company is providing demographic details that can be used to discriminate.
In providing a mechanism for users to complete profiles and upload photos, Airbnb is unlikely to face legal considerations that affect its user interface or design.
Moreover, pictures are an important part of Airbnb’s design: from discussions with Airbnb guests, we understand that pictures help guests accept the Airbnb model, including staying in a property with, or offered by, a stranger. Foregoing host pictures would likely reduce some guests’ willingness to use Airbnb. Hence, if Airbnb were to take action to reduce the extent of discrimination, the decision would be driven by ethics, rather than profit or the law.
The study authors also see these sorts of profile enhancements used to build trust as a step backward as far as the democratization potential of Internet technologies, especially in direct transaction models like Airbnb.
Despite the potential of the internet to reduce discrimination, our results suggest that social platforms such as Airbnb may have the opposite effect. Full of salient pictures and social profiles, these platforms make it easy to discriminate—as evidenced by the significant penalty faced by a black host trying to conduct business on Airbnb.
In their email to Tnooz, the authors want to see more transparency in design choices:
The core question here is how Airbnb designs its site. Right now, a host’s photo is quite prominent – we’d say, it’s the second-most prominent item on the page, after the default photo of the property. Is the host photo truly the second-most important piece of information for a guest to consider when evaluating the property? We are not so sure.
If you had to choose between looking at a host’s face, versus a word cloud of concepts from guests’ reviews or more information about the average score in guest reviews, I think it would be hard to argue that the photo is truly more helpful. Meanwhile the photo has the obvious problem of facilitating and in effect inviting guests to evaluate properties based on a host’s personal characteristics rather than the property’s merits.
More generally, we think P2P services should think carefully about what information they provide, when, and how. Surely a Lyft driver should see an on-phone image of the passenger’s face as the driver nears the pickup point, so the driver can recognize the passenger. Does the driver need to see the passenger’s photo when deciding whether or not to accept the passenger’s request? We think not.
The obvious argument here is that these profile enhancements are doing their job – they are providing social proof and cues to a host’s trustworthiness that ultimately make a guest more comfortable to book the room. Guests will be seeking out hosts that will provide the best experience for them. So if this discrimination is true – and not simply a different market dynamic at play, as mentioned earlier – than is Airbnb simply a factor of a society in which these sorts of considerations are played out everyday?
Conclusions – what does the P2P industry need to do?
Andrew Bate, the co-founder of P2P rental trustworthiness company SafelyStay, sees these sorts of qualitative issues as a key problem that must be solved for the P2P accommodations industry:
We are also seeing some discrimination in the traditional vacation home rental reservation process when hosts and guests communicate prior to the reservation confirmation. Occasionally, homes suddenly become unavailable when the guest has an accent or “ethnic” name. Proving discrimination in these cases is almost impossible.
Individuals are not equipped to make such decisions. We have a variety of cognitive biases that give irrational validity to information presented to us, so when the marketplaces give us information but no real measure of trustworthiness, this information can be used inappropriately. The same dynamics were more prevalent with lending prior to the FICO score, and housing decisions prior to the Fair Housing Act.
Regarding potential changes to the industry, Bate, who has a vested interest in addressing this very problem of trust, suggests the following:
As an industry, we need to begin using empirical and relevant criteria for determining a guest’s and host’s trustworthiness. How much the host looks like “me” is irrelevant to the quality of the stay, and how the guest looks or smiles has little to do with his or her propensity to be a trustworthy guest.
Ratings need to be unpublished. There is an inherent rating quid pro quo in today’s rating systems. Guests are leaving positive, and perhaps untrue, ratings in hopes of also receiving a positive rating. This rating inflations decreases the integrity of the entire system.
Marketplaces need to stand behind their suppliers. Could you imagine going to a supermarket to return smelly fish, only to be told you need to visit the fisherman for a refund? Until marketplaces take responsibility, hosts and guests are forced to search for trust proxies on their own, with no tools and extremely limited information.
There’s also plenty of pushback that the study was too superficial to truly provide evidence of discrimination, including comments that the study didn’t do a good job of controlling for location in areas that are highly racially segregated. This recent graphic on Wired shows just how much American cities are segregated.
Here’s New York’s racial map, which shows whites in blue, blacks with green, Asians in red, Latinos in orange, and everyone else in brown:
Nonetheless, the study used relatively geo-agnostic ratings to compare properties. The location rating isn’t just focused on one area, but on how the guest rated the location overall – which could be positive if a guest was looking for a more off-the-beaten-track location but doesn’t necessarily allow direct objective comparisons between specific locations.
The study has uncovered many more questions than answers, and the vibrant discussion is a sign of unintended consequences as new technologies gain traction -especially in demographically-relevant industries such as travel. The discussion also presents an opportunity to discuss potential technological and design solutions that can improve these P2P marketplaces.
Read the full HBS study here.
NB: The author is an Airbnb host.