For years, our top priority at Zomato has been to ensure that we remain a trusted resource for the millions of people who use Zomato every single day.
An important part of this priority is to keep biased content out of reach of our users. There are a number of automated checks, which learn and get smarter over time. In addition to that, our neutrality team ensures that we are constantly watching for new ways in which people try to game the system; this knowledge is fed back to our engineering team, which makes sure that we always stay one step ahead of business owners who write good reviews for their own business, and bad reviews for their competition.
Over the past year or so, with the growth of popularity of Zomato as a restaurant discovery product, we’ve seen a rise in the number of people creating biased content – a few agencies spread across the world are now offering services to artificially boost restaurant ratings on Zomato, and individuals are offering to write positive reviews for restaurants ‘as a service’. And business owners are happily buying into this, because of the lure of a higher rating on Zomato. For the most part, this doesn’t work. When it does, it means that someone somewhere has figured out a way to get smarter than the system that we have created.
We are constantly working on making sure that we are smarter than the folks who want to game the ratings on Zomato to make a quick buck. Over the last few months, we have been working to completely overhaul our bias-detection algorithms. And tonight (7th June, 2016), we are rolling out a strong new anti-bias algorithm that will help clean up biased reviews retroactively, and also put sophisticated new bias checks in place for the future. While we can’t divulge too much (for obvious reasons), we’d like to highlight some of the key things that will change.
This is our Panda update, so to speak. Here are the details of what’s changing –
Less deletions, more hiding. One might assume that deleting biased content from Zomato is the easiest thing to do. It is, technically, but anyone smart enough to try and game the system in the first place, will also be smart enough to identify patterns in what gets deleted and what doesn’t. So from now onwards, we will be deleting fewer biased reviews (but we will still delete the obviously biased ones), and algorithmically hide such reviews where they won’t be seen by most users. This will ensure that the user experience doesn’t get hurt, and spammers don’t have a clue.
User credibility scores. The new algorithm takes a fundamentally new approach to user credibility, and significantly increases the confidence level with which we can predict a user’s bias in their reviews. The new credibility scores assigned to users increase or decrease their ability to affect the restaurant’s overall rating. Credibility now factors in users’ behavioural patterns on Zomato over a period of time, as well as the quality of a user’s content. There’s enough NLP and manually curated historical data in place for us to cluster users into various ‘bias’ categories.
Moderation history. For ease of explanation, we’re going to use one of the oldest (and slightly more dramatic) cliches in the book – it takes many good deeds to build a good reputation, and only one bad one to lose it. What this means is, if a user has a history of having their content flagged and moderated, their ratings will automatically carry far less weightage in a restaurant’s overall rating. While Zomato is an open platform where we encourage people to write honest and unbiased reviews, we take abuse and bias very seriously, and will do what it takes to keep Zomato free of it.
Recency and decay. Everybody makes mistakes, and consistency is not always a given. Business owners have often complained that they still get “punished” for a bad review which they received 5 years ago, and their streak over the last 2 years has been very good. Fair point. Going forward, the effect of older ratings and reviews on a restaurant’s aggregate rating will taper off over time, giving users a better idea of what they could expect at a restaurant if they were to visit today.
Those are, in a nutshell, some parts of the new anti-bias algorithm. If we tell you more, or give you more details, we’d be doing a disservice to our users by disclosing more than we should to the people we are fighting to keep Zomato bias-free.
Starting tomorrow, some restaurants may benefit from having a lot of biased, low-value content hidden from plain sight, while some may see their rating reduce due to lower ratings from (in)credible users. We hope users and restaurant partners alike will understand and appreciate that this is being done to improve the overall quality and credibility of ratings and user-generated content on Zomato for the long term.
We’ve always told every business owner we have ever gotten in touch with – “improve your business, delight your customers, and the ratings will take care of themselves”.
However, since you got to this point in this post, there’s one important point to remember. Zomato ratings are not simple averages. You cannot calculate the average of the ratings and reviews that you get and say “this rating doesn’t make sense”. And then there’s normalisation. Or classroom ranking as some people call it. For a city, we forcibly fit all restaurants and their ratings on a normal distribution curve. In short, there’s a lot that goes on under the hood to make sure that you get a true sense of what you can expect.
There’s plenty more to come that will help make Zomato an even better and more useful product. Over the years, we’ve kept working on ways to keep the bias out, and it’s something we will continue to do. Folks who try to beat the system will always try and find new ways to do it, so it’s important that we evolve faster – and this is a strong step in that direction.