Last year, FiveThirtyEight held a bakeoff between eight people who thought they could chip away at the secrecy surrounding the Academy Awards by building predictive models. Now that we’re days away from this year’s Oscars, we brought in two of the best, Allison Walker and James England, to chat about their models with chief culture writer Walt Hickey. The transcript below has been lightly edited.
We’re tracking which nominees are favored in the race for eight major Academy Awards. Read now »
Walt Hickey: Allison, what’s your method?
Allison Walker: My model predicts Oscar winners based on the frequency with which certain words are used to describe the movies by users on movie-review websites. Last year I took reviews from Rotten Tomatoes, and this year I’ve added reviews from IMDb. For categories specific to a person (director and the acting categories), I look only at words used in the same sentence as the name of the person nominated, or, in the case of the acting awards, the nominee’s name and the name of their character. Based on what words have historically been more associated with winning the award, the method then assigns a score to each movie that is used to predict how likely it is to win.
Walt: And this worked out really well for you last year, if I remember correctly?
Allison: Yes, I was able to predict all of the categories I attempted to predict except for best picture.
Walt: What are you looking at for best picture this year? What are some of the words that are linked with winning?
Allison: The word most associated with winning that is included in my model is “Hollywood,” which obviously gives “La La Land” a huge advantage. “Oscar” and “best” are also strongly associated with winning, which means that if the reviewers are talking about a movie deserving an Oscar, that improves the movie’s odds of winning.
Walt: So your model actually implicitly teases out buzz? Neat! James, you take a different tack that draws on your experience in analyzing college football, right?
James England: Right. It’s the exact same algorithm as sports rankings. The big difference being that films don’t truly go head-to-head, so I set up a poll and asked folks who’ve watched these movies to select the winners of each matchup.
Walt: And who’s looking good in best picture?
James: I think this one is up in the air a little. “Moonlight,” “La La Land,” “Manchester by the Sea” and “Arrival” all have positive rankings on my site. So then it gets into how high those rankings are, the percentage of people actually viewing the films, etc. If a film is fantastic with a huge rating, but no one really saw it, I have to weigh that a bit. If you’re going with what I’ve got right now, “Moonlight” looks like the favorite. Only it and “La La Land” have made it into the No. 1 spot since the poll launched.
Walt: Last year, you had the only model to correctly predict “Spotlight.” This kind of head-to-head format seems useful especially when considering the funky ranked voting algorithm the Academy uses.
So, what would each of you say are the advantages and disadvantages of your model? I can say for the FiveThirtyEight one, I don’t like how slow it is to respond to events and how it boils down wins that can mean lots of different things into one heuristic. But all models have flaws, right?
James: The most obvious flaw in mine is who is voting. The only real vetting my voters undergo is that they go to my website and say they’ve watched a couple of the films nominated. But like Walt has said before, we don’t really know who the Academy is or what they think, so that randomness might also be a feature.
Allison: I would say the main flaw of my model is that it relies on the opinions of people reviewing movies on the internet, which do not necessarily line up with the opinions of the Academy. For example, it seems that a lot of people on both Rotten Tomatoes and IMDb really seem to like Mel Gibson (“Hacksaw Ridge”) and are talking about how great a director he is. Since my model scores the word “director” highly in the directing category, it has Gibson as second most likely to win best director, with Barry Jenkins (“Moonlight”) first and Damien Chazelle (“La La Land”) third. I noticed that in James’s model, Gibson is the least likely to win, so maybe our models draw from different populations.
James: Jenkins just passed Chazelle in my ratings.
Allison: That is interesting that we both have Jenkins to win!
I’m less confident in my supporting actor prediction this year as it seems to contradict what many other predictions are saying. My model has Michael Shannon (“Nocturnal Animals”) as most likely to win, followed by Jeff Bridges (“Hell or High Water”), Dev Patel (“Lion”), Mahershala Ali (“Moonlight”) and Lucas Hedges (“Manchester by the Sea”). One word that is associated with winning is “career,” which is used often in descriptions of Shannon and Bridges but not Ali. I’m wondering if the fact that Ali has had more high-profile TV roles (such as “House of Cards” and “Luke Cage”) than prominent movie roles means that movie buffs who review movies online are less likely to talk about him in reviews because they might be less familiar with his work.
Walt: If Shannon does win an Oscar one day, it’ll definitely be one of those “recognizing a body of work” ones, I suppose. What do y’all have for supporting actress?
James: For best supporting actor, Ali has about the same rating this year as Leo had last year. The big question in my mind is how many people saw “Moonlight” and if that ends up playing a role in these awards. For supporting actress, I’ve got Viola Davis. She’s crushing it in the ratings, but “Fences” is the least-seen movie in my poll compared to all the other nominees in that category.
Allison: My model has Davis as a strong favorite, followed by Michelle Williams (“Manchester by the Sea”).
James: I am a little surprised that Nicole Kidman is so low. As of this conversation on Tuesday afternoon, I had her with a 0.27 percent chance of winning. I didn’t even know that was her when I saw “Lion” — maybe no one else did either.
Walt: Transforming for a role: good. Literal camouflage: whoops.
Allison: My model could not generate a score for Octavia Spencer (“Hidden Figures”) because none of the words the model looks at were used to describe her. My model also has Kidman as least likely to win, so maybe online reviewers also didn’t recognize her.
Walt: All right, so the supporting acting parts are pretty consistent. Let’s talk about lead actor and actress, the two categories that keep me up at night. What on earth is happening in best actor?
James: My model has been very consistent for Casey Affleck (“Manchester by the Sea”). Denzel has the next-highest score, but, once again, not a lot of folks in my poll have apparently seen “Fences.”
Walt: Going by box office, “Fences” is smack dab in the middle of the pack of best-picture nominees.
James: Interesting. I wonder what the overlap is for folks who may have just watched “Fences” and no other Oscar-nominated movies. They wouldn’t represent well in my model.
Allison: My model predicts Affleck to win with a very narrow margin above Washington (with a 0.6 percent difference). One thing I worry about is that the words my model looks at, such as “performance,” “great” and “Oscar,” are generally used only to describe the actor’s performance, so I don’t think my model can pick up on things the Academy might not like about the actor as a person.
Walt: Which is the million-dollar question this year. The FiveThirtyEight model can’t track that either. So how about best actress? This is by far the most fun category to watch this year.
James: I think this is a dead heat. Natalie Portman (“Jackie”) has the higher rating. Emma Stone (“La La Land”) has the higher viewership.
Allison: My model has Portman as most likely to win followed by Meryl Streep (“Florence Foster Jenkins”) and then Isabelle Huppert (“Elle”).
Walt: This is the best category. So what do you think is causing the chaos?
James: Portman was far and away ahead about a week ago. Monday night’s ratings had Stone in the lead for the first time. And as of Tuesday afternoon, they’re neck and neck. I think Stone is riding the “La La Land” wave. “Jackie” didn’t get nearly the love that “La La Land” got.
Walt: I don’t know if I need to flag the spoiler or anything, but … that “Jackie” movie turns into a real bummer.
James: I drove by Dealey Plaza a week ago. Looks like they got rid of the X.
Allison: Streep might be given an artificial boost in my model because she is so well-known that people dedicate a lot of their reviews to talking about her. I think a movie that was very popular overall might actually hurt an actor in my model because the reviewers might spend less time talking about them if they were just one of the things they liked about a movie. When I look at the reviews for Portman and Huppert, a lot of people talk about how they carried the movie. The same isn’t true for Stone.
Walt: That is such a good point.
James: Huppert is interesting. She actually has a higher rating than Portman but not a lot of total votes. My voters have been … unimpressed with Streep’s performance.
Allison: I haven’t had a chance to see “La La Land” (or most of the nominees) yet.
James: “La La Land” is a magical journey. It’s kinda funny how depressing almost all of the others are compared to it.
Walt: Whoa, “Manchester by the Sea” is 95 percent black comedy, 5 percent dismal tragedy.
So before we wrap up, what categories are you most confident in? Least confident? Like I said, I would not be surprised if we missed on one of the lead acting prizes this year.
James: I think Ali is my No. 1 lock. Davis is pretty close to one as well. Compared to Leo and Brie Larson last year, I think there’s a lot more room for upsets in the big categories.
Allison: I think I am most confident in Davis for supporting actress, as she seems to be the favorite no matter the model. I’m probably least confident in director and supporting actor because my predictions disagree with a lot of other models.
James: Quite honestly, I had no idea that “Arrival” was going to even be in the discussion for best picture, but it could be a dark horse, according to my ratings.
Allison: My model has “Arrival” as seventh most likely to win best picture, but I was wrong about “Spotlight” last year, so who knows!
Walt: That’s the main thread of this: The science of Oscar predicting is way more art than science. After the year we’ve had, for all we know “Deadpool” is gonna win best picture.
James: Well, we all know that the best picture is “Moonlight.” We just gotta figure out how to make that happen.