After 15 years flying F-18 fighter jets for the U.S. Navy, Brian Burke probably knows more about aerial combat than anyone you’ll ever meet. It’s also likely — maybe even more so — that he knows more about National Football League statistics.
Since he left the Navy several years ago, Burke has spent his days analyzing football data and building predictive models as the founder of Advanced NFL Stats and a contributing writer at the New York Times. His latest venture is acting as the brains behind the 4th Down Bot, the brainchild of Times graphics editor Kevin Quealy (who also mans the bot’s Twitter feed) that uses a model developed by Burke to automatically analyze every fourth down call in every NFL game.
At this point, he told me in a recent interview, “I’m very familiar with the difference between what is mathematically optimum and what people tend to do.” And, at long last, the rest of the football-loving world might finally be willing to see the world through his data-obsessed eyes.
Fans are already interested
Thanks to pastimes such as sports betting and fantasy football, NFL fans looking for an edge over their bookies or their competitors have been crazy about data for a long time. Being able to predict which team will win a game by how much, and how well individual players will perform each week – two things Burke and others of his ilk do very well — become matters of great import. However, data sometimes matters even to more-casual fans, who just want their teams to win and see coaches make the right calls.
One could actually argue it was these folks — well, the ones who know Burke, at least — who are really responsible for the 4th Down Bot. He had been crunching the numbers about when coaches should decide to go for it or punt on fourth down for quite a while, and people knew it. Eventually, Burke said, “I got a little tired of people asking about each and every fourth down all Sunday long.”
So, in 2011, he built a tool called the Fourthdownulator to save himself from having to answer so many queries himself. Anyone could go the the Advanced NFL Stats site, enter the relevant information (such as how much time if left and how many yards needed for a first down) and the calculator would return figures for how many points a team could expect to gain (or lose) from each possible choice, as well as the probability any given choice would result in a win.
So that’s what caused the Packers to get blown out.
The metrics Burke chose to calculate with his model are somewhat unconventional, but they’re also critical to its utility. It’s easy enough to predict whether a team will convert a first down or field goal in any given situation, but his Fourthdownulator model — essentially the same one that powers the 4th Down Bot — takes into account additional factors such as resulting field position, opponent drives and game outcomes. Football boils down to a game of points, wins and losses, so it’s arguably better to know the likely effect any decision will have on point differential and game outcome than just whether a single play will be effective.
Burke is a big fan of the second-screen experience for sports, too. He wants to see the 4th Down Bot get a faster data feed next season so it can gives give suggestions before the snap rather than verdicts after the play is over. Then, he said, fans could really be “geeking out with the numbers on their lap.”
Will coaches come around on data? Should they?
And although every situation is different, there is a broad takeaway from all of Burke’s analysis (which other, simpler analyses have found, as well) that should reinforce fans’ confidence when they’re yelling at the television: “We should see coaches erring on the side of abandon as much as we see them erring on the side of caution,” Burke said. “But we don’t.”
Part of the reason is a natural human bias to give more weight to losses than they do to equivalent gains. When someone drops a $20 bill on the ground, Burke explained, they’ll feel twice as bad about that as they would feel happy if they found a $20 bill. “Coaches see failing on fourth down as losing the $20 bill,” he said.
Source: New York Times
Still, he seems alright with this to a large degree, because he understands the the difficulty of actually accurately predicting anything about a given play. Unlike baseball, which has a relatively small number of end states in any given situation and where there’s an orderly procession of individual matchups (i.e., pitcher versus batter), football is a statistical nightmare. There are different downs, distances, penalties, substitutions, injuries, fumbles and a disorganized interaction between 22 players (11 on each side) that makes predicting the outcomes of plays very challenging. With free agency the way it is, players are changing teams and creating entirely new 11-man offensive and defensive units annually.
“The number of variables, it explodes geometrically,” Burke said. Even the 4th Down Bot — built largely as a fun tool for fans — can’t factor in every possible factor, from momentum to weather, that could affect the outcome of a play or a game.
A real game isn’t quite as random as electronic football, but it’s close. Source: Flickr / John-Morgan
Additionally, he said, the relevant sample sizes for NFL data is small compared to a sport like baseball, which makes thing even tougher. There are only 16 games each season (compared with 162 in baseball) and different rule changes and trends in how the game is played (the recent transition into the NFL as a pass-happy league, for example) can make even seemingly recent data irrelevant. Depending on the research he’s doing, Burke said the 2000 NFL season is about as far back as he goes.
But Burke, who consults for several NFL teams and executives, also noted “that day is coming” when team leadership will expect coaches to start taking the data more seriously. Maybe that will come from executive mandate — like CEOs in other businesses are starting to respect and demand data-driven decisions (come to our Structure Data conference in March to learn more about this) — and maybe it will come from coaches taking the time to learn what it is that predictions like the ones Burke’s models generate really are.
“We’re just putting solid numbers on things that coaches think of in nebulous terms,” Burke said, citing the point expectancy of a fourth-down call as an example. And while building the models is hard, he added, using them isn’t: “At the end of the day, it’s not very difficult math. It’s fifth-grade arithmetic.”
Beside, if anyone should be concerned about advances in NFL data analysis, it might be Burke. By the end of March, he predicts the months-old 4th Down Bot will have more Twitter followers than he does. “I built a robot to replace myself,” he joked, “and it’s a very strange feeling.”
Humans: Have questions about 4th downs or how I make my calls? Ask me and I’ll tweet answers ahead of the Super Bowl. #askNYT4thDownBot
— NYT 4th Down Bot (@NYT4thDownBot) January 31, 2014
Related research and analysis from Gigaom Research:
Subscriber content. Sign up for a free trial.
A near-term outlook for big data
Why we must apply big data analytics to human-generated data
Gigaom Research predictions for 2014