You've been hearing more and more about these advanced stats, but are they good for anything and will they help me win an argument? Yes! If you just take a moment to understand what they mean.
More and more often I've been noticing the willful ignorance that people use as a blanket in every day life. It's not as though I couldn't understand it -- the most simple explanation is also the one that allows you to move onto something that you place more value on in your life -- but it still exists. And trying to convert anyone into your way of thinking is often an exercise in frustration not typically felt anywhere outside of an episode of Game of Thrones featuring Joffrey.
I recently listen to Jonah Keri (baseball writer, Grantland) on The Nerdist podcast, and he mentioned a study that showed when you use data in a debate to support your argument the other party will only retreat further and further into their own beliefs. It immediately struck me how spot on that conclusion was.
Despite the fact that you'd think that foolproof facts and numbers would give you the upper-hand that you so desired, I have usually only found that the other person will only dismiss the numbers and stats that they don't understand because understanding them would take time. And effort. And there's always the chance that at the end of that time and effort, you still don't understand.
Rarely do I hear the opinion that "DVOA is a flawed metric because as discussed in the 2008 paper by Billy Von Statsenberg..." blah blah blah. Instead it's: "Yeah well, I judge the sports by what I see!"
Okay, you can have the time when the games are on for your argument. I'll take those and then also the other ~300 days per year when stats are always readily available. If you only want to use stats as a secondary weapon in your arsenal of sports knowledge, that's understandable. But for those that dismiss stats as "useless" simply because they don't understand them, that's ignorance.
And it's by choice, because stats are easy to understand.
During this past season I had a weekly recap and update on the advanced stats in the NFL and where the Seahawks stood, game-by-game. I typically stick to the websites Football Outsiders, Pro Football Focus, and Advanced NFL Stats for my information and resources. What I want to do in a new series is try and breakdown and explain what these "newfangled fancy-boy stats" really mean. If you don't like these stats, then dislike them based on what you know about them, not everything you don't.
We'll start the series with the most commonly thrown around advanced stat in football today: DVOA.
Developed at Football Outsiders by founder Aaron Schatz and his team and based off of a book called The Hidden Game of Football by Bob Carroll, DVOA is an effort by FO to not only rank teams and players against each other in a given season, but across many years. It takes a long time for them to come out with new seasons of DVOA data based on the fact that they have to watch every single game in a given year. And every single play in those games.
EDIT From Schatz:
DVOA is not based on game charting at all. We don't have to watch the games. That's why we come out with it immediately each week after the games. DVOA is based only on play-by-play straight from the NFL. The stuff we do with game charting is totally separate.
(The reason it takes so long to do PAST years of DVOA is that we've now gotten into the era BEFORE PBP was on the Internet and thus we have to hunt down print gamebooks and in some case tapes of games to create our own play-by-play.)
DVOA is not for the lazy. Whether you agree with it or not, you can't sit down at your computer in one day and just pop out a DVOA for the 1972 Chicago Bears on your .xlsx spreadsheet. So let's take a closer look at what DVOA means, how it's generated, and why it's better than using a number like "total yards" in your argument.
(Also, if you're a Seahawks fan, why wouldn't you want to understand DVOA? The 2012-2013 Seattle Seahawks are one of the top three franchises in DVOA history.)
What is DVOA?
Defense-Adjusted Value Over Average. What does that mean? Let's break it down piece by piece but let's start with how that value is calculated. When a stat is as old as the game itself, it can be quite easy for someone to throw it around in debate and feel as though they've won the argument.
"Josh Gordon led the NFL in receiving yards and I ain't no dumb-dumb, that means he's the best receiver in the game!"
Gordon had a phenomenal 2013 season and I wouldn't want to disparage his accomplishments, but when someone points out that Gordon had the most yards in the NFL last season, it's important to acknowledge how many data points should be considered when trying to evaluate who the "best" was in 2013. And it becomes more interesting, for me at least, when you notice that Gordon was ninth among receivers in DYAR and 17th in DVOA.
When you tell the casual NFL fan that Gordon was 17th in a category that supposedly measures the effectiveness and value of a player over the course of a season, more often than not that person may counter with: "But he was FIRST in my fantasy football rankings!" And it's the counter-balance of those two statements that makes a lot of people dismissive of DVOA and DYAR as viable stats. It would appear that the advanced numbers are missing something that the eye tests, the highlight reels, and the final stats clearly show.
But is it actually the opposite? Are the final totals for catches, yards, and touchdowns actually missing something that DVOA hopes to fill-in? Because DVOA and DYAR both take yards and catches into account. The advanced stats simply go a bit further, evaluating when and where those yards were accumulated, and against whom.
EDIT from Schatz:
I mean, obviously Josh Gordon was freakin' awesome last year, and three times as awesome when you realize who the QBs were and how the other receivers did. So it's not really fair to say "DVOA and DYAR say he wasn't the best receiver in the league." He was at least top three, and there's a good argument I think that he was the best.
EDIT from Kenneth:
I think it's fair to say that this right here goes to show that DVOA and DYAR are still just tools to be used in formulating an opinion on value. In the case of WAR in baseball, people that use it as their bible would say that Mike Trout was clearly better than Miguel Cabrera last year based on their Wins Above Replacement. But there's still an argument to say that there's a lot that goes into player evaluation and ranking than any one stat. Gordon was 17th in DVOA, and the guy that invented DVOA is telling you that he thinks he's either the first, second, or third best receiver in the NFL.
It's just a matter of understanding why Gordon would rank 17th in DVOA and ninth in DYAR and then separating that from other factors, like teammates and opponents, and coming to your own conclusion.
As explained by Football Outsiders, down-and-distance is the first thing to consider when assigning point values to plays:
"On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success."
I think that's something that just about any football fan can digest, especially the part where the only thing that matters on third down is gaining a first down. Unless it's a situation where a team is setting up for a last-second field goal, and that's why FO watches every single play of the season to determine not just the outcome on the box score, but the situation in which the play occurred.
Single-game DVOA performances versus total yards:
On November 24 against the Pittsburgh Steelers, Gordon had 14 catches for 237 yards and a touchdown. Those are totals rarely seen in the NFL, but let's take a closer look at all 14 catches:
- On 3rd-and-5, Gordon catches a 24-yard pass from Jason Campbell. This is a first down on a third down, meaning it's a big win for Gordon's DVOA.
- On 1st-and-10, Gordon gains two yards. That is less than 4.5%, therefore it is a failed play.
- On 1st-and-10, Gordon gains 10 yards for a first down. This is another successful play.
- On 2nd-and-6, the pass to Gordon is incomplete.
- On 1st-and-10, Gordon gains 16 yards. Success.
- On 2nd-and-10, Gordon gains 10 yards. Success.
End 1st half
As you can see, Gordon has five catches on six targets for 62 yards. The Steelers take a 13-3 lead at halftime, but the game is well in reach. It's important to note in DVOA measurement, how much is on the line when the yards are accumulated.
- On 1st-and-10, Gordon gains nine yards. Success.
(Four plays later, Campbell is sacked and fumbles, which is returned to the Browns 4-yard line. The Steelers score on the next play, making it 20-3. Nearly out of reach. Especially since Brandon Weeden has now entered the game.)
- On 1st-and-10, Gordon gains eight yards. Success.
- With 6:54 left in the game, Gordon gains 47 yards on 1st-and-10. This is a success, but it's not quite as valuable as you may expect a 47-yard gain to be. The Browns are down 17 points with less than seven minutes left and it did not result in a score. Also, the play started at their own one. They aren't even in Pittsburgh territory yet.
- Weeden is sacked and fumbles the ball back to the Steelers. Cleveland forces a three-and-out, but on the first play of their next drive, Weeden throws a pick-six to William Gay. It's now 27-3 with 4:27 left.
The game is out of reach.
- On 1st-and-10, Gordon gains 42 yards.
- On 2nd-and-10, Gordon gains 19 yards.
- On 1st-and-10, Gordon gains 16 yards.
- On 2nd-and-goal at the 1, Gordon catches a one-yard touchdown.
- As you can see, Josh Gordon gained 125 yards and a touchdown in just 3:37 of game time! That's phenomenal, but can you guess what kind of defense Pittsburgh was playing when they're up 17 and 24 points, respectively when those plays occurred?
- Gordon adds two catches for 33 yards on the final last ditch effort for a total of 158 yards in the last seven minutes. Should that be rewarded as much as gaining 158 receiving yards if the game is close? If they're trying to stop you? If the Steelers were the number one defense in the NFL? (Pittsburgh was 19th in Defensive DVOA, 19th against the pass.)
As you can see, Gordon had a solid game, it could be fair to to even call it "an outstanding display of athletic ability," but ultimately about two-thirds of his yards came when the game didn't matter anymore.
EDIT Aaron Schatz:
In addition, the yards coming when the game is out of reach isn't that big a deal. We compare every play to average -- and the average performance in a blowout isn't much difference than the average performance the rest of the time. Honestly, it's hard to say anything bad about that day against Pittsburgh. Gordon had 117 DYAR that day which is not only the highest DYAR for any WR in 2013 but is the 12th highest DYAR by any WR in a single game in the entire history of DVOA/DYAR going all the way back to 1989!
EDIT Kenneth:
Oops. There's really nothing I can do here short of striking out the last page of my article because I can't exactly just delete it. I charted Gordon's plays on the box score though and gave you a better understanding of how his 237 yards were accumulated, it just turns out that his game was as good as it looked. But Schatz did give me a much better example to use...
Schatz:
A better example of a game where FO stats and standard stats disagree would be Andre Johnson in Week 14 against the Raiders. It is really rare for a WR to have a 100-yard game and come out not only with negative DVOA but with negative DYAR. This was one of the rare ones.
Johnson had 13 catches for 154 yards in this game, and he even added seven yards on a DPI call. However, he was the target on 22 passes.
So you end up with a number of failed plays: 8 incompletes or interceptions (Johnson isn't blamed for the interceptions, they count as incomplete for him) four-yard catch on third-and-10 11-yard catch on third-and-20 (gets "partial" success because it improved FG position)
Then on top of that, the opponent was Jacksonville and they suck. Thus, negative DYAR. (-16.8% DVOA, -7 DYAR) Just like with Gordon, of course, the "you have to consider the QB" caveat comes in here.
Continuing with Gordon...
He was far more valuable the next week, when the Browns played the Jaguars and Gordon compiled 10 catches for 261 yards and two touchdowns. Not just because he had more yards, touchdowns, and a great YPC, but because of the situations in which he gained his yards. On 1st-and-10 from their own 5-yard line with a little more than four minutes to play and down two points, Gordon caught a pass that went 95 yards (the longest reception in the NFL last season) to give Cleveland the lead.
Unfortunately for Gordon he plays for the Browns, and they still ended up losing 32-28.
Every single play run in the NFL gets a "success value" based on this system, and then that number gets compared to the average success values of plays in similar situations for all players, adjusted for a number of variables. These include down and distance, field location, time remaining in game, and the team’s lead or deficit in the game score. Teams are always compared to the overall offensive average, as the team made its own choice whether to pass or rush. When it comes to individual players, however, rushing plays are compared to other rushing plays, passing plays to other passing plays, tight ends to tight ends, wideouts to wideouts, and so on.
So why should we care about a player or team's DVOA anyway? "That number don't make no sense to me!"
It's not just a "made-up" number... well, not exactly
It's been awhile since I've watched it, but I'm sure that Around the Horn still has the exact same format as when I did. Basically, it's a roundtable sports discussion show that's masqueraded as a game show. Four sports writers from around the country come in live via satellite to discuss the day's topics while Tony Bologna just arbitrarily assigns points to what they say.
If you just sit there and listen to what they have to say about said topics, you'll miss the part where Tony Bologna randomly assigns points...
Woody Paige: "I think Tim Tebow should be the quarterback handing out two million free pizzas, Tony!"
*boop* *boopboopboop* *boopboop* - 6 points
Michael Smith: "Woody, once again you don't know what you're talking about. Peyton Manning earned those pizzas."
*boopboop* *boop* *boop* *boop* - 8 points
Skip Bayless: "Tebowvich Tebowvich. Tebowvich. Tebowvich!!!!"
*boopboop*boopboop*boopboop - 12 points
It'll drive you crazy if you actually treat Around the Horn like a real game show. It's not like it really matters, you didn't tune in to see who won, it's just a matter of giving the host something to do and creating a "competitive edge" to a show about sports. It makes perfect sense, but if anyone thinks that DVOA or DYAR are arbitrarily-assigned numbers given to teams at random because "Aaron Schatz is a Patriots fan!" you're just wrong.
Kinda.
We've started to go over what a "successful play" is in the Josh Gordon example, but Football Outsiders just takes those successes and expands upon it by assigning specific points to specific plays:
-A successful play is worth one point; an unsuccessful play, zero points with fractional points in between (e.g., eight yards on third-and-10 is worth 0.54 "success points").
-Extra points are awarded for big plays, gradually increasing to three points for 10 yards (assuming those yards result in a first down), four points for 20 yards, and five points for 40 yards or more.
-Losing three or more yards is -1 point.
-Interceptions occurring on fourth down during the last two minutes of a game incur no penalty whatsoever, but all others average -6 points, with an adjustment for the length of the pass and the location of the interception (since an interception tipped at the line is more likely to produce a long return than an interception on a 40-yard pass).
-A fumble is worth anywhere from -1.7 to -4.0 points depending on how often a fumble in that situation is lost to the defense -- no matter who actually recovers the fumble.
-Red zone plays get a bonus: 20 percent for team offense, five percent for team defense, and 10 percent for individual players.
-There is a bonus given for a touchdown, which acknowledges that the goal line is significantly more difficult to cross than the previous 99 yards (although this bonus is nowhere near as large as the one used in fantasy football).
It's really not that complicated when you break it down like that. At least, no more complicated than a game of D&D (as I understand it.)
(Nerds!)
Perhaps the real issue with DVOA is that people are so unfamiliar with it that the number itself becomes inconsequential. Instead, the only thing that people can even start to wrap their minds around in regards to DVOA is one thing:
- Where do you rank?
I would never have a debate with someone in which I simply said, "The Seahawks had a DVOA of 40.0% last year!" because that doesn't even really mean much to me. What is "40%?" other than the being the odds that I'll die alone? (psych, far too low!) You might as well say that the Seahawks had a DVOA of apple. Or the Seahawks had a defensive DVOA of ^u^u^u^u^. Or the Seahawks had a blurquat of meeshmosh.
What we're left with instead: Seattle ranked first in DVOA last year and also ranked first in 2012. They're one of the only franchises in Super Bowl history to do that.
"Bzzahhh?! What's DVOA?! TELL ME MORE I LIKEY!"
But the number does mean something. It wasn't assigned by Tony Bologna; though the point totals themselves were kind of arbitrary, they're also consistent. They were made at one point, tweaked a little bit as time went on, but at this point when you're comparing Team A to Team B, even if those teams played 20 years apart, it's the same number.
Honestly the comparison for team-to-team isn't really that much different when talking about yards or talking about DVOA. It's easier for me to understand what "25 points per game is" or "400 yards per game" but I'm not even sure I'd know whereabouts that would rank on a given year. "It's a good number of points and a lot of yards, I'd say." Or maybe it's not a lot of yards because it seems like Matt Cassel had 5,000 passing yards last season.
And therein lies the rub, because it is easier to understand individual totals:
- 1,000 yards receiving
- 100 catches
- 1,000 yards rushing
- 10 rushing touchdowns
- 2,000 total yards
- 4,000 yards passing
- 30 passing touchdowns
Forget the fact that those "milestones" are completely arbitrary themselves, it's something that we've been able to wrap our brains around. Saying that Marshawn Lynch had a DVOA of 5.9%? Not so much. Trying to explain to a throng of Seahawks fans why Lynch ranked 17th in DVOA and please-don't-run-away-hey-come-back?
Not so much fun.
At the end of the day, most fans, writers, players, coaches are still saying "Seattle ranks first in points allowed, yards allowed, interceptions" and what have you. It's always judging yourself against the competition. That's what the very basic statistics that have been around for about as long as the game has, are meant to do: Judge you based on your competition.
And DVOA doesn't do...
Wait, yes it does!
You actually can start to get an idea of what the number itself means, quite easily!
I've already told you that last season the Seahawks had a total team DVOA of 40%, which to most of us means about as much as Bill Cosby having a stroke. (beezles-and-da-bozzles-and-call-Camille-gurbble-waaahhhhhh)
Well DVOA, unlike total yards or total points, is not a counting stat. In small samples, it's misleading. Much like a Billy Volek end-of-season hot streak, you don't want to take a player's DVOA in a few games and say, "He's amazing!" or "He's terrible!" Small sample sizes are a concept used a lot more in baseball than in other sports, but even though there are 1/10th the number of NFL games as there are in the MLB, there are still small sample sizes.
Best example ever: David Tyree.
Instead, DVOA is a measurement against zero, with zero being league average. If you know that 0% is where you'd be if you were a perfectly average team or player, then you can start to grasp what "40% means." Especially if you read this:
For teams, DVOA is normalized so that league averages for offense and defense are 0%. (However, because pass plays are more efficient than run plays, league averages for team passing and team rushing are not zero.) For players, DVOA is normalized separately for individual passing, individual rushing, and the three individual receiving groups (wide receivers, tight ends, and running backs) so that the league average for each is 0%.
Of course, one of the hardest parts of understanding a new statistic is interpreting its scale. To use DVOA, you have to know what numbers represent good performance and what numbers represent bad performance. We’ve made that easy.
In all cases, 0% represents league-average. A positive DVOA represents a situation that favors the offense, while a negative DVOA represents a situation that favors the defense. This is why the best offenses have positive DVOA ratings (last year, Green Bay led the league at +33.8%) and the best defenses have negative DVOA ratings (with Baltimore number one in 2011 at -17.1%).
In most years, the best and worst offenses tend to rate around ± 30%, while the best and worst defenses tend to rate around ± 25%. For starting players, the scale tends to reach roughly ± 40% for passing and receiving, and ± 30% for rushing.
Last season, the Broncos had a total offensive DVOA of 33.5%. Manning's DVOA was 43.2%, but Nick Foles still came in second at 35.6%. Whereas his total numbers skewed down a bit because he didn't play all season long, his DVOA accurately represents how valuable he really was over those starts. (So far the third round of the 2012 draft has yielded a QB that just threw 27 TD/2 INT, and that other guy I think won a Super Bowl? Nahh.. can't be true!)
On defense, the Seahawks posted a -25.9%, with a -34.2% against the pass. All of which falls in line with what FO said you could expect, but what's important isn't just Seattle's DVOA, it's how far away second-place was:
The Cardinals were next at -16.4%. The Seahawks were that far away from the rest of the competition.
Speaking of transitions...
VOA just wants the 'D'!
DVOA stands for Defense-Adjusted Value Over Average. So far we've given you the value -- a point-system based on down, distance, situation, yards, gained and more on a play-by-play basis -- and the 'Over Average', but what about the D? Really, adjusting for defense is something that no other simple metric can give you.
And yet it's one of the most important aspect of any sport.
On October 6, the Colts beat the then-undefeated Seahawks 34-28. If that doesn't impress you, it should. That was 10 more points than any other team scored on Seattle all season long, including playoffs. It's 20 more points allowed than their 2013 average. I have to give credit where credit is due, because Andrew Luck and T.Y. Hilton did against the Seahawks something that basically no other team or duo could do.
They made the Seattle defense and the secondary look bad.
Therefore, should we not give them more credit for that than say, putting up 34 points on the Chargers?
Every team in the NFL either benefits from or is at the mercy of their own schedule. It's just a natural part of the game that a team either overcomes or takes advantage of or maybe it's not really a big deal, but if you were Manning would you rather play in the AFC West or the NFC West? Would you rather play six games against the Raiders, Chiefs, and Chargers, or the Cardinals, 49ers, and Seahawks?
Would you rather see your receiver covered by Richard Sherman or by.... or by... or by... (hey, who's a corner in the AFC West??? Who?! Didn't he retire? What?!) or by Charles Woodson? Well, DVOA is here to help normalize what the numbers actually mean, based off of the defenses (or in the case of a defense, the offenses) they have to face.
The biggest variable in football is the fact that each team plays a different schedule against teams of disparate quality. By adjusting each play based on the opposing defense’s average success in stopping that type of play over the course of a season, we get DVOA, or Defense-adjusted Value Over Average.
Rushing and passing plays are adjusted based on down and location on the field; passing plays are also adjusted based on how the defense performs against passes to running backs, tight ends, or wide receivers.
Defenses are adjusted based on the average success of the offenses they are facing. (Yes, technically the defensive stats are actually "offense-adjusted." If it seems weird, think of the "D" in "DVOA" as standing for "opponent-Dependent" or something.)
There are many factors that go into a player or team's final yards total. All that DVOA does is try to explain those factors and break them down for you in one simple number.
'VOA' or 'YAR' (which we'll explain later) is the players value without adjusting for defense. Since we can't really know much about a defense in Week 1, the 'D' is added later, which I believe is Week 5. It continually adjusts throughout the year as we get more and more data on the defenses. Eventually we also get Weighted DVOA, which accounts for the fact that teams change even as the season goes on, so it weighs the most recent eight games more than any games before that. By Week 17, your Week 1 win over Baltimore isn't quite as significant.
And that should be accounted for.
Sum
So there you have it. Defense-Adjusted Value Over Average, or DVOA, is nothing more than a number that looks at every single play in the NFL season and assigns it value based on situation. There is a point-system based off of those values and then they are simply all added up and compared against one another, giving you a percentage that represents how far away you are from 0, whether that's in the positive or in the negative.
That's it.
The single-most difficult thing about football stats, and FO acknowledges this is in their DVOA breakdown, is that we still don't have a good way of knowing how much value lies with one teammate and how much lies with another. When Josh Gordon had those 237 yards against the Steelers, he not only accumulated all of his yards when the game didn't matter, he also gained most of his yardage that day when Weeden subbed in for Campbell.
What does that mean? Something or nothing? How much added value did Weeden get to his awful resume that day just because he had Gordon to throw to? How can we separate not only QB from WR, but QB from blocking? Or the blocking that a guard does from the blocking that the tackle and center next to him do? At this time we can't, but that's not a fact that devalues DVOA.
That's a fact that somewhat devalues every stat, from one yard to one percent, in the most team sport there is.
But at least we can say that we know what that stat is now and knowing is at least 50-percent of the battle. G.I. Joe didn't die for this country in Korea for nothing.
Final EDIT from Schatz (for now):
Also, let's be honest, one of the issues with DVOA is that it is proprietary. I have to calculate it, nobody else has the equations and baselines. It's not the kind of thing that's easy to add up from a box score even if I made all that stuff public. I hope that helps clarify some of the stuff you wrote. Honestly, trying to make this stuff easy to understand is one of the hardest parts of my job. The fact that you didn't quite get everything right, and you're a hardcore FO follower, is just an example of how we still need to do more to make this stuff clear to everyone.
There's plenty of hope for the future!