Way back in 2006, David Berri, Martin Schmidt, and Stacey Brook came out with a book called The Wages of Wins. It was supposed to be Moneyball for sports other than baseball, but for a lot of people, the book read like Moneyball with a serious chip on its shoulder. In a team game with lots of variables, Berri and his co-authors were confident in their regression-based assertions that there were 90 players more valuable than Allen Iverson during the season that he won MVP, that scoring was vastly overvalued while rebounding was too often neglected, and Ray Allen had been just as good throughout his career as Kobe Bryant.
The general feeling among a lot of hard-core basketball fans and analysts was that the Wages of Wins system, which relied only on box-score based statistics, couldn’t possibly accurately capture everything that made a player valuable in a five-on-five game. The logical extreme of that philosophy came in the form of Wayne Winston, the former stat guru for the Dallas Mavericks whose brainchild was adjusted plus/minus, which sought to measure a player’s value without using any box-score statistics whatsoever. As it turned out, he had some even more outlandish conclusions than Berri and co. did. He said that the Knicks should never have traded Tim Thomas, that Lamar Odom was better than Kobe Bryant, and that Kevin Durant wasn’t helping the Thunder win.
After last weekend’s Sloan stats and analytics conference, David Berri has a short post up
on adjusted plus/minus. Here’s the crux of Berri’s argument for box-score bases metrics over adjusted plus/minus:
JC Bradbury and I – in a forthcoming article in the Journal of Sports Economics
— report that only 7% of a player’s adjusted plus/minus is explained by what a player did the previous season (oddly enough, unadjusted plus/minus has a stronger – albeit still relatively weak – correlation). In other words, the correlation coefficient for adjusted plus/minus from season-to-season is below 0.30. And when we look at players who switch teams – as Songaila did – we fail to find a statistically significant relationship. In contrast, any measure (PERs, Wages of Wins measures, NBA Efficiency, Win Shares, etc…) based on the box score will have a correlation coefficient of at least 0.65, and often these marks are above 0.80.
Berri makes a solid point. He uses Darius Songalia as a case study for how inconsistent adjusted plus/minus can be, but he could easily have used Kevin Durant, who started the season as a posterchild for how plus/minus based stats could contradict box score metrics but is now an example of how elastic adjusted plus/minus can be from season to season.
I’m a big believer in using advanced stats to gain knowledge about basketball, but it appears that both Berri and Winston have holes in their metrics. Berri’s box-score based metrics don’t necessarily reflect who was doing what helped his team win the game. For example, let’s say Matt Barnes plays great defense on Kobe for 20 seconds and forces him into a tough fadeaway. Dwight Howard then blocks out Pau Gasol and keeps him from getting to the rebound. The ball caroms off the rim and goes to Vince Carter, who collects the easy rebound. In Berri’s system, only Carter gets credit for doing something right on that play.
Winston’s system would theoretically give Barnes and Howard most of the credit for the play above. However, the issue is that they could have radically different roles on a different team. With another team, Barnes might not be a starter or a perimeter scorer, but a stretch four who provides energy and outside shooting off the bench without giving much on the defensive end. Thus, he could have a radically different value with a different team.
Advanced statistics in basketball are wonderful, but they are far from airtight. For the foreseeable future, the best approach with advanced statistics will be to use a number of different metrics and see how they inform each other rather than wait for one perfect formula to reduce contributions to a single integer.