Way back in 2006, David Berri, Martin Schmidt, and Stacey Brook came out with a book called The Wages of Wins. It was supposed to be Moneyball for sports other than baseball, but for a lot of people, the book read like Moneyball with a serious chip on its shoulder. In a team game with lots of variables, Berri and his co-authors were confident in their regression-based assertions that there were 90 players more valuable than Allen Iverson during the season that he won MVP, that scoring was vastly overvalued while rebounding was too often neglected, and Ray Allen had been just as good throughout his career as Kobe Bryant.
JC Bradbury and I – in a forthcoming article in the Journal of Sports Economics — report that only 7% of a player’s adjusted plus/minus is explained by what a player did the previous season (oddly enough, unadjusted plus/minus has a stronger – albeit still relatively weak – correlation). In other words, the correlation coefficient for adjusted plus/minus from season-to-season is below 0.30. And when we look at players who switch teams – as Songaila did – we fail to find a statistically significant relationship. In contrast, any measure (PERs, Wages of Wins measures, NBA Efficiency, Win Shares, etc…) based on the box score will have a correlation coefficient of at least 0.65, and often these marks are above 0.80.
Berri makes a solid point. He uses Darius Songalia as a case study for how inconsistent adjusted plus/minus can be, but he could easily have used Kevin Durant, who started the season as a posterchild for how plus/minus based stats could contradict box score metrics but is now an example of how elastic adjusted plus/minus can be from season to season.