Imagine you and your friend sat down to discuss a simple basketball question:
Who is better—Carmelo Anthony or LeBron James?
Basketball fans could spend hours debating a question like this. But let’s say you invited me to come along for the discussion. At first, this might seem like a good idea. My co-authors and I have published dozens of academic papers studying the economics of sports, many of which are focused specifically on professional basketball. Studies examining basketball often require measures of player performance, and such measures might seem fairly helpful if one wished to evaluate the merits of Melo and LeBron.
However, inviting me to the conversation creates a very big problem.
But before we get to the details of this problem, let’s briefly discuss what is meant by “measuring player performance.”
Players take various actions on the court (scoring, rebounding, etc.) For our research, my co-authors and I needed a method that could connect these actions to team wins. Existing measures—like Player Efficiency Rating, Player Impact Estimate (found at NBA.com), or Win Shares—fail to empirically justify the weights employed in their statistical evaluations of players. The basic question—what is the value of a rebound? (or steal, assist, etc.)—is not actually answered by these measures. Consequently, you can’t claim that these metrics actually measure a player’s productivity.
The failure of these box-score measures has led some people to turn to plus-minus models. However, these models are often based on suspect empirical methods, or offer suspect empirical findings. Also, they are also not capable of telling us why a player is productive. So these measures don’t work very well, either.
Given the problems with existing measures, I developed the Wins Produced model. This model uses standard econometric techniques to measure how many wins the statistics tracked for the individual player are worth.The purpose of this approach is to address such topics as the efficiency of decision-making, the impact of contract status on productivity, the role managers play in the performance of workers, and the existence and extent of racial discrimination. But the model can also be used to determine how Carmelo Anthony compares to LeBron James.
So let’s say you and I decided to discuss this issue. And let’s imagine you were a huge Melo fan. You might start the conversation by talking about Melo’s ability to score, and/or the failures of his teammates.
And then I would say something like this:
“Prior to this season, LeBron James had produced 182.2 wins in his career and his career Wins Produced per 48 minutes (WP48) was 0.265. In contrast, Carmelo Anthony — before this season — had only produced 37.1 wins with a career WP48 of 0.062 (average WP48 in the league is 0.100). Therefore, LeBron is immensely more productive.”
We could go on for a bit. I could note—as I did at the Atlantic last May—that Melo’s teammates have actually produced more wins than LeBron’s teammates. And I could add that scoring totals are not useful in measuring a player’s impact on wins, since a relatively inefficient scorer can score more points by simply taking more shots. Since inefficient shooting does not help a team win games, inefficient scorers are not that valuable.
You could respond by saying that you don’t like how I am measuring a player’s contribution to wins. You could say I don’t take into account interactions between teammates (although I do). Or that I don’t consider coaching (which I have). Or I ignore team defense (which I don’t). But if you take this approach, you are no longer discussing the merits of two players. You are now debating research methodology. And very few people actually want to spend hours in such a discussion.
This is why a model like Player Efficiency Rating never disappears. As I noted years ago, PER is simply not a very good measure of player performance in the NBA. And I have repeated and expanded upon this observation over the years. But explaining to most people why this model has problems is not a useful exercise (and again, I have tried for years). Most people have no idea which research methods are good or bad. Furthermore—and this is the most important factor—they really don’t care.
What they really want to do is debate sports. And statistical analysis simply ends that debate.
All of this reminds me of a scene from the classic book by Douglas Adams, The Hitchhiker’s Guide to the Galaxy. A computer is about to give the answer to “Life, The Universe, and Everything.” Right before the computer is turned on, philosophers break into the room, and argue that the computer is about to put them out of a job. As the philosophers put it,”what’s the use of our sitting up half the night arguing that there may or may not be a God, if this machine only goes and gives you his bleeding phone number the next morning?”
Sports fans are just like these philosophers. And statistical analysis is just like the giant computer in the Adams classic. Sports fans want to debate the relative merits of players. And stat analysis comes along and puts the sports fans out of work.
To illustrate the speed at which statistical analysis can work—and the amount of fun it can kill–imagine you and your friends wanted to debate the merits of the following players:
- Kobe Bryant or Michael Jordan?
- Magic Johnson or Larry Bird?
- Isiah Thomas or John Stockton?
- Charles Barkley or Karl Malone?
- Dennis Rodman or Dominique Wilkins?
If you went to boxscoregeeks.com (where one can find Wins Produced numbers back to 1977), or just clicked on the above links, you would see the following answers:
- MJ produced more than Kobe
- Magic produced more than Bird
- Stockton produced more than Isiah
- Sir Charles produced more than the Mailman
- The Worm produced more than the Human Highlight Reel
None of the comparisons are really the close. In each case, the player who is listed as more productive is much more productive.
None of this really helps a sports fan. Comparing players on subjective terms can lead to hours of lively back-and-forth. But once you turn on the computer, an objective answer is provided, and the fun ends. Talking sports is what fans love to do, and numbers kill the conversation.
So, if you have ever wished us stats people would go away…. that’s probably not going to happen. In my defense, I really didn’t set out to answer the sort of questions sports fans most frequently talk about. The Wins Produced model was designed to answer research questions in economics, but it also helps us answer the relatively trivial ones listed above. And when I see people struggle with these questions… well, trivial questions or not, researchers love to find answers. Unfortunately, I’ve come to think that answering these questions isn’t really helping. Sports fans want to enjoy the dialogue, but the answers from statistics end the debate, and kill the fun.
David Berri is a professor of economics at Southern Utah University. He was the lead author of The Wages of Wins and Stumbling on Wins, and has numerous academic publications on sports and economics. In addition, he is a past president of the North American Association of Sports Economists, and continues to serve on the editorial board of both the Journal of Sports Economics and the International Journal of Sport Finance. He has written for a number of popular media outlets, most recently Time.com. You may follow him on Twitter, even if you still think Carmelo is better than LeBron.