Don’t hate the list. Hate the model.

(In which Evan actually writes more than 140 characters about something all in one place!)

Got up early this morning (6:00 am) and walked over to the bus stop as my normal routine dictates, and I’m greeted with this tweet:

If you haven’t read about my ezPM metric, you can do so here (when I introduced the model) and here (when I updated it to use play-by-play data) and here (when I added counterpart rebounding to the model) and here (when I added counterpart defense to the model). I developed it several years ago after a long bout of Wins Produced fatigue.

Now, Kristofers who sent me that tweet was clearly looking at the results of my latest ezPM rankings. I could be wrong, but I highly doubt he has read each of the posts I just linked to in the previous paragraphs. Why do I doubt this? Because if he had, surely, he wouldn’t be asking me why Harden’s defensive rating is much higher than Steph or higher than Klay. It’s in the model!

I mean, unless he thinks I’m doing the math wrong, somewhere in the model, the points Harden has accumulated is dictated deterministically (it’s not a stochastic algorithm).

This brings me to the larger point of the post *slash* rant. Human psychology being what it is, when we are presented with lists of any form, the first thing we tend to do is find *outliers*. I don’t know why. Ask Kahneman. But that’s what we do, right? We don’t go down the list looking for things we agree with. We go fishing for things we disagree with so we can call out the list maker. And these days, preferably on Twitter.

I would argue that there are some lists where this is appropriate behavior. When the list is a purely subjective one, with no underlying model, at least, no rigorous one, then we generally have no choice but to argue or question the brain of the list maker. In there lies the only model we can point to.

But when a list is purely objective, such as my ezPM rankings, don’t question me personally about it on twitter. Question my model. Please! I actually encourage you to question my model. Or any model at all, if you think the output is questionable. That’s how models get improved.

When I thought there were issues with how WP ranked players, my first reaction, as most, was to criticize the list itself. But in time, I realized the only meaningful way to handle my criticism of the list, was not to attack it directly. It was to question and eventually attack the Wins Produced model, itself. And then to improve on it by creating my own.

So remember. Don’t hate the list. Hate the model. Everyone, including you, will be better off for it.

1 thought on “Don’t hate the list. Hate the model.”

  1. The posts you link to have a pretty large number of comments, so I may have missed it, but one (admittedly marginal) thing that I’d like clarification about is how you handle possession after blocks.

    I saw some discussion about the handling of foul shots in the 1.0 comments, and it’s something that could be broken out a bit more extensively since there is rebounding after the last foul shot (except for technicals.) It seems inconsistent with the way the rest of the model is built.

    More fundamentally, what kind of question is this model supposed to answer? That is to say, if we could run some kind of experiment to, say test this model against WP (or some other metric), what would that experiment look like?

Leave a Reply