Using Hollinger's Draft Rater to Make a Point About False Positives and Negatives and the Klay vs. Kawhi Question

Which one of these guys has a better chance of being a star?

That title was a mouthful, but hopefully I can make this post a bit more digestable. I should also note before getting into this, that ostensibly, this post is about Hollinger's draft rater and some statistical mumbo jumbo, but really, that's all a prelude to the following conclusion: The Warriors likely made a mistake by passing on Kawhi Leonard in favor of Klay Thompson. I should also note that you can easily find all the statistical mumbo jumbo on Type I and II errors on Wikipedia.

The point of a statistical test is to identify *real* trends, groups, effects, etc. For example, in medicine one wants to know whether a drug has a real effect. The results of a medical diagnostic test are supposed to tell us whether a patient really has *something* or really doesn't have *something*, with *something* being the thing that is being tested for obviously. If a patient really has something, an ideal test would tell us she definitely has that something we're testing for. Conversely, if she really doesn't have the something, then hopefully, our test will tell us she really doesn't have something.

But in the real world, diagnostic tests are not perfect. Sometimes, the patient really does have something, but the test says she doesn't. That is known as a "false negative" or a "Type II error". Of course, this is bad, because the patient might really have cancer, for example, but according to the test, she is perfectly fine. Conversely, the patient might be perfectly healthy, but the test result suggests she really has something. This is known as a "false positive" or a "Type I error". If you were a patient, even though this would be a scary initial result, more careful follow-up diagnostic would hopefully contradict the false positive.

We need a couple more pieces of jargon before continuing. Doctors want to avoid false negatives, because as I said above, if you have cancer and the test says you don't, that's a scary situation. Better to not have cancer but think you do (initially, anyway), than the other way. Or as Clarence famously said in True Romance, "It's better to have a gun and not need it, than to need a gun and not have it." (Is the gun the cancer or the test?) So, we'd probably rather our test give us a false positive than a false negative, and we'd definitely want it to give as many true positives as possible. There is a numerical way to define this, and it is called "sensitivity":

A test that is 100% sensitive finds all true positives and has zero false negatives. In other words, if a test is 100% sensitive and you really have cancer, the test will say you have it. If a test is 50% sensitive, then it will only give you the correct diagnosis half the time.

What about detecting true negatives? If you don't have cancer, a good test will say you are healthy, and not put you through needless further emotional turmoil (and further medical costs). That is called the "specificity" of the test:

Note how that definition is very similar to the previous one, but simply reverses the positives and negatives. If you really don't have cancer, a test that is 100% specific will tell you that you really don't have it, while a test that is only 50% specific will incorrectly diagnose you half the time.

Let's bring it back to basketball now. I'm going to make the argument that Hollinger's Draft Rating system is very similar to a medical diagnostic, or at least, should be treated as such. In this case the *something* we are looking for in a player is *NBA talent*. In that context, I would argue that Hollinger's rating system appears to be quite sensitive, but not extremely specific.

Given the definitions just presented, what that means is that his "test" usually identifies the players who are going to be stars, but also mis-identifies many players as stars, who turn out to be non-stars (or even non-starters or even non-NBA players in the case of Joe Alexander, for example). Putting this another, perhaps, more useful way, if a player is *really* going to be a star, Hollinger's test is almost always going to show that, and if he is definitely not going to be a star, Hollinger's test will almost always show that, too. There are certain caveats, like one and done players, but even there, it appears to be useful most of the time.

So, what does that have to do with Kawhi Leonard and Klay Thompson you ask? Here's a list that Hollinger presented of the wings with the highest ratings since 2002:

1. Kevin Durant 17.67
2. Dwyane Wade 17.05
3. Carmelo Anthony 16.63
4. Danny Granger 15.43
5. Rudy Gay 15.10
6. Luol Deng 14.46
7. Josh Childress 13.37
8. Kawhi Leonard 13.21
9. Mike Dunleavy 12.95
10. Dajuan Wagner 12.72

That's a pretty good list to be on, don't you think? Now, here's the ratings for perimeter players in the 2011 draft class:

1. Kyrie Irving 15.14
2. Kawhi Leonard 13.21
3. Kemba Walker 12.75
4. Tyler Honeycutt 12.56
5. Jordan Hamilton 11.90
6. Alec Burks 11.87
7. Klay Thompson 10.88
8. Norris Cole 10.85
9. Jimmer Fredette 10.45
10. Chris Singleton 10.15
11. Brandon Knight 10.02
12. Darius Morris 9.57
13. Brad Wanamaker 9.57
14. Reggie Jackson 9.45
15. Damian Saunders 9.20

Klay Thompson comes in at 10.88. So, the question is how many players with a rating this low become stars (or even good starters)? If there are plenty of those, than my claim that the test is very sensitive would be wrong.

Going back to 3 previous editions of his draft rater, here are the best guards/wings I could find with ratings lower than 11.0 (oh, and btw, Hollinger specifically states his system actually works best for wings):

• Armon Johnson (10.98)
• Jeremy Lin (10.87)
• Avery Bradley (10.42)
• Willie Warren (10.71)
• Jodie Meeks (9.35)
• Sam Young (9.34)
• DeMar DeRozan (9.26)
• Sonny Weems (10.76)
• Gary Forbes (9.49)

That's it. For reference, Reggie Williams had a 12.22 rating. Stephen Curry had a 14.18 rating. 

Now, you might say, well, Kawhi Leonard may turn out to be Josh Childress or Dajuan Wagner. True. But using this rating system, I would argue his ceiling is closer to Mike Dunleavy (easily better than anyone on the other list) or Luol Deng, whereas the best that we can hope for Klay Thompson is someone who is better than DeMar DeRozan or Jodie Meeks.

Maybe that is good enough for you. But I would have preferred to take a player who has a chance to be very good rather than a player who — at his best — is likely to be a fringe starter or a starter that nobody is really all that excited about having on their team.

I should end by saying, I truly hope that Klay Thompson will be the player that forces Hollinger to significantly overhaul his rating system and figure out how he missed so badly. But *that* would be a significant surprise, and should not be anywhere close to the expectation, based on the objective statistical analysis.