An APBR forum member (back2newbelf) has recently started publishing regularized adjusted +/- (RAPM) data. It's like +/- data, but on steroids. If you want to learn more about how RAPM works, in general, see the paper presented by Joe Sill at the MIT Sloan Sports Analytics Conference in March, 2010. I thought it would be useful at this point in the development of the ezPM model to compare 1yr and 3yr averages with back2newbelf's RAPM data. What follows are the results for regression of total RAPM on total ezPM100 (both metrics are per 100 possessions), with some tables of best/worst players by average of the two metrics, to give some idea of the actual numbers. In a subsequent post, I will perform the same type of analysis on the offensive and defensive components of the metrics.
1 Yr RAPM
The 1yr RAPM data set can be found here. I used a 1000 possession minimum as my cutoff, which left about 200 players to compare. Here are the results in graphical form, followed by regression data from R:
Call:lm(formula = RAPM ~ EZPM, data = data.1yr) Residuals: Min 1Q Median 3Q Max -3.2659 -0.8885 0.0009 0.8501 4.3308 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 0.20883 0.09066 2.303 0.0223 * EZPM 0.27176 0.03348 8.118 4.34e-14 *** ---Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 1.284 on 205 degrees of freedom Multiple R-squared: 0.2433, Adjusted R-squared: 0.2396 F-statistic: 65.91 on 1 and 205 DF, p-value: 4.335e-14
A few things to note here. 1) The regression result is highly significant (p=4.34e-14); 2) The slope of the regression is 0.27, which means that RAPM underestimates ezPM100, or put the other way, ezPM100 overestimates RAPM; and 3) ezPM100 explains about 24% of the variance (R^2=0.24).
Let's look at some of the best and worst players according to the 1-yr data by averaging the two metrics:
Top 20 Players by 1 Yr. Metric Average (ezPM100+RAPM)/2
Bottom 20 Players
Ok, both models are clearly wrong. Did you see that block J.J. Hickson made on Griffin last night?
3 Yr RAPM
The 3 yr. RAPM data can be found here. My data set goes back to the 2008-2009 season through the first week of February. As far as I know, the RAPM data weights each year equally, so I did the same to make the comparison fair. As before, a plot followed by numbers:
Call:lm(formula = RAPM ~ EZPM, data = avg.3yr) Residuals: Min 1Q Median 3Q Max -5.8064 -1.1009 0.0435 1.1527 5.9218 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 0.58592 0.10964 5.344 1.92e-07 *** EZPM 0.43908 0.03725 11.786 < 2e-16 *** ---Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 1.703 on 273 degrees of freedom Multiple R-squared: 0.3372, Adjusted R-squared: 0.3348 F-statistic: 138.9 on 1 and 273 DF, p-value: < 2.2e-16
As expected, there is an improvement in both the slope (0.44) and R^2 (~0.34). Here are the player tables:
Top 20 Players by 3 Yr Metric Average
Bottom 20 Players by 3yr Average (Or the List You Really Don't Want to Appear On)
This was definitely a worthwhile exercise. It's good to see how the ezPM model compares to RAPM. Of course, it should not be expected that the two models line up perfectly. That would be great, but in practice, we should be using multiple models to evaluate players. Some players may look better in one metric or the other. We should have more confidence in players that are highly rated by both an APM model and a box score metric, such as ezPM. For example, what I didn't show here are the players that were ranked in the top 20 by either metric alone. That would have showed that Derek Fisher is one of the best players in the league according to RAPM 1yr data (2.4), but not according to ezPM (-4.03). Kris Humphries looks great according to ezPM (4.52), but not RAPM (-1.4). (His new girlfriend always looks great!)
Anyway, this is a good stopping point, but also a good starting point. Going forward, we'll see if there are adjustments that can be made to ezPM that will make it even more consistent with RAPM. For example, why is Dirk rated so much higher in RAPM? Does it have something to do with usage? His teammates? It's also important to ask which model is a better predictor. If one or the other (or an average) is a better predictor, we probably want to know that, right? As always, to be continued...