So FIP is purported to be a pitcher's ERA with neutral defense and luck. We can easily convert that to an expected runs allowed for a season then. UZR is purported to be the number of runs prevented or allowed above average by a defense. So in principle we should be able to combine the defense independent pitching with the defense to get actual pitching results. I think all would agree that if FIP and UZR do what they are supposed to do then this would be the case. So do they do what they say they do?

I looked at the 7 complete seasons from 2002-2008 for which I had UZR data. For each team in a season I found how many runs they should have allowed according to FIP and the number of innings pitched. I then added in or subtracted the number of runs prevented/allowed above average according to UZR. FIP assumes average defense, so once we replace average defense with "actual" defense as measured by UZR, if the stats are any good then this new number should be a pretty good predictor of actual runs allowed.

Once I had these predictions, I found the R^2 value. Not to oversimplify things too much, but this value basically shows what percentage of the variation can be accounted for by the model. The value ranges from 0 (worthless) to 1 (perfect). For my 210 data points, I had an R^2 value of about 0.78 (i.e. 78% of the variation). If you want to put it in terms of R instead of R^2 it would be about .88. I'd say that is significant. So what of the other 22% of the variation? Well we know that UZR is ignorant to catcher and pitcher defense, and I believe there are still issues with park effects (I think UZR is park adjusted but not sure, but I don't think FIP is). Also FIP assumes pitchers control HR/FB ratio and it completely ignores the effects a pitcher has on BABIP (as opposed to just mostly ignoring them). There are of course other issues, but I think the .78 value is fairly significant and shows that FIP and UZR do a pretty good job. For what it's worth, FIP alone (with the IP adjustment) came in at about 0.69.

Data