In-play average, the aftermath, part 1
 
In the week and a half since we posted our study on the ability of pitchers to prevent hits on balls in play, we've received some very thoughtful email on the subject, and the Baseball Primer web site has a long discussion thread on the topic.

(I don't always have time to keep up with what's being written on Baseball Primer, but when I do, I usually learn something. As with any internet forum, the quality of the posts isn't uniformly high, but there are a number of people doing some very interesting work over there.)

For those of you who are interested, here's a summary of what I've read ...

1. A few Baseball Primer folks chastised me for failing to reference a follow-on article (called "DIPS Version 2.0") that Voros McCracken posted to that site in early 2002. I did make a point of looking for other articles of his, and when I found the Voros home page on www.baseballstuff.com, I thought that was a complete list of his work on this subject. Well, it wasn't, and I wish I had seen the Primer article before publishing mine.

In that follow-on article, McCracken made two points that are consistent with my findings. First, he acknowledged that there are small differences between pitchers in their ability to prevent hits on balls in play, and those differences "appear to be statistically significant if generally not very relevant." He also pointed out that knuckleball pitchers are particularly successful at preventing in-play hits.

As a result, I've made a couple of edits to my article to make it clear that McCracken had made these two observations long before I did my research. (He also had other things to say about fly-ball versus ground-ball pitchers, left-handers versus right-handers, height, and a few other topics, but the two comments I mentioned in the last paragraph are the ones most relevant to my work.)

By the way, I learned this morning that Keith Woolner of Baseball Prospectus published an article shortly after the first McCracken piece appeared. Keith wrote to tell me that his findings were consistent with mine, or maybe I should say that the other way around, since his article appeared two years earlier.

2. A few people claimed that the low correlation of IPAvg from year to year proves that in-play performance is irrelevant in the evaluation of pitchers and the projection of future performance.

These folks point out that McCracken got better results in projecting next-year performance if he used the team's IPAvg from the previous season than if he used the pitcher's own IPAvg.

I haven't done that sort of projection work myself, but I'm willing to accept it. There is a lot of noise in a pitcher's IPAvg, and if a method puts no weight on a pitcher's IPAvg, it's not unreasonable to expect it to produce better projections than a method that gives full weight to his prior-year IPAvg.

But what if we put some weight on a pitcher's IPAvg? What if we could find a pitcher's true level of in-play ability? That may not be easy to do, but it may produce better projections than other systems that are being used today.

In particular, a projection system that uses the team's IPAvg assumes that pitchers with lousy IPAvg results will improve to match the team's IPAvg. But what if the career-length breakdown proves that some of them simply aren't good enough? Wouldn't an approach that assumes all pitchers are equal on IPAvg overestimate their value? Maybe that wouldn't matter to folks who use projections like this for fantasy leagues, because these guys aren't likely to be drafted anyway, but wouldn't that be very important to a big-league front office?

Finally, if pitchers do have some control over in-play outcomes, it casts doubt on the use of team IPAvg as a pure measure of the impact of the park and the defense. Team IPAvg also measures the quality of the pitchers on that staff to some degree, so it has less value in projecting the future performance of individual pitchers on that team.

3. A couple of people attacked my career-length charts as meaningless because (a) after 2000 batters faced the differences are small and (b) there's a selection bias because the guys with short careers were probably just unlucky on balls in play and were out of the majors before they could bring their IPAvg numbers back to the average.

I'm sure there are some pitchers in the short-career group who lost an opportunity for a longer career based on bad in-play luck, but it's a huge (and unsubstantiated) leap to assume that most of them were just unlucky. Those pitchers were also worse than average on walks, strikeouts and homers, so it's likely that they just weren't good enough to get their fair share of easy outs on balls in play.

4. A couple of people have noted that homeruns are balls in play and shouldn't be set aside as they are in McCracken's methodology. I understand completely. After I had crunched all the numbers, I wished that I had looked at in-play average with and without homers included.

It made sense to leave them out because my plan was to test McCracken's idea against a larger dataset, so it was important to apply his method as it was originally laid out. And that method makes sense: McCracken's goal was to isolate events for which the pitcher has sole responsibility, and the defense has very little involvement in the vast majority of homeruns. On the other hand, we might be able to learn something else by adding them in, so both approaches have merit.

In particular, as I was writing the article, I found myself wondering about the interaction between strikeouts and homeruns. McCracken removed strikeouts before computing IPAvg, but didn't remove them before computing HR rates. Isn't it possible that Roger Clemens' HR rate looks good only because he strikes out so many hitters? Maybe his HR rate is merely average when computed as a percentage of balls in play rather than a percentage of batters faced?

It turns out that it's not average, but it is closer to the average. If we assume that the norm was about 25 homers per 1000 batters faced during Roger's career, his surplus of 1355 strikeouts may have saved him about 34 homers. He was 101 homers better than average, so the strikeouts account for about 1/3 of his success in this area. I looked at a few other pitchers and found that, in general, the strikeouts accounted for only a small fraction of their net homerun savings. Still, it would be interesting to look at homers both ways for a much larger group of hurlers.

5. Others have pointed out that when a pitcher makes a lousy pitch that is crushed for a homer, his IPAvg doesn't suffer. But if he makes a slightly better pitch and gives up a double off the wall, his IPAvg does suffer. This may be one reason why differences in IPAvg are relatively small.

Hits on balls in play are intermediate results. Make a great pitch and you get a strikeout. Make an okay pitch and you produce a ball in play. Make a terrible pitch and you give up a homer. IPAvg excludes the best and worst possible results, so it may make pitchers look more similar in that respect.

6. Some suggested that we look at the percentage of in-play hits that go for doubles and triples to see if there are more striking differences among pitchers than when you simply compute a batting average. I agree that this would be worth doing, though we'd have to be very sure to account properly for park effects.

7. Others suggested that we look at GDP rates to see how different pitchers are in that respect. I had the same thought when I looked at the Tommy John career profile, but it was too late to go back and run all the GDP numbers at the time.

8. I think it would be interesting to apply this method to hitters. I haven't done that, and I haven't seen any articles on that subject, but I've heard that others (McCracken among them, I believe) found large differences among hitters when measured by IPAvg. That wouldn't surprise me.

9. I mentioned this briefly in the Greg Maddux comment, but it's possible that the pitcher's own defensive ability has something to do with his IPAvg. I don't think that would change the outcome of the study, but it's worth investigating.

10. Mike Emeigh made the very interesting point on Baseball Primer that differences among pitchers on the same staff may be affected by which parts of the defense they test. As an extreme ground-ball pitcher, Derek Lowe's IPAvg has a lot more to do with infield defense than outfield defense. McCracken's method evaluates pitchers relative to the team's overall IPAvg, so all pitchers on the staff are assumed to get the same amount of help from their defense.

11. It's interesting how the same words can cause very different reactions in different people. I received two email messages from people who said my tone was condescending toward McCracken's work. The same day, I received messages from two others who commended me for going out of my way to praise McCracken's contribution. Meanwhile, a few posts on Baseball Primer described McCracken's original article as condescending, but I certainly didn't feel that way when I read it, and I don't think it's fair to describe it that way.


OK, that's enough for now. Some of these questions represent interesting research projects. We'll be looking at the projection system implications before we release our 2004 Projection Disk in the spring, but I'm not sure when we'll have time to work on the others. Some of you may do this before we can get to it, and if so, I look forward to seeing your results.

Finally, I want to say something about the more extreme reactions to the article. Some McCracken supporters dismissed this research entirely and seemed offended that anyone would dare challenge their hero. Some McCracken skeptics went out of their way to praise me for refuting all of this DIPS nonsense and delighted in the fact that we wouldn't have to talk about it any more.

Neither view is anywhere close to accurate.

To those who say that I refuted McCracken's work, I say "hogwash". It's true that I came away believing that pitchers do influence in-play outcomes, and it's true that I presented some contrary evidence regarding some specific comments in his first article. But my work fully supports several of McCracken's most important observations: that pitchers have much less influence than most of us believed, that there's a lot of randomness in single-season results, and projections of future performance need to take these two things into account.

To those who say I wasted my time because McCracken had already said, or at least hinted at, everything I wrote, I say "hogwash". I believe there are some original findings in the article, but even if you don't agree, it was worth making the effort to see how the ideas would hold up when applied more broadly.

I'm sure we haven't heard the last on this subject. That's one reason why the title of this blog entry says "part 1". In addition, I plan to write about IPAvg in the coming days and weeks, in at least the following ways:

- if any of you would like to see career charts for pitchers not covered in the article, let me know, and I'll see if I can work them up

- in our work on version 9 of the Diamond Mind Baseball game, we've added IPAvg as a stat that can be included in various batting and pitching reports, and we'll be using those reports to identify and write about notable IPAvg performances (projected versus actual, 2002 versus 2003, and whatever else seems interesting).

posted by Tom at 11:12 AM