Learning From Mistakes That Were Made (By Me)

This post has been in the back of my mind for a while, and I couldn’t make it go away. And what better time to dust it off than for my first post of 2011? Although I’m just a hobbyist with too much time on my hands (I still can’t believe the Daves fell for it), I feel some responsibility to own up to when I’ve been wrong. Naturally, a full listing of stuff like that would go on far too long for a blog posting. Moreover, when dealing with stuff like statistical projections, one expects to turn out to be wrong a lot of the time (the trick is to be right, or at least “on the right side” a bit more often than not). Therefore, I’m not going to focus on posts I did that relied on projections that turned out to be wrong, but rather on three particular posts that I think were flawed in ways that I should have seen at the time. In that way, I can do more than just say “well, I guess that projection was way off” and actually take some lessons away from my mistakes.

1. ‘Garrett Atkins to the Orioles’, December 18, 2009. Going way back into the past, in this post I argued that the Orioles giving Atkins $4.5 million dollars guaranteed was okay because it was about his market value as something like a one win player. Of course, Atkins performed much worse than that, the dollar value of a win turned out to be much less than that once the free agent market settled, and the Orioles did not yet know that they would be in a position to sign Miguel Tejada at a relatively low price to play third base. Still, I should have been critical of the signing because it still really didn’t fit the Orioles. While given the projections and what we knew of the market at the time, the signing was “average” in the abstract, the actual situation showed something different. The Orioles weren’t going to contend in 2010, so there was no real advantage point in spending market value for a player who is a part-time/platoon player at best, Atkins was old enough that there really wasn’t enough upside, and finally, the Orioles already had a first baseman disguised as a utility guy by the name of Ty Wigginton at the time. The lesson learned (and I hope applied since then) was that I should going beyond just looking at how the wins/dollars match up and take more careful notice of the overall organizational context.

2. ‘Jeremy Guthrie as Baltimore’s Bright Spot,’ September 17, 2010. Back to Baltimore for the scene of the crime. In his post I basically wrote that Guthrie wasn’t as good as his ERA given his FIP and other peripherals. A number of people really disliked the post, and I can understand why they didn’t. It simply lacked in subtlety. I generally agree with Dave Cameron on why FanGraphs uses FIP for WAR, but in this post I focused so heavily on defense-independent stats that I didn’t sufficiently acknowledge how much we don’t know about pitching, such that that Guthrie might have the “knack” for outperforming his FIP. A better finish might have been to point out that while I wasn’t sure if he could continue to do so or not, that he makes an interesting case, rather than my more negative conclusion. The specific lesson is a reminder to be more careful when employing DIPs theory to analyze a pitcher. The more general lesson is to not be afraid to acknowledge areas of uncertainty and for further research, and to be willing to give open-ended conclusions to posts.

3. ‘Is a GM Gap Behind the AL/NL Disparity?’, January 4, 2010. Oh boy, this will get some laughs, and rightly so. Honestly, I think the overall idea was and is a good one. There is a talent gap between the AL and NL favoring the AL, and I do think part of it is that AL teams generally are run more intelligently (even if a bigger factor, as Rob Neyer pointed out, is that they also have more money as a group). However, the actual execution of the idea, which involved listing whom I then thought were the five best and worst general managers in baseball, was a bit lacking. First, as Neyer pointed out in the post linked above, focusing on the general managers as individuals rather than organizations as a whole (including the whole front office and ownership) is overly simplistic.

Second… well, just look at the “best and worst” lists and I’m sure you’ll have some laughs. I won’t get into defending or attacking each choice, even if people generally thought differently of Brian Sabean (to list just perhaps the most glaring example) prior to the season rather than after his team won the World Series. The one that really stands out to me for consideration now is putting Jack Zduriencik as one of the best. Again, this is a self-criticism based not on what turned out to happen, but on the information we had at the time. And at the time, I liked most of Jack Z.’s moves and the direction I thought he was taking the Seattle organization. But it was still problematic to put him in the “Top Five,” and would have been even if the Mariners had turned out to win 90 games in 2010 (something which I did not, by the way, expect). This isn’t a continuation of the #6 ORG discussion (go here, here, here, and here for that), but a discussion of where I went wrong in my listings, and the lesson here is that I forgot to apply one of the most elementary sabermetric (and statistical) principles: small sample size. I don’t mean in a literal mathematical sense, but in the more general sense that we had just a bit more than a year of Jack Z. and the rest of the new Seattle front office’s work on hand to judge him by. Even though I liked most of what I’d seen thus far, we’ve seen a lot of new GM’s come in and start making good moves that they’ve probably been thinking of making for a while in their previous positions, only to watch them flounder after a couple of seasons. I’m still not sure what will end up happening in Seattle or how good or bad Zduriencik and his staff are, and that’s the point: there wasn’t enough information, so if I was going to make such lists, Jack Z. shouldn’t have been on it. Small sample size even applies to fat front office guys (rhyming = great writing).

Whew. Hopefully this narcissistic exercise was a bit informative and entertaining. Again, keep in mind that this wasn’t meant as an exhaustive list of my mistakes, a list that will inevitably keep growing. To borrow a motif from Bill Simmons (boy, I hope I have enough “honor” to type this): the lesson is that I’m an idiot.