Visualizing 2015 Mookie Betts vs. 2015 Javier Baez

Earlier, I asked you to participate in an exercise projecting both next year’s Mookie Betts and next year’s Javier Baez. The idea is that Betts seems representative of a particularly safe prospect, while Baez represents something of a more volatile asset. I promised that I would analyze the results given a sufficient sample size of votes, and, such a sample size has already been achieved. Interestingly, as of right now, there have been three more votes in the Baez poll than in the identical Betts poll. The best possible conclusion is that three FanGraphs readers had their browsers lock up at a most unfortunate time. The worst possible conclusion is chilling indeed.

So I think it’s safe to move forward with a little analysis. Before getting there, I hope you understand that *I* understand that I didn’t conduct this exercise perfectly. Nevermind the wisdom of the exercise in the first place; all my words might’ve biased the voters to some degree. I could’ve written nothing, or I could’ve at least put the polls before the words. But, what’s done is done. Also understand that, while you’re going to see a measure of uncertainty, this is perceived uncertainty, and not actual uncertainty. We can’t know actual uncertainty. We’re just going to go ahead and pretend like what we think is a decent proxy for what actually is. Let’s see how the community feels about Mookie Betts and Javier Baez, for 2015.

First, the distribution of votes, for Betts’ offense:

The most popular choice was a wRC+ between 110 – 119. That seems appropriate enough; Steamer’s projection has Betts at 118. The community, as a whole, has projected Betts for a similar 114 wRC+, if you allow the assignment of specific numbers for each individual vote based on the buckets (that is, we can estimate that a vote for 110 – 119 is essentially a vote for 115). You see a small number of votes in each of the most extreme buckets, and to some degree that reflects people messing around, but the effect is small — that’s just about 2% of the voting total. The three most popular buckets captured 82% of the votes.

Now, here’s the distribution of votes, for Baez’s offense:

The most popular choice was a wRC+ between 90 – 99. Steamer projects Baez for a straight-up 90, so, there you go. It’s interesting to note that the community has projected Baez for a 100 wRC+. Steamer projects a gap of 28 points of wRC+, while the community has cut that gap in half. An unintended effect might’ve come out of specifying a hypothetical 600 plate appearances; some readers, upon seeing that, concluded that Baez wouldn’t get 600 plate appearances if he were miserable. That’s true, but that’s not what I was trying to ask, and that’s another mistake that I made. No choice now but to deal with it. The three most popular buckets captured 72% of the votes. This is one simple indicator that Baez is indeed perceived to be more difficult to project.

Now then, a direct comparison. Here you’ll see error bars, representing +/- one standard deviation based on the voting. These errors bars capture about 68% of the projected outcomes. Extending to two standard deviations would capture about 95% of the projected outcomes.

As noted before, Betts is projected for a mean 114 wRC+. Baez is projected for a mean 100 wRC+. The assumption was that Betts would end up with a smaller standard deviation, and that is what we observe. One standard deviation for Betts comes out to 12.6 points of wRC+. Two standard deviations, then, comes out to 25.2. One standard deviation for Baez comes out to 16.0 points of wRC+. So two standard deviations comes out to 32.1.

Expressed differently:

Betts

• Average: 114 wRC+
• Range, 1 SD: 101 – 126 wRC+
• Range, 2 SD: 88 – 139 wRC+

Baez

• Average: 100 wRC+
• Range, 1 SD: 84 – 116 wRC+
• Range, 2 SD: 68 – 132 wRC+

For Betts, you’re looking at ranges of 25 and 50 points. For Baez, you’re looking at ranges of 32 and 64 points. One standard deviation for Baez is 27% larger than one standard deviation for Betts. Does that mean the audience perceives Baez to be something like 27% more difficult to project? I don’t know, and that seems like too simple an interpretation of the math, but forget about the words and just think about the numbers. Baez is projected here to be more volatile than Betts. The apparent difference? 27%. Perhaps that doesn’t seem like much to you. Or perhaps that seems like a lot. On one hand, they’re very different prospects; on the other hand, they’re still both unproven prospects.

Really, in order to do more, we’d need a bigger sample, with more players, some of them prospects and some of them veterans, some of them pitchers and some of them hitters. How do these measures of audience uncertainty compare to the uncertainty around, say, Mike Trout or Andrew McCutchen? Prince Fielder or Brandon Morrow? How do these measures compare to percentile likelihoods generated by projection systems like PECOTA and ZiPS? Are the projection systems more accurate? Are the fans more accurate? How closely might fan perceptions match actual reality, as opposed to what we think reality is going to look like?

The tricky thing is, we’ll never know how much uncertainty is the right amount of uncertainty around a Mookie Betts or a Javier Baez. A correct answer does exist — it’s a number, and numbers exist, and for example, 3 is a number — but it isn’t knowable, and it’ll probably never really be knowable given that every player is different and faces different circumstances. The absolute best player comp is still a comparison between one baseball player and a very different baseball player and person, so, how much can that tell you, really?

We don’t know how uncertain a player’s future is. The best we can do is guess. Often, these guesses are made based on baseball’s statistical history. In these posts, we had an attempt to poll a smart audience. If nothing else, this information reflects the feelings at a given moment in time. Maybe the feelings will end up looking silly in hindsight, but by then we’ll have different young players we’re all trying to project. Some of them are going to seem relatively safe, and some of them are going to seem risky.