An Improved KATOH Top-100 List

Back in January, I made some tweaks to my KATOH projection system, and have been using that updated model for the past several months. That model was unquestionably better than the previous versions, but it left me unsatisfied. While it addressed many of the flaws from previous iterations, there was still a lot of information it wasn’t taking into account.

I’ve been plugging away behind the scenes, and finally have a new version KATOH to share with the world. In what follows, you’ll find some detail on the new model, including its notable updates. I’ll be using this model in all of my prospect analysis from this point forward. Below, you’ll find a quick run-through of the notable tweaks, followed by an updated top-100 list.

*****

Added Features

Choosing projection window based on level, rather than age

In my previous model, I projected out based on a player’s age. If a player were 22, I projected him through age 28; If he were 24, I projected through age 30. This resulted in KATOH undervaluing players who were old for their level. The goal of KATOH is to predict the value a player will generate during his six-plus years of team control. By projecting a 22-year-old through age 28, KATOH failed to capture some of that value in cases where the 22-year-old was still in A-ball.

This time around, I chose my windows based on level, rather than age. I projected the next six seasons for players in Triple-A. I did the next seven for players in Double-A, eight for A-ballers, and nine for Rookie ballers.

Accounting for defensive performance, not just position

KATOH got significantly better when I added defensive position to the mix last winter, but there was still a lot for which I wasn’t accounting on the defensive side of things. Most notably, I was ignoring how good or bad a player was at the position he was playing. As we learned last decade when defensive metrics became a thing, the gap between a good defender and a bad defender at the same position can be huge. I’ve addressed this issue by incorporating minor-league defensive data compiled and published by Clay Davenport.

Somewhat surprisingly, the defensive metrics don’t add a ton of predictive value to what was already picked up by a players’ defensive position. A strong minor-league defensive performance will help a player’s projection, but won’t make a huge difference in most cases. It’s my guess that this has something to do with the fact that minor leaguers are often learning new positions, where they might be prone to making a lot of errors.

Incorporating ground-ball rates for pitchers

I also incorporated ground-ball rates for pitchers that I pulled from Clay Davenport’s site. With all else being equal, ground-ball pitchers have rosier outlooks than fly-ball pitchers, though not by a wide margin. In other words, ground-ball rates don’t add a ton of predictive value that isn’t already picked up by more conventional metrics.

Incorporating Baseball America top-100 lists

I’ve been conflicted about whether to include prospect rankings in my projections. There will always be important information the stats just don’t pick up that actual humans can. As a result, adding a variable for “BA rank” improves the models.

So why not include something that improves the projections? Well, for one, there’s potentially an issue of data consistency. The methodology behind 2016’s Baseball America rankings is unquestionably different than the methodology behind their 1991 rankings. They were done by different people who were using different information, so they aren’t necessarily consistent across years.

Secondly, there’s the philosophical question regarding what we want from KATOH. Incorporating scouting data makes KATOH a more accurate projection system, and makes it more useful as a stand-alone. But KATOH isn’t meant to act as a stand-alone. Rather, it’s best used as a tool for identifying potentially under- or overrated prospects. Including a variable for prospect ranking causes KATOH to shade closer to the industry consensus. This makes it more difficult to identify the guys KATOH likes and dislikes relative to the establishment. Perhaps I’m overthinking things, but I don’t think “more accurate” necessarily equates to “more useful” in this case.

Rather than picking a lane, I decided to create two parallel KATOHs: one that incorporates top-100 ranking and one that doesn’t. This is similar to what FiveThirtyEight does with their election forecasts, where they have a “polls only” model and one that also incorporates an index for economic performance. I will call the scouting-infused version “KATOH+” and will leave the original moniker to the one that excludes the BA rankings.

*****

Top-100 Lists

The models and lists I’m unveiling today aren’t perfect, but if held off on publishing projections until I had a system with which I was completely happy, I’d never write another article for FanGraphs. All in all, I’m content with what I have (for now, at least). I still have some ideas to improve things, but I’m saving them for another day.

This considers all players with at least 200 plate appearances or batters faced (plus Jose De Leon, since I know someone would ask). The figures in the far-right column refer to each player’s projected WAR over his first six seasons in the major leagues. As a reminder, these forecasts are not gospel. Take my math as seriously as you wish.

First, the version that considers stats only, and not prospect rankings.

*****

This next list incorporates stats, as well as rankings from Baseball America’s Midseason Top 100.

Note: The KATOH+ top 100 list I originally published was slightly inaccurate. My code was not correctly pulling BA rankings for hitters at the Low-A level, which caused the system to ignore the fact that Victor Robles, Kyle Tucker, Eloy Jimenez, Brendan Rodgers and Josh Naylor were top 100 prospects. As a result, these prospects were underrated in the second list. The issue has been resolved.