Can Scouts and Statcast Coexist?
For some time, it seemed like the battle between analytics and scouts had died out.
The divide first surfaced in the public consciousness following the publication of Moneyball 14 years ago. Michael Lewis recounts in his book how some in the A’s front office contemplated a future in which scouts were redundant and no longer necessary — at least not in such numbers. It was an extreme view.
In the meantime, however, a sort of peace appeared to have been brokered. It was generally accepted that the best clubs, the model organizations — like the St. Louis Cardinals for much of the 2000s — successfully integrated both camps.
And then in 2015 something happened: Statcast was installed in every major-league stadium.
I remember, in the summer of 2015, meeting with several scouts for drinks after the conclusion of a game at PNC Park in Pittsburgh. One scout in the group expressed his concern that Statcast was soon to make them redundant, as it promised to record data like exit velocity, pop times, lead distance, route efficiency, and first-step quickness, etc. — that is, much of the same data for which scouts were responsible.
The promises came true: now, a couple years after that conversation in Pittsburgh, Statcast can do all that. And whatever its flaws at the moment, its reliability is only likely to improve in the future.
It’s with that as background that I took interest in some news broken by Peter Gammons on Friday.
As a reward for their part in their successful season, the Houston Astros today fired eight scouts.
— Peter Gammons (@pgammo) August 18, 2017
Astros GM Jeff Luhnow immediately told the Houston Chronicle that the move didn’t represent a reduction in the club’s scouting staff.
Luhnow insisted the Astros’ scouting department will remain roughly the same size in the future. The Astros are in the process of realigning their scouting departments, the GM said.
“This is not a cutback in scouting,” Luhnow said. “We are reconfiguring within and across the three scouting departments – international, domestic and pro.”
Luhnow declined to disclose details of the planned reconfiguration. Their realignment will consist of new hires, the GM said.
It will be interesting to see how, and with whom, the Astros restructure.
There are some in the scouting community threatened by Statcast, which can automate tasks their profession has performed with eyes, stopwatches, and radar guns for so long. Is technology threatening to disrupt this labor market like it has done with so many others this century? Is there a chance the size of the scouting workforce is reduced? FanGraphs’ request to speak with the Astros over the weekend was not immediately granted. But I had asked Texas Rangers GM Jon Daniels recently about whether scouts are endangered by technology.
“I look at it the other way: it frees the scouts up,” Daniels told FanGraphs. “I do think scouts’ role has changed a little bit. We don’t need them to focus as much on the minutiae of the play by play. They can actually kind of sit back and watch the game a little bit more away from the ball. They can focus on some of the information gathering, the pregame stuff, the work that is being put in. The role has a chance to change here with the information we have.”
And what is it that player- and ball-tracking frees scouts up to do? It’s the human element, Daniels believes.
“I think context on a personal level,” Daniels said. “I think in general, when we talk about evaluating players, that is the piece [missing]… You’re not there everyday. And that includes myself… That’s becoming more and more the skill of the scouts; their top responsibility is to identify that context. Identify that personal piece… that is an element you cannot access unless you are on the ground. The best scouts are able to fill in that blank.”
Is there a good example of that amongst the current Rangers?
“Carlos Gomez,” Daniels said. “Whether it was traditional performance or a deeper dive, an analytical look, there weren’t a ton of exciting things there.”
Gomez posted a .594 OPS in Houston last season. He was released by the Astros on Aug. 18. 2016. Two days later, the Rangers signed him and he posted a .905 OPS the rest of the season. He’s posted a .795 OPS this season.
What did the Rangers’ scouts see?
“[A scout] knew he might respond to our environment and leadership both in regard to staff and teammates,” Daniels said. “Sometimes, for whatever reason, things don’t fit in certain environments. It wasn’t working for Carlos in Houston, and Sam Dyson with us. It wasn’t working here… Sometimes we lump it in with change of scenery, but there is a subtext to that where there are other elements going on. Our guys [evaluating] Carlos had done homework in the past, following him since Milwaukee, knowing the player, knowing the person. We felt he could thrive in our environment and it played out that way.
“The term makeup. Is it values? Boy scouts? What are the things you are looking for? You can’t send a scout in on a special assignment to uncover makeup. There is history, there are years, there are relationships. You wouldn’t hire a person for a position based upon one interview. You would want to know past performance, you want to check references, you want to know this person on a deeper level, and that is where having the history and scouting standpoint is enormous. I don’t how that would be replaced.”
A Cleveland native, FanGraphs writer Travis Sawchik is the author of the New York Times bestselling book, Big Data Baseball. He also contributes to The Athletic Cleveland, and has written for the Pittsburgh Tribune-Review, among other outlets. Follow him on Twitter @Travis_Sawchik.
This came up in an earlier article on Bradley Zimmer. He’s the third fastest guy in the league this year, according to Statcast. How did he wind up with a 60 grade on his speed from every major outlet?
My theory was that scouts were biased by his size. jdbolick strongly disagreed and said it was likely due to acceleration, which may not be captured by sprint score. It stands to reason that even if sprint score doesn’t capture this, some other aspect of Statcast can.
I am pretty confident that individual scouts will be rendered less valuable, but as the data becomes more ubiquitous there are going to be things we haven’t yet quantified that will provide an even greater edge. Performance psychology, physical development and decline, and instruction still have lots of qualitative elements. I think it may be better to think of future scouts as “qualitative researchers” on a mixed-methods research project.
I like the way you put that. Anyone who isn’t approaching this as a “mixed-methods research project” (never heard that term before, but it sounds right on) isn’t doing this right, and there is definitely an important place for qualitative research.
Though the comp is inferior on my end, it’s where I put my focus any time I do fantasy sports. At this point, the raw amount of processed quantitative information means there is little advantage to be had in that realm, so the biggest room for an advantage when it comes to player analysis is in the qualitative arena. For actual MLB analysis in a billions revenue industry, then, I’d say it becomes that much more of a necessary piece.
Mixed methods research designs are becoming much more popular in public health.
There are many different mixed-methods designs. Some rely on qualitative data to theorize about what the quantitative data means, others try to use multiple types of data to get convergent validity, some use qualitative data to generate testable hypotheses. Qualitative data can also supplement quantitative data by answering complementary (but different) research questions. My personal favorite is an iterative method of moving back and forth between qualitative and quantitative data.
The key is that the qualitative data needs to be analyzed in some systematic way too, so it’s not just anecdotes. There’s a long tradition of analyzing field notes in ethnographic research, so I doubt this is a problem (and indeed, may be how front offices synthesize different scouting reports anyway).
Correct me if I am wrong, but top speed has been around for a while – it was measured in mph, now it is fps and it is new? In any case, not really my point.
I think you are greatly overestimating the role of technology in evaluation. I don’t think scouts will be any less important. They may not pack around radar guns and stop watches, but that will only mean that they have to spend more time looking at the game as opposed to an lcd display. You seem to think that Statcast is going to get even more powerful and accurate – I wouldn’t hold my breath on that. There isn’t that much left to measure… Statcast just takes rates for the most part, it doesn’t tell you anything about baseball – route efficiency is one exception I can think of – which also rewards players for being out of position and penalizes them for being in the right place.
As for Zimmer’s sprint speed, acceleration is more important than top speed, unless you are having a top-speed contest, which nobody has ever had. I think your assumption – not just yours – that Statcast can capture anything is false. Well, it may be able to capture some acceleration score, but it is not going to tell the rest of the story. It will just be another relatively worthless indicator of what Zimmer is. Those who place their faith in technology to this degree will always be looking for the “missing link” and it will always appear to be on the horizon.
So Statcast says Zimmer is the 3rd fastest player? Who cares if he is? Is he the third best base stealer? 3rd best center fielder? What does that even tell you? Perhaps scouts were right in that his speed would play out as a 60? We are talking about baseball here, and I think a lot of people don’t get that. Speed grade is not sprint speed, it is how well it will play on a baseball field. I would call a scout who simply used a stopwatch and a 60 yard dash score worthless just as I do sprint speed. Well, not worthless, but not far from it. Watch a few games for something much more meaningful. I am not saying scouts were right or wrong, just saying that I don’t get where Statcast even enters this conversation. I actually think most scouts are pretty worthless – not because of Statcast, but because most of them fall back on the easy measurables and they generally just follow consensus.
I think you are off in your view of the future scout. It is not the immeasurable that are the problem – it is the baseball part. It is not like we are going to get ten more accurate rates and then scouts are rendered obsolete. More data is going to require people to sift through the trash. The context is more important than the data and humans are best-suited to that task.
Its funny to me that many readers and authors think we are on the brink of some breakthrough. I think we are a few years away from some concrete examples of how all this excitement was misplaced. Once we can see that this data doesn’t tell much of the story, then we call call it what it is, which is “pretty neat”, but not the answer that people are looking for.
I think the real question is whether these new scouts will be anything like the scouts we see today. I’m thinking they probably won’t, since this will require a different skillset. To some extent, it sounds like you agree. I had assumed that the ranks of “data analysts” would grow and that the role of “scouts” would continue to capture the things we have trouble quantifying, but if you want to call the people sifting through the data “scouts” I think you’re right on this too.
I agree with you. I think the term “scout” is problematic as I don’t really know what it refers to either. I have coincidentally spent a lot of time around scouts (as opposed to the types that brag about some conversation with a scout), some are big-shot opinion makers (evaluators) and most are guys with a radar gun and stopwatch (literally there just to report what they came to see). I have sat next to them and overheard the lack of value that the low-level guys are reporting. I imagine that there are guys who are actually evaluating players as opposed to reporting details and I don’t think their job will change much. I think the real baseball experts will be the same guys with the same job. I don’t think they need the advanced data to do their job, which is use their eyes. Think about it – that is what the machines lack. I think there will be numbers guys and baseball guys – I envision separate divisions that each try to uncover players.
He has the third fastest top speed, which is different from being the “third fastest” player. If you follow the NFL combine you’ll see that there is often a difference between a player’s ranking in maximum speed and their ranking in shortest time covering a set distance. As I explained to you in that discussion, scouts record home to first times over and over again, so they have quantitative data to back up their speed scores. They also prefer bigger players over smaller ones, so there was never any justification whatsoever for your repeated insistence that scouts are biased against taller body types. Basically, you’re being unfair to scouts by accusing them of having a prejudice that is the complete opposite of what they actually prefer.
I did not intend to re-litigate this one, but here we are. I thought I captured your position adequately but if you disagree, I am glad you clarified.
I thought about bringing up the combine as well. They take a very numbers-centric approach and it produces laughable results pretty frequently. How often does some guy put up an exceptional 40 yard dash, just to later affirm that he is a poor football player? If we were drafting a team of sprint-speed guys, then that list would be awesome! Scouts don’t evaluate sprint speed – seems pretty straightforward to me.
Part of it is an odd race bias. Sort of like “looks the part is scouting lingo for a Latino middle infielder