A Conversation With Red Sox Amateur Scouting Director Paul Toboni

The Red Sox have upped the quality of their farm system in recent years, and Paul Toboni has played a key role in that ascent. Boston’s director of amateur scouting since September 2019 — he was assistant director for three years prior — the 32-year-old University of Notre Dame MBA has helped facilitate drafting players such as Marcelo Mayer and Nick Yorke. Originally hired by the Red Sox as an intern, Toboni became an area scout in 2015 before climbing up the ranks to his current position.

On the road scouting when this conversation took place, Toboni talked about the process itself, the philosophies that shape an organization’s decision-making, and some of the notable draftees the Red Sox have brought on board.

———

David Laurila: Let’s start with the basics. What exactly does an amateur scout do?

Paul Toboni: “It depends on your responsibility. We have a number of area scouts across the country who are responsible for areas ranging in geographical size, and also in player density. There are a lot of players coming out of Southern California, so maybe we have two scouts there. We also have a scout covering Arkansas, Oklahoma, and all the way through North Dakota, South Dakota, Nebraska, and Kansas. An area scout’s job, really, is to be at the grassroots level and get to know these players really well. They talk to coaches, turn in reports, and come up with their lists.

“It flows upward from there, to our regional cross checkers. I think teams around the league are organized pretty similarly in that [the cross checkers] cover four or five area scout’s areas. From there it goes to national cross checkers, who are responsible for covering the whole country. Then it’s the front office.”

Laurila: What goes into the reports that scouts turn in?

Toboni: “We grade out the tools. For position players, it’s the ability to hit, their plate discipline, what we call their raw power — basically how far they can hit the ball — their power frequency, their defense, their arm, their overall athleticism. Then there’s where you value them in terms of their place in the draft. At the end of the day, it’s 1) How much do you like this player? 2) How do his tools grade out? 3) What level of risk should we be ascribing to this player?”

Laurila: I assume there are projected grades, and not just current grades?

Taboni: “Absolutely. That provides context. You might start with a 40, but at some point down the line you think the player is going to be a plus hitter at the major league level, which in that case, we’d put a 60 on him. We work off the traditional 20-80 scouting scale.”

Laurila: You mentioned draft placements. Do those tend to be fairly accurate?

Taboni: “I would say they are for the most part. That said, it’s easier to be more accurate the higher the player goes in the draft. Typically, we’re right on the money with the players that are going to go in the first round, or even the first 50 or 60 picks. As we get past the fourth or fifth round, we might have a 15th-round report on a player that goes in the sixth round. It’s tougher to make precise predictions as you get farther along.”

Laurila: Publications often include a player’s signing scout. What exactly does that mean?

Taboni: “What it means is that you were the area scout in charge of the area that the player was signed out of. For instance, Josh Labandeira covers Northern California, and he had [2020 first-round pick] Nick Yorke from the time he was a sophomore in high school. Labby had seen him a chunk of times, right in his backyard. All the way leading up to the draft, he was in communication with Nick and his family. Basically, you can think of Labby as having spearheaded the process.”

Laurila: How are scouts evaluated? For instance, a scout can recommend a player highly — Yorke would be a good example — only to have the team go in another direction for whatever reason.

Toboni: “You’re hitting on a really important point. It’s really tough to judge a scout based on the players they’ve signed. An area scout can have a player tagged perfectly — his report explains exactly what the player is going to be five years down the road — but the powers-that-be may not consider him [in the round] where [the scout] had him, because there are so many other players across the country.

“Evaluating scouts is a really challenging proposition. In my mind, a lot of it relates back to the quality of the person, the quality of the relationships they make, and the type of worker they are. Obviously, a feel to evaluate is really important, and I think you come to understand that over time.”

Laurila: To what extent do organizational scouting philosophies exist around the game?

Taboni: “That’s a tough one for me to answer, because outside of interning for the Oakland A’s for three months, all I really know is the Red Sox. So I can’t speak for other organizations, but as an outsider looking at how teams draft — the [consistency of the profiles they’re drafting] — I think they very much do exist. What I find fascinating, whether I’m right or not, is that there’s real variance in what teams value.”

Laurila: “Best player available” is a mantra in every draft, but it’s not nearly that simple…

Toboni: “That’s exactly right. And [Boston’s] best player available likely isn’t the New York Yankees’ best player available, and vice versa. Once again, I think it all derives from what you value organizationally. You’re putting a premium on those qualities, and hopefully selecting players that carry those qualities.”

Laurila: What changed when Chaim Bloom replaced Dave Dombrowski?

Toboni: “That’s a good question. I think they’re both great leaders. I think they have different leadership styles. My experience with Chaim has been a little bit different, because I was elevated to my current position right before he came over. I’ve had a lot more interaction with Chaim because of my job responsibilities, so honestly, I don’t know if I’m the right person to ask.”

Laurila: How much has data impacted the job since you came to the Red Sox?

Toboni: “Significantly. That’s one of the things that has changed in scouting. Within an amateur scouting department, there are two variables. There is a scouting process, and there is a valuation process. The valuation process has started to change a little bit throughout the game — basically, how we combine all the scouting reports and stats, and arrive at a ranking for a player, and eventually a full board. We do things differently now than we did when I first came to the Red Sox.

“An influx of new data has opened up this whole new world, and it’s not just all the newer tech, like batted-ball data. Some of it is as straightforward as the data that is housed in scouting reports. That is now structured and organized in a way that it wasn’t in years past. Taking it a step further, there’s been a greater emphasis on figuring out what to make of all the data, and how to account for it appropriately. For instance, by studying years of how we’ve evaluated prospects, what profiles — what general characteristics — have we systematically overrated as scouts? And which ones have we underrated? How do we responsibly account for that?”

Laurila: Can you give an example of something that has historically been underrated, or overrated, by scouts?

Toboni: “That’s an important question, but I’m not sure that I can give you a good answer. This kind of fits into, ‘What I should say, and what I shouldn’t say.’”

Laurila: Fair enough. Let’s circle back to Nick Yorke, who a lot of people viewed as an overdraft. To this point, he’s looked like anything but that.

Toboni: “That was a tricky year, right? We had less data on players in [the 2020] draft than we’ve had in any draft I’ve been a part of with the Red Sox. But we did have data that came through in scouting reports, as well as biographical information that was really positive. The quality of the hit tool grades that we were giving Nick, the power grades, the overall role grades and where we saw him fitting in the draft — where we thought the player should be selected — I think we appropriately considered all those different variables. Ultimately, they vaulted Nick to a really good spot on our board.

“The point you bring up — him being viewed as [an overdraft at 17th overall] — is something we had to take into account. We felt like we were some of the only ones seeing what we were seeing, and there is some risk with that. The wisdom of the masses. I think you can make a strong arguments that accounting for that is a real thing, and it probably shouldn’t happen, right? So it kind of works both ways. At the end of the day, what won out was the quality of the scouting reports, and layered on top of that was our comfort with how good of a person and competitor he is. That really only helped his case.”

Laurila: How similar were Yorke and [2020 third-rounder] Blaze Jordan at the time they were drafted?

Toboni: “Very different. They’re very talented in their own ways, but Nick was this really polished hitter; I didn’t worry too much about him going out there and hitting from the get-go. Now, when he hit .190 in his first 90 plate appearances in A-ball, I was avoiding Chaim in the Fenway offices. But no, seriously, Nick was someone who didn’t have ‘wow’ power, but we thought he would get there eventually. Blaze was kind of on the flip-side; he had this jaw-dropping power. And I think Blaze would say this himself: he didn’t have this polished hit tool. Remember, Blaze was a year younger — he reclassified — so he needed to have a lot more plate appearances under his belt to catch up to his peers.

“A trait that they share is, neurologically, how they process information out of the pitchers hand, where they deliver the barrel, the timing of it… that’s pretty tough to teach. If you can identify that at a younger age… I think that allows for a greater capacity to grow as a player. All that said, Nick was a little bit further ahead in terms of his level of polish. Blaze had this incredible raw toolset, but was probably a little bit behind Nick on the development curve.”

Laurila: How does a scout identify the neurological component you referred to?

Toboni: “This is part of the art of the game. It’s one of the cooler parts of scouting. When you see a player take a breaking ball… we talk about the comfort of their takes. I don’t know if you’ve seen the clip with [pitching prospect] Bryan Mata from right after Nick signed. He threw an 89 mph slider to Nick that he hit hard to right field. Nick had never seen a slider of that velocity, or even movement, but he was still able to deliver the barrel on time to the right place. That clues in a scout. To do that, you have to neurologically be in a different spot.”

Laurila: Marcelo Mayer wasn’t expected to be available when you drafted him fourth overall last year. Given the perceived likelihood that he’d go in the top three, if not first overall, did you maybe not scout him as heavily as you otherwise would have?

Toboni: “No, I don’t think so. The way we approach it is to have a list of players that we think have a real chance of going to us — let’s say it’s a list of 10 players — and get to know them better than any other team in the industry. Marcelo fell in that bucket. One mistake we don’t want to make is have a player get to us, and we didn’t have a good process on him. So we saw Marcello a ton. We saw him early, late, and everywhere in between.”

Laurila: I’ve often asked hitters and pitchers if they see their craft as more of an art, or as more of a science. What about scouting? More art, or more science?

Toboni: “That’s definitely a question I wasn’t expecting. It’s some combination of the two, but I’m not sure if I can give you a better answer than that.”

Laurila: What about this: Does it become more one than the other as you get deeper into the draft?

Toboni: “There, I’d say it’s more of an art. The players are grouped so tightly together. Subjectively, it’s really tough to separate players in the 15th round, whereas when you’re looking at the first 10 players in the draft, there are some real differences in absolute value. As it gets later, it becomes of less of, ‘Hey, let’s take the player that has the highest numerical term’ — whatever we’re optimizing for — and more, ‘Hey, let’s take a calculated bet and take this player over that player.” We’re folding in some art that way.”

Laurila: Any final thoughts?

Toboni: “How do you build out a board? I’ve kind of already touched on that, but it’s something that fascinates me. There are different ways you can slice it, and again, a lot of it depends on where you are on the board. Up top, there is a greater chance that there is a significant difference, and the deeper you get, the curve flattens out a little bit in terms of how we have players valued.

“It all starts with optimizing, with figuring out what you’re optimizing for and then having it be represented in numerical terms. Spencer Torkelson is this value. The player above or below Torkelson is this value. Adley Rutschman is this. Or whoever. In that respect, it becomes pretty straightforward how you rank out a board. And, relative to each other, you need to understand the differences in valuations from player two to player three, and player three to player four. Stylistically, while I might be biased here, I think that’s a good place to start. Really, it’s no different from what you’re doing at FanGraphs. An article is written on how you’re valuing a player, and how that valuation is arrived at. Scouting departments are faced with the same issues.”