Mike Matheny, Robin Ventura, and Managing Statistics
Mayhaps this is the season of hiring the un-experienced?
Last October, the White Sox announced a surprise decision to hire Robin Ventura — a once-great third baseman who had been hired four months prior as a special assistant. Ventura, who played some high quality baseball in some three decades, has managed nary a single MLB or MiLB game in his life.
Then, weeks later and a few dozen hours ago, the St. Louis Cardinals hired Mike Matheny — a once-illustrious catcher and he of the ne’er .300 wOBA — to replace likely first ballot Hall o’ Famer Tony La Russa. Like Ventura, Matheny has not yet scribed a professional-league lineup card or even tossed a base in anger. He enters the 2012 MLB season a complete unknown to fans and writers and presumably of good number of his own players.
But then again, what do fans and writers really know about even Tony La Russa? He made some strange decisions, he made some smart decisions, and he made some bad decisions. He gave a good interview, and he presided over a few championship teams.
But what does that really tell us? Where does that leave the ledger? For years we have debated the veracity of certain managers, but what measures and records can we go to?
One thing is for sure: The world of managing needs more statistics.
A quick grammar note to get the nerdy reader feeling frisky: The title of this piece — “Mike Matheny, Robin Ventura, and Managing Statistics” — is not using the word managing as a verb, but rather a gerund, which is an -ing verb that functions as a noun (then I’m using that noun in an adjectival function on the word statistics — GRAMMAR!).
In other, far less geeky terms, I want to investigate the statistics of managers, not the statistics that managers use. We have much of the latter — Tango’s The Book is essentially dedicated to that purpose — but little of the former to my knowledge. (Or at least it’s too hard to find.)
Well, to say we have no MLB manager statistics is incorrect. We do have one: Wins.
So we have practically none. Because, frankly, wins tell us nothing. Yet tragically, wins are the go-to statistic for fans and the media. How many games has he won? How many pennants? How many championships?
Yes, managers do help teams win, but probably less so than general managers and certainly the players. If Tony La Russa gets credit for his world championship teams, does he also get debit for the 73-89 Cardinals season in 1997? Does his 68-94 Oakland A’s team in 1993 hurt his reputation, or do we wash it away in the majesty of the sum of his career?
I don’t know. How do we know? Wins capture information from so many sources, that we cannot begin to parse the guilty characters. I’ve heard some give La Russa credit for out-managing Ron Washington in the 2011 World Series, but without David Freese hulking out, La Russa probably does not get that same credit.
Essentially, it comes down to this: We need statistics for managers. Some sort of record that will help us better evaluate managers and managerial decisions. The moment we do, we start on a path towards better analyzing individual managers and then better understanding the impact managers have on a game and a season. WAR for managers could some day be real, but first we need our OBP, our most basic tool — something to get us started.
To that end, I want your participation! Yes, I just pointed my finger through your computer monitor / mobile device. Consider this…
An Open Call for Statistical Ideas!
You dear reader, thinker, and stats-lover: Tell me, how would you build a record of a manager’s performance? What events would you record? Is there some long-forgotten research on the matter?
Obviously, some things will be harder to observe than others, such as ego-management and conflict negotiation, but with what we can, is there some way can record and catalog it? If we were the GMs of the Chicago White Sox and the St. Louis Cardinals — and we wanted some statistical aid in evaluating Matheny and Ventura — how would we do it? Where would we start?
Put your best thinking stockings on and let us know in the comments. I have a few ideas of my own, but I would prefer to refine them with your suggestions first. Let’s talk, and then take it to the next level. Of awesome.
Totally spitballing while in class right now (stats class actually), one factor might be reliever WPA? Bullpen usage is a substantive part of managing baseball, and some kind of reliever WPA/162 or something like that could speak to how effective a manager is at using his relief pitchers. It also obviously speaks to how good his relief pitchers are, but I don’t know hw you can get around that.
This exactly the path I started wandering down. Some sort of WPA added per bullpen appearance or something.
But, like you noted, that’s going to be heavily influence — so I suspect, at least — by the quality of the bullpen.
Continuing down this path a bit, maybe one could figure if there’s any trend in Leverage Index when he chooses to make a bullpen move/pinch hits/etc. Try to find out if making a move is more likely to be a success or failure given a certain amount of leverage.
Another wrinkle in this is leaving starters in too long. Joe Maddon — whom I anecdotally and personally believe is the best active manager — has a tendency to let some pitchers go maybe a batter too long.
Is this something that just seems like a trait of his because his pitching staff is exceptional and they’re pitching deep into many games, or is it an actual flaw? Would other managers make the same mistake with a similar rotation? I don’t know.
What does a manager generally have control over?
Lineup composition, SB attempts, bunts, hit and runs… Defensive alignments/shifts, pitching changes..
WPA can be attributed to some of these moves, but generally I think there will need to be qualitative analysis done in conjunction with the numbers as all managers will have different cards (players) to play.
Yes, definitely. There is no way a manager’s full value will be statistically observable. In fact, we can say the same thing — to a lesser extent — about players.
Players are paid to produce, though, so we have a better idea whether they are producing, but managers deal with more ethereal and — like you said — qualitative matters. Still, let’s try to observe what we can and see what we find.
First thing that comes to mind is some sort of “synergy” metric. Does a manager exhibit an ability to produce more wins for a team than the sum of its parts would indicate it could? A lot more thought is required, but to first order you can compare combined WAR or RC vs. actual wins. This would be similar to the idea of clutch for a hitter and an analysis of consistency from year to year would need to be done to demonstrate a real-world skill.
Ooh, that’s a great thought.
(WAR win-total) — (actual win total) = (managerial residuals)
There is a danger, I imagine, in attributing all the residuals to just the manager. For instance, the third base coach can cost or add a significant number of runs based on whether they consistently send runners in good or bad situations.
Still, I think this is also the right track.
Oh yeah, there’s definitely going to be a lot of noise involved at first. Obviously the more complex it gets, the better it’ll be at filtering out that noise.
Well, generally the manager will have a say in who that third base coach will be, so some of that responsibility is still the manager’s.
That is, the manager will choose who he wants as his coaches and what their role will be on the team.
Similar to this, ESPN recently looked at managers who consistently beat their pythagorean W/L and Scoscia was the clear winner there, Gardenhire pretty good too.
I agree with this. We would need to control for team skill and luck (BABIP). It would be nice if we could isolate the effects of a manager, but we would likely be measuring an entire coaching staff. Coach of the year awards are usually given to the most overachieving team, so it would make sense to measure how much more than expected they can get out of their team.
Beyond this, we need to define what we are trying to measure. Are we measuring their application of game theory, like good or bad decisions? Are decisions judged as the perspective when they are made or only retrospectively using results? Getting the most out of their players? Turning careers around? Influence in recruiting players? I think the first thing to define here is what events we are looking to measure. Once we have that, we can make some hypotheses and begin testing to see what matters.
How about (as a crude first approximation) team WPA from the 7th innings onwards, adjusted by overall team WPA (i.e. their W-L record)?
Then you could try to overlay some sort of adjustment for bullpen & bench quality, though perhaps the reason why the bench is quality is because the manager’s starting the wrong players. It’s all very knotty.
That’s an interesting thought. Of course, the problem does persist — as I noted in a comment above — that the manager still decides when to pull their starter. If the manager consistently makes his starters slough through 7 innings, regardless of how many runs they allow, then their LI in the 7th through 9th would be ultimately lower.
But that would be factored in (if the manager was leaving the starter in for the 7th when they shouldn’t be, then their WPA for innings 7-end would be worse). It doesn’t take into account starters continuing into the 6th when they shouldn’t (you could draw the line at the 6th but you’ll include a lot more ‘normal’ innings without substitutions if you do).
I have felt for a while that an individual manager can’t do much to actually make his team perform better than expected but he can certainly do things to make it perform worse. The manager should be on the same page as management and implement management’s vision. If there is any truth to the story of Art Howe flatly refusing to play Billy Beane’s Moneyball players in the first portion of the season (as represented in the recent film) then here is a perfect example of how a manager can make things much worse.
Joe Maddon will likely be presented as a manager who makes his teams better. I’d suggest that Maddon is simply the best current example of a manager who is totally buying in to what his GM is attempting to accomplish and is implementing this strategy completely. The vast majority of the credit in Tampa Bay should go to management for evaluating talent extremely well and by simply behaving completely rationally by exploiting inefficiencies and refusing to overpay for talent (like the very large majority of its irrational peers). I suppose the manager of the team has to have some credibility among the players, that the manager has to be viewed to a certain extent as one of them as opposed to some guy who is a tool of ownership/management. But this is not a baseball skill, this is a management skill, a people skill.
I personally don’t think it’d be useful, but you could compare team fWAR with actual win-loss record.
Could you look at teams that consistently outperform their WAR? Managers don’t seem to outperform their Pythag W-L, but maybe the manager plays a part in getting to those runs scored-runs allowed measures in the first place.
Second, if we’re talking about the “clubhouse chemistry” aspect of managing, we also have to talk about it for the players, which makes it even more difficult to capture.
Third, I bet that however you measure it, you find that the variability in manager performance has decreased over time. There are too many practices set in stone (closer in the 9th) for managers to deviate much from the script. But maybe early in baseball, someone like Joe McCarthy was innovating in ways that are measurable.
Along the same line as ICEYhawySTUNNAZ, how about measuring all the stats of pinch hitters against what the expected stats of people they replace? Also, you could compare situational stats against league averages to look for areas where the team consistently outperforms the league. Obviously, this is going to be biased for a good team when compared to a bad team, but it’s a start. I realize this is vague, so an example: if team A hits .200 in full counts with RISP and the league average is .225, then it counts somewhat against the coaching staff. Obivously, you’d want to drill down to as specific a situation as possible while still maintaining a decent sample size for comparison (so taking this down to batter/pitcher matchups, in the third inning of day games after night games, and full count with RISP is probably not good for a comparison). Clearly, the formula to calculate this could get to be massive and difficult to calculate quickly. Also, it would need to be normalized to a finite number of outs, as a team that plays above or below the averages would skew that direction in a longer game.
How about some measure of Actual Wins per WAR? A measure of how well the manager is able to turn the contributions of his players into wins. It would have to be adjusted for the number of wins of an all-replacement team.
To address the geeky grammar stuff: “Managing” is never a verb. It is either a participle (which can be compounded with a linking verb to create a finite verb) or a gerund. The former is a verbal adjective and the latter is a verbal noun. You are correct that you are using the gerund but you are not exactly correct that you are using the noun as an adjective. Instead you are creating a compound noun and omitting the needed hyphen, an aspect of grammar that is quickly disappearing. Your phrase “Managing Statistics” is in fact an abbreviated form of “the Statistics of Managing,” where the “of” indicates an aboutness-relationship. The prepositional phrase is an adjectival phrase, which is why “Managing” as you use it appears to be an adjective itself. However, if we understood “Managing” as itself an adjective, we would not be able to recover the proper meaning of your compound noun phrase because we would be taking your phrase to mean “Statistics that manage,” this being the meaning of a participial use.
An astute correction.
Actually, “managing” is a verb — it’s the present-tense form of the verb “to manage.” Were it not, it could be neither a participle nor a gerund.
Also, I disagree with your construction. “Managing” is here a participle, because it is indeed functioning adjectivally to modify the noun “statistics” — that’s why it takes the “of,” because it’s an adjective in the genitive case. The thing is, the genitive can be either objective or subjective; one could indeed mean “statistics that manage,” but the other means “statistics related to managing.” (Which is which? Alas, I can never keep the objective and subjective genitive straight as to which means which order — whether subject and object in that case refers to the noun or the adjective.)
As Josh Goldman suggests, you can put together a “synergy” estimate.
Wins – TeamWAR and Wins/TeamWAR are good start. I think comparing WPA and WAR might also have some value.
As the other Nathan says, we can look strategies. We can take a simulation based approach and compare a manager’s actually strategies to the best strategies we can find via simulation. Line up optimization being the easiest one of these. So, for instances, runs_actual/runs_optimal would be a good indicator. And net change in run expectancy due to hit and runs and steals and such would be useful as well. Defensive alignments are a bit trickier, but again we can take a simulation based approach.
A manager’s (and staff’s) influence on player development is a different matter, though. How much of Jose Bautista’s improvements are due to the Jays organization. If it’s even a quarter, then that’s already a couple of wins by itself. Has Mark McGwire taught the younger Cardinals to hit with more patience? How can we tell these things? I have no idea there, since we’d credit the players’ improvement to the players themselves.
A great point — and something I am addressing in my second piece.
Do managers miss enough games per season to do an analysis along the lines of “win % differential” for basketball players? For reference: http://skepticalsports.com/?p=1181
This might be a stretch, a lengthy calculation process, but could there be a “runs from optimal lineup” calculation? Compare the optimal lineup the manager could have used (based on maximizing runs) to the lineup they did use. No manager will be perfect, obviously, but the closer to zero, the more optimal the lineup construction.
I had thought of this myself, but this system could unfairly punish managers who set lineup according to correct expectations, but then nagging injuries or unusual under-performance makes them look bad.
One of the problems is that optimal lineups are just numbers. They ignore egos. It’s all fine to say a certain player should bat 2nd, but if he’s always seen himself as being “the man” and batting 4th, it might make sense for a manager to stick with batting that player 4th (provided he delivers good production) to avoid a confrontation. Many people talk about how conservative managers are in their lineups, but many players are as equally conservative.
This is a case of managing lineups vs managing personalities. Sometimes it wouldn’t be worth winning the lineup battle.
I don’t see this as being a problem. If a manager’s optimal runs/actual runs ratio is seen to be below a given threshold, then you could easily say that he has to slot batter x into the cleanup spot for the sake of managing egos. It would be much like the fact that batters with low wOBAs (relative to their career numbers) have a low BABIP – that is, it would be noted as an explanation for poor performance.
Yeah, but winning the lineup battle would be a manager skill just like wanting to make the better lineup would be in the first place. An inability to convinced Bobby McPasthisprime that he needs to slide out of the #4 slot represents a failing on the manager’s part, just like putting Bobby there in the first place out of ignorance would be.
What about measuring lineup optimization by figuring each position’s performance vs. league average. Figure, for example, the wOBA of the #2 hitter vs. the league average #2 hitter.
That’s likely a better measure of GM performance than managerial acumen. The Yankees will have better wOBA from their #2 hitter than the Orioles (for example) because they are able to get those players, not because Girardi is a better manager than Showalter.
Good point. Maybe then you take the context of the team into account. Say, using Runs Created you figure what percent of your team’s Runs Created come from the #2 spot, and compare that to league average? So if someone’s #7 spot is creating a greater percentage of his team’s runs than the rest of the league’s respective #7s, while the #2 spot is creating a lower percentage of team runs than the rest of the league’s #2s, then perhaps the manager in question loses points for not swapping the two.
Brad,
Why does the world of managing need more statistics?
Hi Edwin.
Well, philosophically, nothing needs more statistics. Baseball is a pass-time for most, an occupation for others, and it had hummed along just fine for 150 without managing statistics. Frankly, if we’re talking about need, there are some world economics issues that deserve the term (see: Somalia, Sudan, Indian slums, etc.), but baseball notsomuch.
Managers, though, do not have any analytical stats for evaluation. Why should we spend so much time and effort evaluating all but the manager? We can look at a GM’s trades and signings, a player’s WAR, and umpire’s strikezone — but only a manager’s wins? Surely there’s something else yet-uncovered.
Brad,
I think you took my question a little too far.
I’m not against statistics. I like using statistics to measure batter or pitcher performance, or even GM performance, because those things are easily measured, and the value from using those statistics can be significant. These types of statistics help a team better deploy their resources. That is why we should spend so much time and effort evaluating these things.
I don’t see value in coming up with a statistic to determine who the most valuable managers are, since managers have such a low impact on season success compared to player performance. I agree that that there is more to a manager than W’s and L’s, but even if you come up with a statistic to better measure manager performance, how does this help a team?
I did something like this in a college statistics project a few years back. I used only for the batting component, e.g. how the manager managed his position players (so that my project had a managable scope).
Methodology: I found a bunch (about 5) different projection systems for each hitter’s offensive and defensive performance, then predicted based on that what each team’s optimal starting lineup would be. I compared how similar the actual team’s lineup was to the optimally projected one in terms of scoring runs (using a lineup analysis tool like http://www.baseballmusings.com/cgi-bin/LineupAnalysis.py), and ranked managers within different projection systems.
Every month, I revisited my project and compared the actual team’s performance in terms of RS (and individual players’ performance), and basically gave managers positive points if their lineup did better than an optimal one would have, and took away points for worse performances. Then, I redid the optimal lineup using that first month’s performance by each player, as well as anecdotally changing some lineups due to some injuries, etc (that part was somewhat less than scientific).
I redid that process every month or so throughout the season, and tallied each manager’s month by month final score, to get a score for the year.
Obviously, within this methodology, there is a big “luck” component in the short term, because a manager could start the wrong guy, or have a bad batting order, etc. etc. and get “lucky” that the guy hits much better than projected (although it could also just be the case that he had a talent or “behind the scenes” knowledge that allowed him to project better than Marcel or whatever other system). However, in a large enough sample (5 years of month-by-month data maybe?), I think this would lead a pretty reasonable result a the luck evened out.
Maybe something sorta similar to this could be applied to pitchers, to create a TMS (Total Manager Score)? Thoughts?
used *data* in the first paragraph… sorry, long week.
Comparing WAR to team records doesn’t seem to me to have any value. WAR, as the Fangraphs staff constantly remind us, is not a measure of true talent. It is more a measure of actual production and amount of playing time. The latter, at least, is more in the manager’s control than team wins and losses. Also, it could be argued that the manager’s people skills or lack thereof have motivated the players to be more or less productive (WAR) than their true talent would predict.
WPA from the seventh inning on (adusted to the talent of the relievers or not) sounds so silly to me as not to even merit discussion.
This is an extremely difficult problem. Anybody who can come up with a method that will even partially measure the manager’s effect on the game will be a genius in my eyes.
M-war. Do it up fangraphs
I know one thing…… After all these statistics take off….. we will realize that there are about 28 league average managers…… Most of the overpriced managers will be let go, and then Joe Maddon will get a sweet raise…
Would have to say shut downs and melt downs should would be part of mwar. So many times you see a manager bring out someone no one in their right mind would use.
I don’t really see what else you could use…every thing else seems to be on the talent of the team not the manager.
What’s the old quote, “Everyone loses a quarter, everyone wins a quarter, I’m paid for what I do in the other half.” (Could be third third third, I forgot)
But, maybe use fWAR W-L (subtracting 40 from W and 40 from L) / Actual W-L (subtracting 40 from W and 40 from L.)?
I’d say a plus/minus system using WPA to determine if the move was the “correct” move. For instance, was the win probability added by a successful bunt worth the risk? If not, it’s a negative. Was the reliever matchup a favorable one based on splits? If not, it’s a negative. If so, a positive.
Would each event be worth 1 point, or the amount of lost/gained WPA?
I would say each event would be worth 1 point.
As for Edwin’s question, couldn’t you give a +1 for not doing something because that would be the right approach per WPA? Wouldn’t every manager be often put in a situation where a move could be made regardless of approach? For instance, a runner on 1st in the 3rd inning has nothing to do with manager style, it’s just a common situation. Then the question might be whether or not to bunt and that’s where the +/- system comes into play.
I haven’t really thought this out completely. I was just throwing it out there.
I think the Plus/Minus idea is the best approach, but how do you separate out opportunity from this? Won’t some managers have more chances to gain/lose points simply by the random number of decision situations they find themselves in, or the managing style they prefer?
If I were tackling this, I would first endeavor to create some sort of xWPA, creating a context neutral baseline so we could judge managers. I just thought of this, so suggestions are definitely needed. xWPA wouldn’t just be based on the base-out states, but also on the best guess true talent of the players involved, or expected to be involved assuming the manager makes no changes. xWPA would also have to include some sort of best guess for a true talent platoon split for each player since that’s what managers manage most. Anyways, for each play actual WPA – xWPA = manager WPA, which is in wins above replacement (where a replacement does nothing).
This idea gives managers credit for players outperforming expectations and for having the players most likely to succeed in the right situations. In the long run, a manager that motivates his players right gets rewarded with this stat, but more importantly, managers that make canny substitutions are rewarded. As an added bonus, the end result is already in WAR. I think this is a good place to start. Who’s up for inventing xWPA?
I’ve given a lot of thought to this prior to this discussion.
Here goes:
Any time a manager changes the flow of the game by bringing someone into the game to replace another player can be measured. Was it the best possible solution? And then the follow up. How did it work? This is the case for relievers, pinch hitters, and pinch runners. I guess you could include batting lineup as well. How well is his lineup ranked vs. how well did it perform.
Then there is the “strategy” side to things. The general play by play that a manager usually has control over. Intentional walks, sacrifice bunts, defensive positional alignment, pitch calling.
Then we have to consider the moves that a manager didn’t make. Should he have brought in a reliever? Should we have seen a pinch hitter there? Would a sacrifice had been a better option for the team? etc. OR, why did he bring in that reliever, obviously it was the wrong choice.
Perhaps on the reliever side of things, the available options could be ranked for the situation and the manager could get a rating based on who he brought in. Then the number could be altered based on the relievers performance.
Just some ideas.
Chris Jaffe wrote a book on evaluating managers that introduced some advanced metrics. Well worth reading.
This.
I mean… it’s a whole book. I don’t imagine it’s perfect, but it’s good. So there’s already more rigorous thinking than “practically none” out there, if you remember to look for it before claiming it doesn’t exist.
Dusto: I agree with your thoughts. Maybe WPA and the hypothetical xWPA would help quantify them?
Another complicating factor is the way managers sacrifice future performance for present success. Billy Martin comes to mind as a manager who ruined the careers of many young pitchers by overpitching them to achieve short-term success. Therefore, there should be some mechanism to reward the sustained success of TRL, Earl Weaver, etc.
I think there are many lines of research to be undertaken on this topic. The first thing that comes to my mind is player performance vs expectations. So say something along the lines of preseason zips WAR vs actual end of season WAR for players managed. Granted this will ignore fielding, but I think it is a useful measure. Obviously this won’t take things like players playing through injuries into account, but I’d argue that this hits on something that should be attributed to the manager anyway.
It seems to me that based on the above comments there are some very interesting ideas, but perhaps too many, leading to us trying to boil the ocean. I would suggest that, contrary to what some have suggested, an all-encompassing “mWAR” is too great a task to take on at the moment. Rather, let’s look at a few select
Rather, let’s look at a few select scenarios in which the manager plays a part, and worry about a WAR-like, universal statistic later. In no particular order, the things that a manager has control over, and how these things can be analyzed, isolating out the manager’s assets (aka players) to see which he is optimizing:
Batting order
Pitching Rotation
Bunts (ignoring the difference between sacrifices and bunting for hits)
Steals (I will not include hit-and-runs because it is not easy to tell when a hitter has chosen to swing at a pitch rather than being told to do so)
Pitchouts (although this is a debatable one to be included here)
Pinch Hitter usage
Relief pitcher usage
And in no particular order, the things which are in the Manager’s control (at least partly) which I would suggest are too difficult to quantify and measure, and where our efforts should not be focused:
Team chemistry/synergy
Managing egos
Positioning of players
Player growth/development (the Jose Bautista effect)
Player skill
I think you’re on the right track. Start simple, then expand.
As stated by ‘dusto’ above, for bunts, steals, pinch hitters and relief pitchers there should definitely be a way to take into account moves that weren’t made as well as the ones that were.
With steals and bunts there should be a handicap of some sort to account for the times a player executes them on his own.
Perhaps with the pinch hitter stat you could also look at not only whether the best bench player was used in a certain at-bat but also whether or not the guys on the bench are actually the optimal bench players at all. i.e. Are there guys rotting away in AAA who would be demonstrably (or at least hypothetically) better in that situation? (e.g. Mathis v. Conger)
I’d say one of the goals of a manager is to get his players to overachieve. Maybe we could see if the players on the team consistently outperform their projections (whether you use PECOTA, ZiPS, Marcel, or whatever).
I can tell you where not to look.
I spent a few nights going over Pythagorean W/L and Pythagenpat data to see if Mike Scioscia really had some sort of elusive skill to enable the Angels to outperform their Pythagorean regularly.
Going back to 2000 and comparing the +/- for every team up to the end of 2011, I found that the numbers themselves are random but collectively the distribution follows a pattern. It’s already known that the best and worst teams tend to outperform and underperform their Pythagorean, respectively, but for the teams in the middle it seems to be random flux.
As for Scioscia, he’s at +25 for outperforming Pythag W/L since taking over the Angels in 2000. It looks great but it doesn’t mean anything. Torre was +41 with the Yankees but ?5 in his time with the Dodgers.
I also thought that being with one team for a long period of time might be a factor as well so I looked at some managers who had at least 10 years with one club and how they fared Pythagorean-wise in their last 7 years with a team, giving them three years leeway. Once again, nothing. Bobby Cox was an average of ?3.42 his last 7 years with the Braves. There’s a chance he just got worse as a manager but I doubt it.
I’ll post my findings in more detail in the community blog if anyone’s interested.
The question marks should be minus signs… Don’t know what happened there.
Assuming that Scioscia is a bad manager in lieu of data is getting away from this whole objective knowledge thing. Just sayin.
I’m not assuming Scioscia is a bad manager. I’m just saying that outperforming the Pythagorean W/L is not an indicator that he’s a good manager.
Another stat may very well find that he is.
I see a lot of people focusing on the bullpen, but the thing we need to realize is that a lot of these teams are seeing the manager as someone whose most important job isn’t in-game managing, it’s handling personalities and motivation.
The Cardinals in particular made it clear they were looking for someone with “leadership”, not experience managing games or handling certain situations like maybe the Cubs seem to be with their rigorous interview process.
How can we quantify something like players just playing better?
As a Cardinals fan I always noticed that LaRussa always seemed to have a large number of high performing bench players, and it always seemed to me that one of his greatest attributes was getting the bench players contributing in positive ways throughout a season, moreso than other managers I got to watch regularly. I have no clue if there’s any statistical evidence to support such an assertion, but I think we really need to be looking at things like this instead of just in-game management.
There’s probably too much noise to pull much of anything on the manager’s influence out of all that, though, so I’m not sure where to start.
In terms of in-game management, couldn’t we just look at the optimal WPA for managerial moves like bunts and pitcher changes for every game state and then compare that to what the manager actually did? Kind of a batting average for managers, with 1.000 being all moves the best? I mean we have tons of articles about this stuff every postseason, so I’m not sure why it’s never been compiled into something like this. Maybe it’s just too much data to sort through, but it doesn’t seem like it’d be too hard to make a program to do it.
We’d then want another metric that measure how often the “sub-optimal” moves produce positive or optimal-or-better results, to see who really is “pushing all the right buttons.”
This is exciting. I feel like I’m watching the future of managerial grading begin right in front of my eyes
I tried expected WAR wins-wins, but it had a 0 correlation from year to year. So instead I decided to do something else. I did the linear weights for SB and caught stealing, divided it by team speed score, and multiplied it by league average speed score. I converted this to above average format. Then I found the wOBA of bunts and put it into wRAA format. I added them both together.
The leaders were
MIke Scioscia14.25610337
Joe GIrardi 10.16566916
Ron Gardenhire 10.11724406
Buck Showalter 9.066025265
John Farrell 8.63586869
It is too time consuming, but a good thing to do would be to use regresed wOBA based on PA’s and the same thing with UZR/150 and find out how many runs the team could have saved/scored with a perfect lineup and subract actual runs scored/ saved from that, then compare to league average. I would also add some wRAA for intentional BB’s and WPA compared to WPA/LI
In terms of a manager’s situational decision-making, evaluate him based on his tinkering vs. what the status quo would have looked like (i.e. if the manager was completely hands-off and never called for a bunt, mid-inning pitching change, hit-and-run, etc.) For example, if Matt Kemp is up with Dee Gordon on first base and one out, run a simulation of the probability of the Dodgers scoring a run that inning if Kemp is simply allowed to swing away. If Don Mattingly makes a move, like having Gordon steal, and the Dodgers score a run, award Mattingly accordingly (maybe by subtracting the difference between Gordon’s run and the probability of the Dodgers scoring otherwise). If Gordon is thrown out and the Dodgers do not score, deduct points from Mattingly (let’s say the Dodgers had a .3 chance of scoring, meaning Mattingly would lose .3 from his managerial score).
The same concept would apply for a mid-inning pitching change, with the probability of pitcher x giving up a run weighted against the actual outcome, based on either a manager’s action or inaction in going to the bullpen.
Obviously there are a ton of other variables that factor into run-scoring outcomes beside a manager’s moves, and this theory also does not account for the manager’s clubhouse presence or teaching ability.
Why not just actually watch the games and count how many times a decision works out vs how many times it doesn’t? You could use fWAR and the pythagorean W/L, but those have faults themselves. If a guy steals and actually steals, it’s a plus 1, if not, minus 1. You could say “well it depends on the player” which is true, but knowing your talent is part of managing.
Trying to keep it simple and stupid here, and echoing some suggestions above:
How about Team WPA – Team WAR?
Whoops, that’s too simple/stupid, because WPA gives you wins compared to 50/50.
So you’d want (WPA winning %) – (WAR winning %). Something like:
(81 + WPA/2) – (WAR + 50)
(assuming 50 is the wins for a replacement-level team)
Hmm, just tried this over the last five years and I’m getting a 95% correlation between WAR wins and (WAR wins) – (WPA wins)… So all the good teams (BoSox, Yankees, Phillies, etc.) come out last. Can’t tell why, but then it is late and I am stupid. So either I’m screwing something up, or this method won’t work for some probably-obvious reason that I can’t see just yet.
Bill James wrote a book on this in the 1990s – he laid out 15 different statistics to measure managers by and has a variety of essays in the book on the problem of measuring managers and how he developed or decided to use the variables he did.
On a site like this – I would hope that authors would do a bit more homework – it is especially irksome when a post like this one misses something as big as a book on managers by someone who helped to popularize the field of sabremetrics like Bill James.
Bill James manager statistics are in his annual handbook. They mainly tell tendencies and don’t really give you much insight into how good a manger’s decisions are, only how often he did something. I have been meaning to get Jaffe’s book but have not so I don’t know what he did in the book. If you really wanted to break new ground, we should set up something like Bill James’ Project Scoresheet where a few volunteers for each manager would follow the manager and note the 5-10 decisions each game where he makes a decision that is questionable/able to be studied. My guess is this would give us 10-15 issues that could be studied on a leaguewide basis. I think you would have to do this at this level to get a good read. For instance, some formula might take points away from a manager because he didn’t use his lefty relief specialist vs Prince Fielder in the 8th inning. Someone closely following the team might know from watching the postgame interviews or the newspaper that the lefty wasn’t available because his wife had a baby, he threw 25 pitches the night before, had a sore elbow, etc.
There are so many different places to go with this topic, and so much inherent noise in almost all of them, that I wonder if it might be more productive to take an approach similar to Tom Tango’s Fan Scouting Report – allow fans to rate managers on the 20-80 scale in any number of different categories, such as:
Use of in-game offensive strategies (hit and run, sac bunts, etc.)
Use of substitutions (pinch hitters, runners, defensive replacements, double switches)
Leveraging of bullpen
Appropriate workload for relievers
Appropriate workload for starters
Managing/development of young talent
Lineup construction
Control of team, media relations, etc. (the intangibles)
I’m sure I’m forgetting something, and each category could certainly be weighted appropriately to produce a final grade; it’s true that that grade would have a more subjective basis than might be ideal, but I have a hard time envisioning an objective statistic that cuts through the noise cleanly enough to provide a statistically reliable measurement of managing skill.
As will said the annual bill James handbooks have a number of interesting managerial stats. Check it out.
It is hard to measure the moves a manager could have made, but you can look at the decisions they actually follow through on. Looking at things like bunts, stolen bases and pinch hitting, you can see the WPA of making the move compared to not making the move. Baseball isn’t that simple where it is plainly bunt or don’t bunt, pinch hit or leave the guy in, but at least it is something. You can have some kind of accumulated +/- based on the WPA of the manager’s active decisions as compared to the WPA of the passive opposing decision.