Some managers are more active in managing a game than others. Earlier this month, Ben Lindbergh posted the 2015 results of his Manager Meddling Index, which is “not a ranking of the worst and best managers; it’s more like a ranking of the least and most managers.” John Farrell and Torey Lovullo did not rank among the game’s most meddling managers in most categories, but the Red Sox did stick out in two categories in particular: hit and runs, and replay challenges. On the latter, it looks like the team may be onto something.
Umpiring is inconsistent, and it’s not like replay-worthy plays follow a regular schedule. Yes, the Red Sox upped their challenge total from 35 in 2014 to 52 in 2015, but it could just be that there were more calls worth challenging. It could also be, though, that the Red Sox revised their threshold for what that means, challenging more calls that were on the fence. “Clear and convincing evidence” is one of those objective standards that can only be applied subjectively. It’s possible that the Red Sox decided to go for the replay when the chances a play would get overturned seemed like they were “probably not, but maybe.”
If the idea is to win as many challenges as possible, then we don’t care at all about a team’s rate of success with its challenges; we care only about maximizing the number of overturned calls.
As the number of challenges rose by 114 in 2015 (an increase of 11 percent), the number of successful challenges also rose, from 556 to 610 (an increase of 10 percent). If the idea is to win as many challenges as possible, then we don’t care at all about a team’s rate of success with its challenges; we care only about maximizing the number of overturned calls. Two seasons’ worth of challenges can’t prove that more challenges means more overturned calls, but it looks to me like they can prove that it might mean that.
Rise Across the Game
The number of replay challenges was up this season, from 1052 to 1166 — not a very big difference, about one challenge per team per four games (.24 per team game). That 11 percent increase seems pretty significant over nearly 5,000 team-games in a season. The Red Sox may have had more bad calls go against them in 2015, but that doesn’t seem to work as an explanation for the rise of replay challenges across the game. It’s a little more than that.
Take a look at the number of challenges issued by each team in 2015:
Do you see what I see? Some of the teams generally considered to be forward-thinking are near the top of the list: the Astros, Cubs, and Rays, in addition to the Red Sox. Of those four, two dramatically increased their number of challenges this season (Astros +18; Red Sox +17). The other two were also among the league leaders last season (Cubs 1st, +21 over league average; Rays T-2nd, +13 over league average). The Pirates also saw a healthy increase, for what it’s worth. And if you want to make your own “forward-thinking” determinations, here are all the team deltas and totals for both seasons, retrieved from the incredible Baseball Savant.
That may not be clear and convincing that it’s smart to challenge more plays than the average team, but maybe it is “evidence.” The total number of plays that could be reviewed (more or less, total batters faced by pitchers + total PA) also clearly can play a role here, dragging teams with good pitching and meager offenses down toward the bottom of the totals, and those with poor pitching and strong offenses toward the top. Those don’t actually change a whole lot, though, and didn’t for the Red Sox (12,461 BF + PA in 2015, 12,457 BF + PA in 2014). It looks like there’s more going on.
The Downside
During the regular season, managers get just one replay challenge to play with per game (in the postseason, it’s two), and an unsuccessful challenge means that opportunity is gone. One thing you’d want to know is how likely it is that you’d regret losing your ability to challenge later in the game. That’s almost no consideration at all in the ninth inning, but it has to be enormous in the first. We can actually look at that one! With challenges issued about once every four games, maybe it’s not so hard to tell how likely it is you’d want to challenge a play later in a particular game. Halfway through? One in eight. First play in the fourth? About one in six. But that’s probably not the right question, because you wouldn’t pass on an iffy challenge only because you might get as iffy a chance later in the game.
Instead: you probably want to know the chances that a call will be made that’s not iffy. That’s the nightmare: don’t let the guys with the calculators push you into challenging a call with an iffy chance of success in the second inning, because if you lose, you might end up helplessly watching a clearly blown call later in the game, grilled by a glut of reporters and torched on talk radio. To figure that out, I think we want to use the rate of overturned calls via replay.
If the nightmare is a thing that ever happens, then there are at least some obviously blown calls that are not challenged by a team, which would push the number of overturned calls lower than the number we want. And yet, we’re starting with the premise that issuing more challenges with iffy chances of success could turn into more success, and we just said that those aren’t part of the nightmare we’re worried about; those overturns would push the number higher than what we want.
Combining both seasons’ data, there were 0.12 overturned calls per game, a little less frequently than one every eight games. It seems, then, that even in the most extreme circumstance—the decision of whether to challenge the very first play of the game—the downside can’t get bigger than that one-in-eight chance.
Is it Smart to Challenge More Plays?
So all we have so far is: an 11 percent increase in challenges was met with a 10 percent increase in overturned calls, and the downside of challenging an iffy play is a chance of somewhere between 12 percent and zero percent (game-ending play) of missing the chance to redress a very redressable wrong later in the game. If there’s a way to game this, the proceeds are not very likely to be high.
If we assume for the sake of argument that an extra iffy challenge has a 10 percent chance of success, then we’re in break-even territory toward the beginning of the game—but from there, it’s a steady march to “no brainer” territory late in the game. It’s hard to go to war for a small and uncertain upside when there’s a small but measurable downside that would result in getting pilloried, but it gets easier as the game advances. If teams were trying to capitalize on this potential return from issuing iffy challenges, it stands to reason that those teams would end up issuing more challenges later in the game. But it just doesn’t look like that’s been true so far: the top teams in terms of total challenges didn’t have them skew toward the late innings. If it is smart to issue more iffy challenges late in games, it doesn’t look like anyone is being smart exactly that way.
It might be too much to ask of the guys who are in the trenches, who already have to weigh the importance of winning a challenge on a particular play, the chances of success, the size of the gap between the truth and what the video might actually show, and all the human factors at work: icing pitchers, backing your players, and how everyone would feel knowing no challenge is available.
In the end, it might be that all we have is a story of the New York review crews getting a little more assertive, and maybe lowering their threshold for clear and convincing evidence. I’m not sure how to measure that. But if that is the explanation for the rise in challenges in the game, that would still be a reason to challenge more plays—and a reason to watch to see how teams handle challenges in 2016.
Photo by David Butler II/USA Today Sports Images
Interesting article. One thing looking at the data confirmed what I have been saying all year, “We need to move replay review out of New York.”
Yankees replay challenge success rate 2014-5 was 78%. Red Sox success rate was 46%. MLB average was 53%. Something smells or at least calls for further review.