The Increasing Over Reliance on WAR

[chop2chip]

I'm not a big fan of long winded rants of any sort. Mainly because I can't justify being that passionate about something that's ultimately meaningless outside of the realm of fandom banter. However, it's interesting reading a few of these threads along with the Insta-Twitter analysis of every move and rumor just to see it ultimately be equated to "Player X projects for 6 WAR over the next three year while Player Y is only 4.5 thus team A is wrong." As an economist with decent analytical expertise I find this to be too simplified.

First of all, most of us (myself included) look at WAR as an output and don't quite understand the ingredients. It's the same pattern I noticed in the days where VORP was the stat de jour. If you don't understand all the mechanisms, how can you possibly justify blindly accepting it as face value? This is especially true considering the year to year noise that WAR has based on the questionable defensive metrics that are baked in. This isn't even addressing the question of how much WAR actually explains wins and losses in the long run. I think it's a fair assumption to concede there are still existing elements of baseball that just can't be encapsulated by a catch all statistic.

So in relating this to the Braves, I think it's fair to criticize the offseason thus far. Jason Heyward was likely every bit the player WAR portrayed him as and Nick Markakis is probably every bit the overpay it seems based off steamer. I do believe, however, it is justifiable to acknowledge that there could be factors involved that allow everybody to give the FO the benefit of the doubt for the time being.

 

[GovClintonTyree]

You had me until your conclusion.

 

[cajunrevenge]

I just don't like how WAR factors in defense. Both for pitchers and hitters. If a starting pitcher puts up a 3.00 era with a 4.00 FIP he had a better year than someone who had a 4.00 era with a 3.00 fip. When we start disregarding actual runs scored in favor of number of runs we think should have scored when talking about present or past stats I think we have gone too far.

If Shelby Miller gives us 200 innings with a 3.40 era I don't give a rats ass if that comes out to 1 war or 6 war.

With Heywood defense relative to offense is hard to calculate. Just going by put outs Heywood made 70 more outs than Markakis in 3 more innings. That's just depressing, but who knows if even Heywood will be that great defensively again. Andrelton couldn't repeat his magical 2013 defensive season. Heywood wasn't this good previously in the field.

 

[mfree80]

I'm not a big fan of long winded rants of any sort. Mainly because I can't justify being that passionate about something that's ultimately meaningless outside of the realm of fandom banter. However, it's interesting reading a few of these threads along with the Insta-Twitter analysis of every move and rumor just to see it ultimately be equated to "Player X projects for 6 WAR over the next three year while Player Y is only 4.5 thus team A is wrong." As an economist with decent analytical expertise I find this to be too simplified.

First of all, most of us (myself included) look at WAR as an output and don't quite understand the ingredients. It's the same pattern I noticed in the days where VORP was the stat de jour. If you don't understand all the mechanisms, how can you possibly justify blindly accepting it as face value? This is especially true considering the year to year noise that WAR has based on the questionable defensive metrics that are baked in. This isn't even addressing the question of how much WAR actually explains wins and losses in the long run. I think it's a fair assumption to concede there are still existing elements of baseball that just can't be encapsulated by a catch all statistic.

So in relating this to the Braves, I think it's fair to criticize the offseason thus far. Jason Heyward was likely every bit the player WAR portrayed him as and Nick Markakis is probably every bit the overpay it seems based off steamer. I do believe, however, it is justifiable to acknowledge that there could be factors involved that allow everybody to give the FO the benefit of the doubt for the time being.

Awesome post. I have been thinking of making such a post for the past few days. You said it better than I could have.

 

[Hawk]

Yes, thank you for the post.

I have a lot of issues with WAR, but I'm far from a numbers guy, so it's nice to get an objective opinion from someone who is used to dealing with them on a more intense level.

 

[dak]

Good post. There are elements of player value that no advanced stats account for. And some of the underpinnings of WAR are imperfect.

But I still feel using projected WAR to assess players is the best tool the average guy like me has to gauge a player's approximate value. We should just acknowledge that it has some limitations.

As Winston Churchill would say if he were a SABR guy (and not dead) . . . "Projected WAR is the worst tool for internet player valuation, except for all the others."

 

[nsacpi]

I'm not a big fan of long winded rants of any sort. Mainly because I can't justify being that passionate about something that's ultimately meaningless outside of the realm of fandom banter. However, it's interesting reading a few of these threads along with the Insta-Twitter analysis of every move and rumor just to see it ultimately be equated to "Player X projects for 6 WAR over the next three year while Player Y is only 4.5 thus team A is wrong." As an economist with decent analytical expertise I find this to be too simplified.

First of all, most of us (myself included) look at WAR as an output and don't quite understand the ingredients. It's the same pattern I noticed in the days where VORP was the stat de jour. If you don't understand all the mechanisms, how can you possibly justify blindly accepting it as face value? This is especially true considering the year to year noise that WAR has based on the questionable defensive metrics that are baked in. This isn't even addressing the question of how much WAR actually explains wins and losses in the long run. I think it's a fair assumption to concede there are still existing elements of baseball that just can't be encapsulated by a catch all statistic.

So in relating this to the Braves, I think it's fair to criticize the offseason thus far. Jason Heyward was likely every bit the player WAR portrayed him as and Nick Markakis is probably every bit the overpay it seems based off steamer. I do believe, however, it is justifiable to acknowledge that there could be factors involved that allow everybody to give the FO the benefit of the doubt for the time being.

It is true that certain important things like grit, leadership, clutchness and maturity can't be measured that easily. Just because they can't be quantified doesn't mean they are not important. We lost of lot of those things with the departure of Laird and I'm glad Markakis (and maybe a yet to be acquired player) are being brought in to make up what we lose with Laird.

 

[thewupk]

I just don't like how WAR factors in defense. Both for pitchers and hitters. If a starting pitcher puts up a 3.00 era with a 4.00 FIP he had a better year than someone who had a 4.00 era with a 3.00 fip. When we start disregarding actual runs scored in favor of number of runs we think should have scored when talking about present or past stats I think we have gone too far.

If Shelby Miller gives us 200 innings with a 3.40 era I don't give a rats ass if that comes out to 1 war or 6 war.

With Heywood defense relative to offense is hard to calculate. Just going by put outs Heywood made 70 more outs than Markakis in 3 more innings. That's just depressing, but who knows if even Heywood will be that great defensively again. Andrelton couldn't repeat his magical 2013 defensive season. Heywood wasn't this good previously in the field.

Heyward's UZR/150 the last 3 years

2012: 21.6
2013: 20.0
2014: 20.5

Yes, DRS (defensive runs saved) has 2014 as his best year yet with a pretty decent gap. I'm not sure if that mark will be reached again but UZR has him pretty much being the same defender the last 3 seasons and that's what's actually used in fWAR. Yes, Heyward could fall off defensively and never be the guy he has been the last 3 years. But that seems unlikely considering he's just now entering his prime.

I also understand the people that say the year to year noise of the defensive numbers doesn't make WAR reliable. You really need a good 3 year sample of defensive numbers to get a good picture of how good or bad a player is. However WAR is there to show what a player did. So while defensive WAR for a player can be skewed in a particular year for a player based on a small sample the same can be said offensive WAR even though the sample is larger. Take Chris Johnson for example. I don't think many people see him as a WRC+ 127 hitter even though that's what he did in 2013. He had a 2.7 WAR and that number is likely going to be the best of his career. So yes he was that type of player in 2013 but likely not going to be that going forward. So there is some context and understanding that has to be used when looking at WAR. But that is true for any stat.

As far as pitching WAR goes I see myself looking at baseball ref's version more these days. FIP is great to predict future performance but it doesn't 100% account for what actually happened. That said I don't buy into ERA 100% either as defense does matter and not all defenses are equal. Luck is luck just like it is for hitters (See CJ 2013).

 

[nsacpi]

I just don't like how WAR factors in defense. Both for pitchers and hitters. If a starting pitcher puts up a 3.00 era with a 4.00 FIP he had a better year than someone who had a 4.00 era with a 3.00 fip. When we start disregarding actual runs scored in favor of number of runs we think should have scored when talking about present or past stats I think we have gone too far.

If Shelby Miller gives us 200 innings with a 3.40 era I don't give a rats ass if that comes out to 1 war or 6 war.

With Heywood defense relative to offense is hard to calculate. Just going by put outs Heywood made 70 more outs than Markakis in 3 more innings. That's just depressing, but who knows if even Heywood will be that great defensively again. Andrelton couldn't repeat his magical 2013 defensive season. Heywood wasn't this good previously in the field.

Here is something to be considered about the defensive stats, including the defensive component of WAR. Each season a player might have an opportunity at close to 400 defensive plays. The vast majority are routine. Some are impossible. The number of challenging plays that can be used to differentiate a good defender from a weak one might be about 50 for an outfielder and a little more for an infielder. It is something to keep in mind when using defensive stats. It would be crazy give much weight to 50 at bats in evaluating a player.

So for the purposes of defensive evaluation you want to look at as much data as possible. But there is a problem with that. If you are using a ten-year average, then you really are using data for different players in the sense that 34 year old player is not the same as his 25 year old self. Maybe he's lost some speed. Maybe he's had an injury or two that had long-term effects. Maybe he has gotten smarter with respect to defensive positioning. But you can't tell how much that can offset physical decline.

I would suggest as a good compromise going 5 years back on defensive data and using that to compute the defensive component of a players value. And of course with projections you need an idea of what defensive aging curves look like. And even after you do all that you need to taking into account whether there are indications in the recent data (and it should go without saying this should be supplemented by what you eyes tell you from watching the games) that a player is losing something defensively, so that the five-year average might be misleading.

I will say that after you do all of that (look at 5 years of data, look for possible recent changes, consider aging curves, and supplement with what your eyes tell you) it is difficult not to conclude that if anything the defensive metrics understate how good Jason has been defensively in recent years and is likely to be over the next two to three years.

With respect to FIP and ERA, the point to be considered is that if you try to predict ERA a year ahead and compare the predictive power of the previous season's FIP and ERA, you find that a player's FIP is a better predictor of his ERA going forward. So to me this means in making projections of future value FIP is definitely preferable.

With respect to Andrelton not being able to maintain his 2013 performance, I'd say some of that was real and some reflected the changes in the pitching staff. We took out two groundball pitchers (Hudson and Medlen) and add two guys who were more flyball pitchers (Harang and Santana). So that reduced the importance of infield defense and increased the importance of outfield defense. Btw it is interesting that we have degraded our outfield defense for 2015 while going with a staff that so far has three flyball pitchers (Miller, Teheran and Minor) and one groundball pitcher (Wood).

 

[BravesfanMike]

What ever happened to " see ball hit ball" ?

 

[weso1]

What impact does the fact that Hewyard played in a park with a deep right center field have on his defensive stats? I've been curious about that and don't know the answer. Seems like maybe he would have a few more opportunities to catch deep fly balls compared to say Markakis who played in a park with a shallow RCF. Also I would think the CF you play with would have an impact as well.

 

[GovClintonTyree]

Good post. There are elements of player value that no advanced stats account for. And some of the underpinnings of WAR are imperfect.

But I still feel using projected WAR to assess players is the best tool the average guy like me has to gauge a player's approximate value. We should just acknowledge that it has some limitations.

As Winston Churchill would say if he were a SABR guy (and not dead) . . . "Projected WAR is the worst tool for internet player valuation, except for all the others."

I would propose halving the dWAR component for infielders and thirding it for outfielders. So if an outfielder is worth 1 oWAR and 6 dWAR, he's worth 3 WAR. Same guy as an infielder is worth 4 WAR.

Hear me out, please, WAR guys, I'm going to go lawyer on you and choose my words very carefully.

The subjective nature of the data collection (at this time - isn't there a new system on the way?), the wild year to year swings (sorry, I just can't see a guy going from dreadful to pretty good to dreadful and explaining that you need to average five years of data, when the average career doesn't last five years), and my own sense after 40 years of playing, coaching and watching tells me it's not possible for a fielder, particularly an outfielder, to impact a game to the extent he can with the bat.

Somebody's got to decide what is routine, what is exceptional, what should have happened in order to derive those numbers. I don't believe there's enough training and consensus to see it close to the same way every time.

Most plays are routine. 80%? 90%? Most guys fielding percentages are well over .900, so the routine play is nearly always made. A right fielder might see three plays a game, 18 a week. So he's got 2 chances a week to make a difference defensively. I'm oversimplifying it because it's much more contextual and subtle than that.

Between the two factors - subjectivity and an (admittedly difficult to quantify) sense that it is not possible to impact a game with defensive that often, I propose tamping down the defensive component. At least until better defensive metrics are available.

Guys, when I played I was a middle infielder, leadoff hitter with no power type. Dirty uniform every game. I had to really work to overcome my shortcomings athletically. Defense was an import way for me to contribute to the team and get on the field. And I was keenly aware that defense was not as valuable as offense. I'm not saying valueless, I'm saying offense is more important in baseball. That's why Justin Upton will be choosing from $200m offers next year and Andrelton Simmons, the best defender I've seen since Omar Vizquel, will be well-compensated at $8m a year.

Not valueless - valued less, and reduced for the inefficiency of the data collection. Halved for infielders and thirded for outfielders.

 

[thewupk]

What impact does the fact that Hewyard played in a park with a deep right center field have on his defensive stats? I've been curious about that and don't know the answer. Seems like maybe he would have a few more opportunities to catch deep fly balls compared to say Markakis who played in a park with a shallow RCF. Also I would think the CF you play with would have an impact as well.

I think playing in a park with a shallow RCF makes you look better. You don't have as many plays to screw up on. And having a great defensvie CF will make the corner outfield guys look better as well.

 

[dak]

What impact does the fact that Hewyard played in a park with a deep right center field have on his defensive stats? I've been curious about that and don't know the answer. Seems like maybe he would have a few more opportunities to catch deep fly balls compared to say Markakis who played in a park with a shallow RCF. Also I would think the CF you play with would have an impact as well.

Great question. Wish I knew. The other thing I LOVE about Heyward is his desire to get to EVERY ball possible, even when it was probably the CF or the 2Bs ball to field. Especially last year, it seemed that BJ and our collective of 2Bs were deferring to Heyward a fair amount. I've wondered about the effect of this dynamic on Jason's UZR. In other words, will his UZR decline a bit if he's simply surrounded by better, more aggressive defenders.

 

[nsacpi]

One thing that I haven't seen discussed much but that I think leads to the statistics understating the value of elite defenders is something that could be called "fielding externalities." An outstanding defender with great range allows changes to the positioning of other players that can improve overall defensive outcomes without the player getting credit for it.

For example, in the outfield Jason's range allows BJ to shade over to left more, which means we get better results on balls hit to left center. I'm pretty sure the defensive metrics do not give Jason credit for this.

Same with the infield. With Simmons at short, CJ plays closer to the bag which cuts down on doubles down the line. I don't think Simmons defensive data reflect that.

 

[GovClintonTyree]

I think playing in a park with a shallow RCF makes you look better. You don't have as many plays to screw up on. And having a great defensvie CF will make the corner outfield guys look better as well.

Sure. But the park with a deep RCF will make it more possible to make exceptional plays. So it cuts both ways, doesn't it?

And I've heard the "guy next to this guy is good, so this guy looks better" argument a lot. If a great CF ranges into RCF and takes a ball that the RF would have possibly gotten, how does that make the RFs defensive metrics look better? I don't think it does. The team is better off, but not the RFs defensive rating.

And in the infield, similar observation - "CJ looks better on the metrics because Andrelton plays next to him..." No, he doesn't. CJ gets to the ball or he doesnt. There will be a few to the left that he doesn't get to that Andrelton will and that will improve the team's defense. But not CJs.

 

[GovClintonTyree]

One thing that I haven't seen discussed much but that I think leads to the statistics understating the value of elite defenders is something that could be called "fielding externalities." An outstanding defender with great range allows changes to the positioning of other players that can improve overall defensive outcomes without the player getting credit for it.

For example, in the outfield Jason's range allows BJ to shade over to left more, which means we get better results on balls hit to left center. I'm pretty sure the defensive metrics do not capture this.

Same with the infield. With Simmons at short, CJ plays closer to the bad which cuts down on doubles down the line. I don't think Simmons defensive data reflect that.

Right. And if you tried, you'd have a whole 'nother layer of subjectivity.

 

[GovClintonTyree]

Great question. Wish I knew. The other thing I LOVE about Heyward is his desire to get to EVERY ball possible, even when it was probably the CF or the 2Bs ball to field. Especially last year, it seemed that BJ and our collective of 2Bs were deferring to Heyward a fair amount. I've wondered about the effect of this dynamic on Jason's UZR. In other words, will his UZR decline a bit if he's simply surrounded by better, more aggressive defenders.

Absolutely true. And the desire not to get trucked by a 6'5", 245 pound man running full speed will put a damper on your metrics real quick.

 

[Julio3000]

[video=youtube_share;vQkz_sGmQZ4]http://youtu.be/vQkz_sGmQZ4[/video]

 

[nsacpi]

I would propose halving the dWAR component for infielders and thirding it for outfielders. So if an outfielder is worth 1 oWAR and 6 dWAR, he's worth 3 WAR. Same guy as an infielder is worth 4 WAR.

Hear me out, please, WAR guys, I'm going to go lawyer on you and choose my words very carefully.

The subjective nature of the data collection (at this time - isn't there a new system on the way?), the wild year to year swings (sorry, I just can't see a guy going from dreadful to pretty good to dreadful and explaining that you need to average five years of data, when the average career doesn't last five years), and my own sense after 40 years of playing, coaching and watching tells me it's not possible for a fielder, particularly an outfielder, to impact a game to the extent he can with the bat.

Somebody's got to decide what is routine, what is exceptional, what should have happened in order to derive those numbers. I don't believe there's enough training and consensus to see it close to the same way every time.

Most plays are routine. 80%? 90%? Most guys fielding percentages are well over .900, so the routine play is nearly always made. A right fielder might see three plays a game, 18 a week. So he's got 2 chances a week to make a difference defensively. I'm oversimplifying it because it's much more contextual and subtle than that.

Between the two factors - subjectivity and an (admittedly difficult to quantify) sense that it is not possible to impact a game with defensive that often, I propose tamping down the defensive component. At least until better defensive metrics are available.

Guys, when I played I was a middle infielder, leadoff hitter with no power type. Dirty uniform every game. I had to really work to overcome my shortcomings athletically. Defense was an import way for me to contribute to the team and get on the field. And I was keenly aware that defense was not as valuable as offense. I'm not saying valueless, I'm saying offense is more important in baseball. That's why Justin Upton will be choosing from $200m offers next year and Andrelton Simmons, the best defender I've seen since Omar Vizquel, will be well-compensated at $8m a year.

Not valueless - valued less, and reduced for the inefficiency of the data collection. Halved for infielders and thirded for outfielders.

I don't think WAR is predicated upon the concept that defense is as important as offense. It is predicated on the idea that run prevention is as important as run production. But a major part of run prevention is pitching. So the defensive component's overall value is the value of run prevention minus the contribution that pitching makes to run prevention. And this is in part why a stat like FIP (fielding independent pitching) fits conceptually.