The Increasing Over Reliance on WAR

[GovClintonTyree]

Here is something to be considered about the defensive stats, including the defensive component of WAR. Each season a player might have an opportunity at close to 400 defensive plays. The vast majority are routine. Some are impossible. The number of challenging plays that can be used to differentiate a good defender from a weak one might be about 50 for an outfielder and a little more for an infielder. It is something to keep in mind when using defensive stats. It would be crazy give much weight to 50 at bats in evaluating a player.

I will say that after you do all of that (look at 5 years of data, look for possible recent changes, consider aging curves, and supplement with what your eyes tell you) it is difficult not to conclude that if anything the defensive metrics understate how good Jason has been defensively in recent years and is likely to be over the next two to three years.

This is exactly the starting point I use to say that runs prevented will not have as much value as runs created in a player's WAR calculation, then you reach a completely different conclusion.

I would say of the 50 plays where the fielder can make a difference Jason is the very best and makes .... 80% of them (maybe). So maybe he saves 40 runs. On the other side, the offensive side, he creates, I don't know, 100 runs? You guys know the specifics better than me.

So the very best fielder won't have the chance to prevent as many runs as the very best hitter will get a chance to create.

So I think the defensive component of WAR ought to have less value relative to the offensive component. And I see many examples of where this leads to absurd results, which means I question the validity of the statistic.

I'd like a defensive component, but because of its subjectivity and volatility and the relative difference between the deltas in players' ability to prevent and create runs, I would suggest agreeing to a percentage, different ones for different positions, to discount dWAR.

You discounted Jason's overall WAR for his dWAR arbitrarily a few weeks ago for a particular illustration you made. It struck me as fair(er).

Why not agree to a convention? Help me understand.

 

[GovClintonTyree]

As I mentioned earlier. It's not equal in the amount that it applies to WAR. An elite hitter will produce 50+ runs WAR while an elite defender only provides around 20. So based on the term of 'elite'. Hitters are more valuable and I don't think anyone has ever argued against that. I think the problem occurs with some is when you see someone with great offense and ok defense rated the same or below as someone with good offense and elite defense.

Right. To me, this is a huge problem. A huge problem. It is THE problem. And I can't minimize it, and I can't dismiss it.

You don't agree with fixing the stat by under weighting the more volatile, less predictive component.

How would you fix the stat?

The "democracy is the worst form except all the others" is not a valid answer.

 

[GovClintonTyree]

I think it's fair to keep the same formula. You don't want to interfere with the theory because that would be bad science. I just see it as more of an indictment on the concept of WAR itself.

I know what you're driving at here - a run prevented should be valued the same as a run created. But I think it's fair to discount for subjectivity and volatility of one component versus another and still be good science.

Call it adjusted WAR. aWAR. Like FIP and xFIP. Don't throw out raw WAR, but acknowledge it may not value the player(s) as accurately as aWAR.

 

[nsacpi]

...and so you're going to attribute that to Simmons' impact. What if CJ actually had (shocking, but possible) more range than Chipper?

See what I mean? Somebody's going to make a judgement.

CJ wasn't on the team during Simmons' rookie season. So the point I was making did not relate to CJ. It related to the fact that our team infield defensive stats improved enormously after Simmons became the shortstop and that a significant chunk of the improvement showed up in other players defensive numbers improving (post Simmons versus pre Simmons) during that season. This is what I mean by a "defensive externality." An elite player led to improvements in team defense that did not show up in his own stats. In effect, his own defensive stats were understating the impact he was having. For elite defenders I suspect it is more likely that the defensive stats understate their value than overstate it.

 

[Metaphysicist]

I just don't like how WAR factors in defense. Both for pitchers and hitters. If a starting pitcher puts up a 3.00 era with a 4.00 FIP he had a better year than someone who had a 4.00 era with a 3.00 fip. When we start disregarding actual runs scored in favor of number of runs we think should have scored when talking about present or past stats I think we have gone too far.

Yes, the pitching WAR on Fangraphs is flawed in this way. But the WAR on baseball-reference is based on actual runs allowed. You should just look at that one instead.

 

[nsacpi]

This is exactly the starting point I use to say that runs prevented will not have as much value as runs created in a player's WAR calculation, then you reach a completely different conclusion.

I would say of the 50 plays where the fielder can make a difference Jason is the very best and makes .... 80% of them (maybe). So maybe he saves 40 runs. On the other side, the offensive side, he creates, I don't know, 100 runs? You guys know the specifics better than me.

So the very best fielder won't have the chance to prevent as many runs as the very best hitter will get a chance to create.

So I think the defensive component of WAR ought to have less value relative to the offensive component. And I see many examples of where this leads to absurd results, which means I question the validity of the statistic.

I'd like a defensive component, but because of its subjectivity and volatility and the relative difference between the deltas in players' ability to prevent and create runs, I would suggest agreeing to a percentage, different ones for different positions, to discount dWAR.

You discounted Jason's overall WAR for his dWAR arbitrarily a few weeks ago for a particular illustration you made. It struck me as fair(er).

Why not agree to a convention? Help me understand.

Let me focus on one sentence in your response: So the very best fielder won't have the chance to prevent as many runs as the very best hitter will get a chance to create.

Now I think we can agree there are more opportunities to make a difference on offense. 700 plate appearances in a season, compared with 50-100 challenging defensive opportunities. No doubt about that. But does it follow that that the range of offensive runs created between a good and poor offensive player should be scaled by a factor of 700 while the range for defensive runs saved should be scaled by 50-100. I don't think it necessarily follows. A thought experiment (an extreme one I admit) will illustrate why. Suppose in 50 challenging defensive plays, the best right fielder saves 100 bases more than the worst right fielder. And in 700 at bats the best right fielder generates 20 more bases than the worst one. I'm not saying this is what it is but in that kind of world defense makes more of a difference. My point is it doesn't necessarily follow that the fact there are fewer consequential defensive plays make defense less important.

In the real world of baseball I'm sure defense is less important by offense. This is because pitching is such a bit part of run prevention. All of the outs generated by strike outs for example involve essentially no defense. All of the outs generated by weak contact involve little defense. So defense should have less weight than offense in a position players valuation. I don't think there is any doubt about that. The question is the appropriate ratio between the two weights. And my point is it isn't as simple as saying 700 plate appearance is 10 times as important as 70 challenging defensive plays.

 

[GovClintonTyree]

CJ wasn't on the team during Simmons' rookie season. So the point I was making did not relate to CJ. It related to the fact that our team infield defensive stats improved enormously after Simmons became the shortstop and that a significant chunk of the improvement showed up in other players defensive numbers improving (post Simmons versus pre Simmons) during that season. This is what I mean by a "defensive externality." An elite player led to improvements in team defense that did not show up in his own stats. In effect, his own defensive stats were understating the impact he was having. For elite defenders I suspect it is more likely that the defensive stats understate their value than overstate it.

I understood and agreed with your point. Did you miss mine? There could be reasons other than Andreltons arrival that made a difference. Maybe Chipper got a cortisone shot that allowed him to move more freely laterally for a few months.

 

[chop2chip]

Let me focus on one sentence in your response: So the very best fielder won't have the chance to prevent as many runs as the very best hitter will get a chance to create.

Now I think we can agree there are more opportunities to make a difference on offense. 700 plate appearances in a season, compared with 50-100 challenging defensive opportunities. No doubt about that. But does it follow that the that the range of offensive runs created between a good and poor offensive player should be scaled by a factor of 700 while the range for defensive runs saved should be scaled by 50-100. I don't think it necessarily follows.

Considering WAR is essentially a glorified counting stat scaling doesn't matter.

 

[Metaphysicist]

You don't agree with fixing the stat by under weighting the more volatile, less predictive component.

I don't think you would agree with this either. Do you think WAR should be based on ERA or FIP? FIP is less volatile and more predictive.

WAR is not a predictive stat. It is supposed to be a sum total of value based on the events of a season. If you start re-weighting the events because you don't like how volatile they are, or you don't like the result, then you are defeating the whole exercise.

The only fix WAR (on b-ref, at least) could use is a better, more reliable defensive metric. But I think DRS is pretty good; it's definitely better than UZR. And until we get sabr-nerds with access to Field F/X data, this is probably the best we can honestly expect.

Fangraphs WAR is worse for both pitchers and hitters. Nobody should use it.

 

[depley]

If Shelby Miller gives us 200 innings with a 3.40 era I don't give a rats ass if that comes out to 1 war or 6 war.

DITTO!

With Heywood defense relative to offense is hard to calculate. Just going by put outs Heywood made 70 more outs than Markakis in 3 more innings. That's just depressing, but who knows if even Heywood will be that great defensively again. Andrelton couldn't repeat his magical 2013 defensive season. Heywood wasn't this good previously in the field.

Was Heyward a great defensive RF? absolutely, is defense in RF worth 20+M a year when you look at his hitting? No way. Heyward has potential, well potential and a dollar will buy you a cup of coffee at MacDs

Heyward might end up producing what people think he should but other than his first year he has not shown the ability to do that, you cannot pay people on what you think they might do someday, only real results count. And Heyward was good but not great and certainly not worth 20+M a year at this point, not until he proves he can do the things everyone thinks he can.

 

[nsacpi]

Was Heyward a great defensive RF? absolutely, is defense in RF worth 20+M a year when you look at his hitting? No way. Heyward has potential, well potential and a dollar will buy you a cup of coffee at MacDs

Heyward might end up producing what people think he should but other than his first year he has not shown the ability to do that, you cannot pay people on what you think they might do someday, only real results count. And Heyward was good but not great and certainly not worth 20+M a year at this point, not until he proves he can do the things everyone thinks he can.

I think you are off by about 5M. My understanding is the Braves valued Jason more highly than they did Freddie (who they are paying 21M/year in his free agency years). So it wasn't a question of 20+, but rather 25+. The Braves were south of that and Heyward's agent was asking for north of that. That's my take anyway. ymmv

 

[nsacpi]

Considering WAR is essentially a glorified counting stat scaling doesn't matter.

Yes it is a counting stat. My point had to do with looking at the range of outcomes on offense and defense among major league right fielders, whether the two ranges (the counting stats on both offense and defense) should be tightly tied to the fact that the ratio of plate appearances to challenging defensive opportunities is something like 7:1.

 

[GovClintonTyree]

Let me focus on one sentence in your response: So the very best fielder won't have the chance to prevent as many runs as the very best hitter will get a chance to create.

Now I think we can agree there are more opportunities to make a difference on offense. 700 plate appearances in a season, compared with 50-100 challenging defensive opportunities. No doubt about that. But does it follow that that the range of offensive runs created between a good and poor offensive player should be scaled by a factor of 700 while the range for defensive runs saved should be scaled by 50-100. I don't think it necessarily follows. A thought experiment (an extreme one I admit) will illustrate why. Suppose in 50 challenging defensive plays, the best right fielder saves 100 bases more than the worst right fielder. And in 700 at bats the best right fielder generates 20 more bases than the worst one. I'm not saying this is what it is but in that kind of world defense makes more of a difference. My point is it doesn't necessarily follow that the fact there are fewer consequential defensive plays make defense less important.

In the real world of baseball I'm sure defense is less important by offense. This is because pitching is such a bit part of run prevention. All of the outs generated by strike outs for example involve essentially no defense. All of the outs generated by weak contact involve little defense. So defense should have less weight than offense in a position players valuation. I don't think there is any doubt about that. The question is the appropriate ratio between the two weights. And my point is it isn't as simple as saying 700 plate appearance is 10 times as important as 70 challenging defensive plays.

Ok. I'll buy that. So how about taking a shot at the ratio between the two weights?

 

[GovClintonTyree]

I don't think you would agree with this either. Do you think WAR should be based on ERA or FIP? FIP is less volatile and more predictive.

WAR is not a predictive stat. It is supposed to be a sum total of value based on the events of a season. If you start re-weighting the events because you don't like how volatile they are, or you don't like the result, then you are defeating the whole exercise.

The only fix WAR (on b-ref, at least) could use is a better, more reliable defensive metric. But I think DRS is pretty good; it's definitely better than UZR. And until we get sabr-nerds with access to Field F/X data, this is probably the best we can honestly expect.

Fangraphs WAR is worse for both pitchers and hitters. Nobody should use it.

It looks back to value, so it is not predictive. Then people use it to predict how a player will perform going forward. "Jason Heyward had 6 WAR last year, and Shelby Miller only 2, therefore the Cardinals won the trade."

 

[Knucksie]

It's useful as as measurement. It also runs the risk of being the "flavor of the month" for the amateur posters, who believe themselves to be experts on advanced metrics. First it was OBP. Then it was OPS (with walks suddenly coming into vogue). Now, it's WAR - with its multiple variants.

 

[thewupk]

Right. To me, this is a huge problem. A huge problem. It is THE problem. And I can't minimize it, and I can't dismiss it.

You don't agree with fixing the stat by under weighting the more volatile, less predictive component.

How would you fix the stat?

The "democracy is the worst form except all the others" is not a valid answer.

I mean it depends. A good hitter with elite defense can be +40 runs above replacement. 20 on offense and 20 on defense. A great hitter with say average defense could also be on the +40 runs above replacement. Say 35-40 on offense and 0-5 on defense. So they could both provide the same amount or runs above replacement. Then you have to factor in things like playing time and the position they play.

To me a run created is the same as a run saved. Elite defenders will never be able to save as many runs as an elite hitter will be able to create. But that doesn't mean the runs they do save or create aren't equal.

 

[chop2chip]

Yes it is a counting stat. My point had to do with looking at the range of outcomes on offense and defense among major league right fielders, whether the two ranges (the counting stats on both offense and defense) should be tightly tied to the fact that the ratio of plate appearances to challenging defensive opportunities is something like 7:1.

Well inherently offense will be be a heavier component. ITd not necessary to inflate one's importance over another.

Just the same I don't see any purpose to what Gov was suggesting of tinkering with the theory. Just take the stat at face value and if you are actually going to analyze players, then just don't use WAR. There are far less volatile offensive statistics and using multiple defensive metrics over multiple years corrects for a portion of the unreliability. But Still give FO's the benefit of the doubt when it comes to defensive evaluations because they simply have much better sources of data then us especially in relation to defense.

 

[GovClintonTyree]

I don't think you would agree with this either. If you start re-weighting the events because you don't like how volatile they are, or you don't like the result, then you are defeating the whole exercise.

If "I don't like the result" because I get absurd results, as an empiricist I owe it to my work to question, review, and improve my model.

You seem to be suggesting I take whatever this invented formula is as truth. I don't think that's wise.

I remember a few years ago, in Matt Kemp's (non-steroid) MVP year, he was absolutely hitting the dog**** out of the ball first half with average defense. Michael Bourn was having a great year before completely tailing off. But the WAR calc in July said Michael Bourn had created and prevented more runs than Kemp. Ain't no way.

 

[chop2chip]

I mean it depends. A good hitter with elite defense can be +40 runs above replacement. 20 on offense and 20 on defense. A great hitter with say average defense could also be on the +40 runs above replacement. Say 35-40 on offense and 0-5 on defense. So they could both provide the same amount or runs above replacement. Then you have to factor in things like playing time and the position they play.

To me a run created is the same as a run saved. Elite defenders will never be able to save as many runs as an elite hitter will be able to create. But that doesn't mean the runs they do save or create aren't equal.

I agree 100% with this in theory. My whole premise is that it's highly unlikely that run prevention for position players is accurate. I Agree that it's the best we currently have but that doesn't make it a good measure.

 

[Metaphysicist]

It looks back to value, so it is not predictive. Then people use it to predict how a player will perform going forward. "Jason Heyward had 6 WAR last year, and Shelby Miller only 2, therefore the Cardinals won the trade."

That is over-simplistic, yes. But it is predictive to whatever degree things in the past tend to continue in the future, and WAR represents past events the best. I'm specifically comparing it to stats like FIP which does NOT represent run saved in the past, but which are designed to predict future events.