The Increasing Over Reliance on WAR

[MadduxFanII]

What a horrible experiement that was

Hey, when you have an opportunity to awkwardly shoehorn Vinny Castilla into your lineup, you have to do it.

[GovClintonTyree]

I don't mean be assholish; I'm just being honest. If you tell me you aren't going to bother to understand the think you are criticizing, I don't expect your criticism to be very well grounded. That's true about basically anything.

Gentlemen of the Chop Country Statistical Society:

Meta was right. I needed to understand WAR better, the components and the derivation of the stat. So I've spent a bit of time researching it, and it is a complex and well-devised stat. It will benefit from fieldF/X, I'm certain.

WAR must include a defensive component, we all agree. There are differing opinions on the usefulness of the current DRS system. I voiced skepticism because I didn't agree with some of the results. More specifically, I didn't agree with the degree of impact that defense would have on a player's WAR. As you'll recall, my initial proposal was that we should discount the fielding element of WAR by a half to two-thirds depending upon the position.

I was basically ridiculed by a lot of you. Labelled a Luddite, called an old man, told that I shouldn't use stats if that's how I read them, told that my suggestion of discounting would be bad science, and, by Meta, told that I didn't know what I was talking about and should shut up.

In my research, I ran across this conclusion in an article by John Dewan on FieldingBible.com:

Estimating Runs and Wins

One thing we want to do in the future is translate these plus/minus numbers and all the other defensive metrics we have in this book into one number. That might be a number similar to Runs Created, but for defense not offense. Maybe it’s called Runs Prevented. But between you and me, you can use the rule of thumb that Bill James used in his article on Derek Jeter and Adam Everett.

That is, use a number a little less than half of the plus/minus number as an estimate of runs prevented. Since the value of a single is a little less than half a run, you can use a “little less than half” of the plus/minus figure to estimate runs prevented. Adam Everett’s plus/minus figure of +33 could be estimated as preventing about 15 runs. Then using another rule of thumb that estimates the value of a win at 10 runs, Everett’s defense generates an extra 1½ wins for the Astros in 2005. Since the Enhanced Plus/Minus System also factors in the value of extra bases, and each extra base is worth somewhat less than a single, you might use an even lower value (.20 for each plus/minus, perhaps?) for the difference between the Enhanced value and the Basic value. Since defense is not an exact science, however, I would suggest that using half and rounding down for both Basic and Enhanced Plus/Minus is close enough as an estimate. After all, it’s not horseshoes or hand grenades.

So, guys, if John Dewan suggests discounting the stat at 1/3 to 1/2 its raw weight, will you consider his proposal?

Respectfully, your Luddite old man moron buddy,

GovClintonTyree

[chop2chip]

Nice post

I disagree with arbitrary discounting. I understand the volatility aspect, but I maintain it defeats the purpose of the exercise. What discounting is essentially saying is that a run prevented is not worth a run created which contradicts the logic behind having a defensive component of WAR.

I understand what Mr. Fielding Bible is suggesting. What he's stating is that that particular metric exaggerates the number of runs prevented. He's suggesting a potential fix. I personally disagree with his method of arbitrary weighting since better, more accurate methods are possible (such as regression analysis)

[Metaphysicist]

I was basically ridiculed by a lot of you. Labelled a Luddite, called an old man, told that I shouldn't use stats if that's how I read them, told that my suggestion of discounting would be bad science, and, by Meta, told that I didn't know what I was talking about and should shut up.

At no point did I tell you to "shut up." Come now, let's be fair.

I was basically ridiculed by a lot of you. Labelled a Luddite, called an old man, told that I shouldn't use stats if that's how I read them, told that my suggestion of discounting would be bad science, and, by Meta, told that I didn't know what I was talking about and should shut up.

In my research, I ran across this conclusion in an article by John Dewan on FieldingBible.com:

Estimating Runs and Wins

One thing we want to do in the future is translate these plus/minus numbers and all the other defensive metrics we have in this book into one number. That might be a number similar to Runs Created, but for defense not offense. Maybe it’s called Runs Prevented. But between you and me, you can use the rule of thumb that Bill James used in his article on Derek Jeter and Adam Everett.

That is, use a number a little less than half of the plus/minus number as an estimate of runs prevented. Since the value of a single is a little less than half a run, you can use a “little less than half” of the plus/minus figure to estimate runs prevented. Adam Everett’s plus/minus figure of +33 could be estimated as preventing about 15 runs. Then using another rule of thumb that estimates the value of a win at 10 runs, Everett’s defense generates an extra 1½ wins for the Astros in 2005. Since the Enhanced Plus/Minus System also factors in the value of extra bases, and each extra base is worth somewhat less than a single, you might use an even lower value (.20 for each plus/minus, perhaps?) for the difference between the Enhanced value and the Basic value. Since defense is not an exact science, however, I would suggest that using half and rounding down for both Basic and Enhanced Plus/Minus is close enough as an estimate. After all, it’s not horseshoes or hand grenades.

So, guys, if John Dewan suggests discounting the stat at 1/3 to 1/2 its raw weight, will you consider his proposal?

I am very happy that you have read up on these things. Having everybody on the same page makes these kinds of discussion a lot less pointless.

However, you are misreading what is being said. Dewan is NOT saying to discount the defensive component of WAR. "Plus/Minus" is a measure of plays, not runs. The above quote describes a rough but simple method to convert those "plays" into "runs." Essentially, this passage is saying that saving one single is worth about half a run.

But WAR doesn't use straight "plus/minus" numbers; in fact, it already includes a much more precise conversion of plays into runs. That's what DRS is: Defensive Runs Saved, and that's number that goes into WAR. To quote fieldingbible.com:

In order to translate each component to Runs Saved, we consulted the “24-States Run Matrix”. We compared the expected number of runs allowed before and after each play and calculated the average change in run expectancy for each event. We then apply these average run values to convert to Runs Saved.

Basically, the discount described is already built into WAR, only done in a more exacting way.

[Metaphysicist]

I understand what Mr. Fielding Bible is suggesting. What he's stating is that that particular metric exaggerates the number of runs prevented. He's suggesting a potential fix. I personally disagree with his method of arbitrary weighting since better, more accurate methods are possible (such as regression analysis)

That's not what he's saying. He's saying this metric measures plays, not runs. Thus, if you want a measure of runs saved, you will need to convert those plays to runs. DRS (and WAR) already do this.

[gilesfan]

The main issue with this(at least for me) is that we have a clear understanding of how many runs a player generates on offense. He drives in 100 runs and scores 100 runs. Thus, he has created 200 runs. We can see that.

You actually believing this makes me really scratch my head, but I should have expected it.

[gilesfan]

Now you're just being an asshole. Why are you so vested in this?

Gerardo Parra, 2013. Also find it interesting that he's worth 4 dWAR that year, then he's minus the next year. How can I possibly figure out what to spend based on that? At $9m a WAR in ***y this off season, he was worth $36m for defense alone that year! And now he's negative? I overpaid!

He wasn't that good then and he isn't that bad now.

Do defender splay to the same level every year?

[Metaphysicist]

Do defender splay to the same level every year?

He do splay.

[gilesfan]

He do splay.

That athleticism is off the charts.

[GovClintonTyree]

That's not what he's saying. He's saying this metric measures plays, not runs. Thus, if you want a measure of runs saved, you will need to convert those plays to runs. DRS (and WAR) already do this.

But Meta, the year referenced in Mr. Dewan's article - 2005 Adam Everett - he refers to Everett's plus-minus as plus 33. Then he suggests discounting the impact to a little less than half, 15 runs, and suggests 1.5 dWAR is a fair approximation of Everett's impact.

When I look at Everett's actual dWAR for 2005, I see that it is 3.3. Help me understand.

[thewupk]

But Meta, the year referenced in Mr. Dewan's article - 2005 Adam Everett - he refers to Everett's plus-minus as plus 33. Then he suggests discounting the impact to a little less than half, 15 runs, and suggests 1.5 dWAR is a fair approximation of Everett's impact.

When I look at Everett's actual dWAR for 2005, I see that it is 3.3. Help me understand.

Not to steal from Meta or anything but on baseball ref's page it actually shows Everett that year as having 25 DRS in 2005. So of those +33 plays he saved roughly 25 runs comapred to the average shortstop. Not the half Dewan was suggesting but it is discounted. As for as why Evertt showed a dWar of 3.3 that year. dWar jsut isn't DRS divided by 10. The positional dificulty gets added in as well which for shortstops is +7. And the amount of runs per win in a given season is always different. Thus the 25 DRS + 7 positonal ranking equals 32 comes out to a dWar of 3.3. So why it's close to 10 it's not exact and is generally used as a rough estimate.

[chop2chip]

That's not what he's saying. He's saying this metric measures plays, not runs. Thus, if you want a measure of runs saved, you will need to convert those plays to runs. DRS (and WAR) already do this.

That's fair. I misinterpreted that. It still seems though that he's using a pretty crude measure to convert the number of plays into runs.

[thewupk]

That's fair. I misinterpreted that. It still seems though that he's using a pretty crude measure to convert the number of plays into runs.

That article with Dewan that Gov quoted also was written in ~2005. We've learned a lot since then.

[GovClintonTyree]

Not to steal from Meta or anything but on baseball ref's page it actually shows Everett that year as having 25 DRS in 2005. So of those +33 plays he saved roughly 25 runs comapred to the average shortstop. Not the half Dewan was suggesting but it is discounted. As for as why Evertt showed a dWar of 3.3 that year. dWar jsut isn't DRS divided by 10. The positional dificulty gets added in as well which for shortstops is +7. And the amount of runs per win in a given season is always different. Thus the 25 DRS + 7 positonal ranking equals 32 comes out to a dWar of 3.3. So why it's close to 10 it's not exact and is generally used as a rough estimate.

...but none of that is 15. You're saying its discounted, but nowhere near as much as the author of the stat says it should be.

So somebody has to cull through each of those plays and determine a run value to whether it was made or not made? 76% of fielders make that play, a medium speed ground ball in vector 17. It is worth .004 runs, so he gains .0031 runs. 24% of fielders make that play, a hard liner in vector 21. It is worth .006 runs, so he loses .0014 runs.

Is that how it works?

It seems like we're assigning a high degree of specificity to something that really doesn't have as much specificity as we attribute to it.

[thewupk]

...but none of that is 15. You're saying its discounted, but nowhere near as much as the author of the stat says it should be.

So somebody has to cull through each of those plays and determine a run value to whether it was made or not made? 76% of fielders make that play, a medium speed ground ball in vector 17. It is worth .004 runs, so he gains .0031 runs. 24% of fielders make that play, a hard liner in vector 21. It is worth .006 runs, so he loses .0014 runs.

Is that how it works?

It seems like we're assigning a high degree of specificity to something that really doesn't have as much specificity as we attribute to it.

As I said that article you referenced is from around 2005. We've learned a lot since then. And the fielding bible is who comes up with DRS. So while he may have said that in 2005 it is currently not how they do it today. As for what Everett did in 2005 with his plus minus +33. As you went into there each play is not equal. 33 is the overall amount of plays he made more then the average SS. As good as Everett was that season there were plays he messed up on I'm sure. Those plays would have counted negatively against him in plus minus and with the run value of them that go into the DRS. But yes each play is weighted on it's on and summed up at the end. That's an easy way to explain it.

[Runnin]

He do splay.

That ball will be halfway to 1B before most SS could get to their feet.

[Metaphysicist]

But Meta, the year referenced in Mr. Dewan's article - 2005 Adam Everett - he refers to Everett's plus-minus as plus 33. Then he suggests discounting the impact to a little less than half, 15 runs, and suggests 1.5 dWAR is a fair approximation of Everett's impact.

When I look at Everett's actual dWAR for 2005, I see that it is 3.3. Help me understand.

First off, let me be clear: "discounting" is not what is happening here. He is talking about converting plays (e.g., saving a single) into their run values (e.g. a single is worth about .5 runs). This is a conversion, not a discount. No value is being discarded, it is just being converted into a different unit so it can be added to oWAR, which uses offensive runs created. Calling it a "discount" makes about as much sense as saying you "discount" miles into kilometers.

Second, The article you are referencing is from 2005-2006, and was included in the original The Fielding Bible. Published WAR didn't even exist back then (except as a statnerd dream), and the defensive stats were just being formulated in their first incarnations. Dewan has since published The Fielding Bible II and The Fielding Bible III, wherein he further refines his methods. The current procedures for plus/minus and converting to DRS have been changed and presumably improved. I don't know if Everett's plus/minus is still calculated as 33 (I don't believe these are freely published), but I do know that Dewan today says those plays should be converted to 25 runs, or 25 DRS, since those are the numbers he publishes. It is incorrect to attribute any other opinion to him.

Third, as thewupk essentially covered already: those plays are worth 25 runs. That number is then added to a positional adjustment of 7 runs to get dWAR, which is supposed to approximately defensive value across all positions.

[Metaphysicist]

That's fair. I misinterpreted that. It still seems though that he's using a pretty crude measure to convert the number of plays into runs.

Yes, it is an extremely crude back-of-napkin way of calculating the conversion that Bill James made up off the top of his head. This was before the invention of DRS, which is a more rigorous way of doing an accurate runs conversion.

[Metaphysicist]

...but none of that is 15. You're saying its discounted, but nowhere near as much as the author of the stat says it should be.

Again, it's not discounted. He's talking about converting units, and that article from 2005-2006 was a not-particularly-accurate way to roughly estimate that procedure before he developed his real method. Here's the actually methodology of his conversion (link):



So, Dewan says SS plays are worth .76 runs. Let's look at Adam Everett's 33 Plus/Minus plays. 33*.76 = 25, and that's exactly what his DRS is listed as.

So, mystery solved.

So somebody has to cull through each of those plays and determine a run value to whether it was made or not made? 76% of fielders make that play, a medium speed ground ball in vector 17. It is worth .004 runs, so he gains .0031 runs. 24% of fielders make that play, a hard liner in vector 21. It is worth .006 runs, so he loses .0014 runs.

Is that how it works?

Mm... close but not exactly. And it's not a person who does this; it can all be handled by computer.

1) A player only gets plusses for plays he makes, and he only gets minuses for plays he doesn't make. If he makes a play that 24% of other players don't, then he gets +24. If he had missed that play instead, then he would have gotten -76, since 76% of other players made that play.

2) The plus/minus scores for every play all year are then added up.

2) The total plus/minus score is converted to Runs Saved using the plays-to-runs conversion chart I listed above. Those are the average number of runs each play at a position is worth.

It seems like we're assigning a high degree of specificity to something that really doesn't have as much specificity as we attribute to it.

This might be true, I don't know. If I was going to assail DRS, it would probably be something along these lines. It's hard to know exactly since Baseball Info Solutions (Dewan's company) keeps some stuff proprietary.