Unifying Replacement Level: A Conversation With Sean Forman

WAR values have long been discussed by those of us within the sabermetric community and often times it has been hotly debated because there are three different versions of WAR(P), which means three different methods of valuing players. Beyond the Box Score’s very own Bryan Grosnick introduced us to the WAR Index last year as he decided to develop a method for comparing the three different methodologies on the same scale.

Now that Baseball Reference and FanGraphs have agreed on using the same replacement level for their calculations of WAR, at least that portion of it, we have come to a moment in sabermetric history where we inch that much closer to further legitimizing WAR as a valuable tool for evaluating overall player value and performance that everyone can appreciate.

Below is my conversation with Sean about how the discussion on replacement level came about and what else may be coming from him and Baseball Reference.

Lance Rinker: How did this (agreement on replacement level) come about with FanGraphs?

Sean Forman: Well, I had thought about it for a while. I think we both wanted to increase the viewed legitimacy of WAR, get more use from the public, and try to be sensitive to some of the critiques that people have of it. I saw a couple notes where people were complaining about the two systems not matching up and one of the big aspects of that is replacement level.

Replacement level is one of those things where there are a number of arguments as to where you should set it and I thought it would be a place where we might be able to agree on a middle road that would serve both of us and have long term benefits for the community.

Lance: As far as the major differences, do you think that replacement level is one of the easiest areas to come to terms as far as an agreement is concerned?

Sean: Yes, I think it is. Replacement level, I think it’s a good concept in that it gets out the value of what an average player has, but where exactly you set replacement level I think is open to interpretation among different groups.

So, if we agreed that it was a reasonable place and we also talked to Tom Tango and saw what he thought and based on that it seemed like a middle course between where we both were originally would be suitable and would be a good choice for what we were looking to do.

Lance: Are you completely happy with the end result here or do you think that more improvement could be made?

Sean: I’m happy with the end result. I think that the only open issues are how does this change over time, especially going historic because we have more of a historic viewpoint and I think FanGraphs is more forward looking and projection oriented and I think that they would agree with that.

So I think for us, what is the relative value between the AL and the NL both now and going back in the past? What about the War years? Where do you set replacement level for the Federal League? What does replacement level even mean in the National Association?

Right now...I feel pretty good, in Major League Baseball, except for perhaps a few Japanese and Cuban players, that we have pretty much all of the top players playing in the league right now so replacement level is a fairly more concrete idea in a league where you have everybody from the world playing.

If you’re looking at the National Association it’s possible there were superstar players playing semi-pro ball and not on a major league team, so what is replacement level when you have that sort of setting?

So, I don’t think they’re super pressing, I don’t think people are super concerned about where Cap Anson rates relative to Joe Morgan but they're interesting questions and something I’d like to get to at some point.

Lance: Do you think there are other areas where you and FanGraphs, or other sites, could come together and make WAR more unified all-around?

Sean: I think it’s going to be a little harder in those situations. I know there are some differences and we have a list on our site (Baseball Reference) of 40-something differences between the various systems. I wouldn't expect a whole lot of movement beyond this.

I think each system has its strengths and obviously we like ours and I think it’s really good and think it’s probably the best out there, but each is going to have their own horse in the race. But we’ll see, I’m certainly open to it if someone has an idea about what we might do.

Lance: Did you have any other discussion with someone such as Baseball Prospectus regarding that, or was it just Tom Tango (and FanGraphs)?

Sean: We certainly made other groups aware of it and asked them for some feedback and I think it’s ultimately, obviously, up to each site what they decide to do with those changes. They may have their own reasons for going in one direction or the other so I certainly don’t want to speak for anybody else in terms of how they do things.

Lance: Was there anyone that just decided they didn't want any part of it though, or were some of the ideas floating around not really mesh with what you and David Cameron wanted to do?

Sean: There were some discussions but nothing very substantial about what they may or may not do. David Appelman and Dave Cameron and I had lunch at the SABR Analytics meeting and basically just discussed it. We had discussed it a little over email before that and when we kind of decided where to set it and let other people know, it was really up to them to decide whether they wanted to go along with that or not.

Lance: So you guys already had this in the bag so-to-speak, or had decided upon what you wanted to do then?

Sean: Right, and we had reached out and talked a little bit with Colin Wyers about what he thought was a reasonable approach, but like I said I don’t want to talk for Colin because he’s a really bright guy and knows what he’s doing but we did reach out to him at one point.

Lance: How long did you guys have this in the works? Was it going on for a while or did you sit down, have a conversation, and really worked it out?

Sean: I think I sent them an email in February and I was pretty confident they were going to the SABR Analytics (conference) so I kind of figured we could get something settled by then.

Lance: Is there anything in the works for you or Baseball Reference?

Sean: There are a couple of things; One, I'm pretty sure we’re going to be able to get together historical Japanese baseball stats back to the 1930’s or 1940’s this summer and at some point have that up on the site. Beyond that, I've been looking at the minor leagues and trying to flesh out some of the minor league stuff a little bit, maybe add a few more sabermetric things to the minor league area of the site.

In terms of, kind of, broad areas we really don’t have anything on baseball executives, we really don’t have anything on ballparks, our manager stuff is very simple at this point so I think adding stuff in those areas would be interesting to have.