Beyond the Box Score: An SB Nation Community

Navigation: Jump to content areas:


Sports blogs for fans, by fans.
Around SBN: Spencer Hall's Sports Meme Power Rankings

Saberizing a Mac #3: Adding Win Shares & Other Data to the BDB

Based on the lack of comments in the previous post, I'm going to guess people had no trouble installing the Baseball Databank database.  If you did, feel free to ask questions in the comments of that post.  To follow the process I personally used to get it done, read this comment.  All Saberizing a Mac posts can be found here.

This next challenge was suggested by Jeff (TucsonRoyal).  The Baseball Databank is awesome, but what if there's additional information from other files that we want to add to the database?  How do we do that?  Well, let's figure it out, starting with Win Shares data provided by Dave Studeman.  There are two .csv files in there, both of which include BDB ids, allowing us a way to hook into the data that's already in the BDB.

Let's tackle the total WS file first, then the WS above average.  We have some choices to make.  Do we want this as a separate table in the database, or do we want to add fields to an existing table?  I'm not sure both are even an option with this WS data, but it's an interesting question to discuss.

Star-divide

As a challenge, let's see if everyone can get the data imported and then write a SQL query to find the ten oldest teams since 1980, where team age is defined as the weighted average of each player's age and their total Win Shares.  As an example, to find the 2008 World Champion Phillies' weighted age, you'd multiply Chase Utley's 2008 age by his WS, add in Ryan Howard's 2008 age times his WS, add in the same thing for everybody else, then divide by the total team Win Shares.  (I've chosent his definition because it requires us to use WS data, and weights team age by the players who are actually providing value to the team.  You could also weight by PAs or IPs to find the team age based on playing time instead of performance.)

Calculating age can be a bit tricky, and I'll refer people once again to Colin's two primers on SQL.  Colin also has some additional sample code available here.  He's offered to help folks with their own queries, which can be something else that happens in the comments of this post.  If you challenge yourself to come up with a query to answer an interesting question, but are struggling, ask for help.  If you succeed, share the code.

Once you've got the Win Shares data incorporated, try adding some of the data Tom Tango's made available here, like primary defensive position and wOBA.  And if you find any other data we can add, please share it.

Get your celebratory age-appropriate beverages chilled... set... go!

(And yes, this is something PC-users can play along with, too, although your results won't be as aestheticlly pleasing and you can't share in our yuppy attitudes.)

Poll
Are you playing along at home?
Yes!
13 votes
Yes, but on a PC.
7 votes
Not yet, but I will.
9 votes
I already have this stuff set up.
2 votes
I want to, but I'm scared.
7 votes
No, you're all dorks. (Keep in mind we know this already.)
5 votes

43 votes | Poll has closed

0 recs  |  Comment 27 comments |

Story-email Email Printer Print

Comments

Display:

My answer
Do we want this as a separate table in the database, or do we want to add fields to an existing table?

Definitely separate tables. Otherwise you have major troubles when either piece of data is updated.

If you want to get the data in one easily accessible “table” try creating a view. It doesn’t have any performance improvement over a straight SQL statement, but it saves you having to write the SQL each time.

by Dan Turkenkopf on Jun 10, 2009 9:20 PM EDT reply actions   0 recs

Sounds like good advice.

But what do you know about this sort of thing?

How is a view different from a query?

Beyond the Boxscore // Calling BJ Upton lazy is lazy.

by Sky Kalkman on Jun 10, 2009 9:32 PM EDT up reply actions   0 recs

A view is simply a query that the database remembers.

And without getting too far into the normal forms and whatnot – it’s generally more logical to introduce a new table for these sort of things.

by cwyers on Jun 11, 2009 2:50 AM EDT up reply actions   0 recs

We could get into a normalization discussion though and really confuse Sky

What’s your feeling on the tradeoffs between normalization and performance? :)

by Dan Turkenkopf on Jun 11, 2009 7:45 AM EDT up reply actions   0 recs

When it comes down to it, I prefer performance.

The Retrosheet schema I use is not very normalized at all, and it doesn’t bother me any. (I don’t even think it follows the facts table setup of a data warehouse, but I’m not an expert here.)

by cwyers on Jun 11, 2009 11:24 AM EDT up reply actions   0 recs

Me neither, really

I’ve never bothered to normalize my retrosheet tables either.. they’re basically all 100 or so fields laid out in a single table. It suits my purposes so I haven’t spent a lot of time to make it better.

by Dan Turkenkopf on Jun 11, 2009 6:01 PM EDT up reply actions   0 recs

This first step has worked for me so far

In the Pro Sequel BDB database I created a table from the Query view:

 CREATE TABLE `BDB`.`2008_Historical_WinShares` (
`playerID` VARCHAR ( 10 ) NOT NULL ,
`yearID` VARCHAR ( 6 ) NOT NULL ,
`Age` VARCHAR ( 6 ) NOT NULL ,
`Multiteam?` VARCHAR ( 6 ) NOT NULL ,
`Multi-Same Team?` VARCHAR ( 6 ) NOT NULL ,
`stint` VARCHAR ( 5 ) NOT NULL ,
`teamID` VARCHAR ( 6 ) NOT NULL ,
`lgID` VARCHAR ( 4 ) NOT NULL ,
`Pos` VARCHAR ( 5 ) NOT NULL ,
`WinShares` VARCHAR ( 10 ) NOT NULL ,
`BatWS` VARCHAR ( 10 ) NOT NULL ,
`FieldWS` VARCHAR ( 10 ) NOT NULL ,
`PitchWS` VARCHAR ( 10 ) NOT NULL ,
`TotWS` VARCHAR ( 10 ) NOT NULL
) ENGINE = MYISAM

…then I imported the 2008_Historical_WinShares.csv file (File>Import…). I just went with the default settings.

(At least the data is there; I don’t claim to know what I’m doing. Your mileage definitely may vary.)

by RFK on Jun 11, 2009 2:01 AM EDT reply actions   0 recs

Sweet.

Can you run a query using the table?

Beyond the Boxscore // Calling BJ Upton lazy is lazy.

by Sky Kalkman on Jun 11, 2009 8:46 AM EDT up reply actions   0 recs

Yes, here are the top 50 total Win Share years for the SF Giants, for example:

SELECT playerID
, yearID
, TotWS
, teamID
FROM 2008_Historical_WinShares
GROUP BY playerID
HAVING teamID = “SFN”
ORDER BY TotWS Desc
LIMIT 50;

What the heck, it’s something!

by RFK on Jun 11, 2009 11:47 AM EDT up reply actions   0 recs

Uhhh

yeah, what Colin said!

by RFK on Jun 11, 2009 5:24 PM EDT up reply actions   0 recs

If you want to know the truth

I created the table from the MAMP/phpMyAdmin/Operations/Create new table on database tool. It winds up giving you the code. Copy and paste, baby!

by RFK on Jun 11, 2009 5:32 PM EDT up reply actions   0 recs

And

I just aped Colin’s second code from the historic November 16, 2008 “Building a sabermetrician’s workbench, part II”. Colin is my Zeus!

by RFK on Jun 11, 2009 5:38 PM EDT up reply actions   0 recs

MySQL uses several different storage engines.

That just tells it which one to use. MyISAM is probably the best engine choice for the vast majority of the things that you’d want to use MySQL for (and by you I mean a sabermetrician).

by cwyers on Jun 11, 2009 4:19 PM EDT up reply actions   0 recs

Not really.

Although I can’t tell you what your install will default to (mine will make a table MyISAM by default). It’s not a bad idea to do so, however.

by cwyers on Jun 12, 2009 12:36 AM EDT up reply actions   0 recs

Uhhh

yeah, what Colin said!

by RFK on Jun 11, 2009 5:25 PM EDT up reply actions   0 recs

Challenge I'm working on...

For all teams to make the playoffs since, oh, 1980, where did they rank within their league in total homeruns?

Beyond the Boxscore // Calling BJ Upton lazy is lazy.

by Sky Kalkman on Jun 11, 2009 10:14 AM EDT reply actions   0 recs

FWIW I'm working on this, too

But it might be a few days before I have time to get it together. (At least it’s easier than the Ten Oldest Teams Since 1980 Challenge.)

by RFK on Jun 12, 2009 10:30 AM EDT up reply actions   0 recs

Rank is tricky...

…as MySQL is missing some functionality in that regard. Here’s the workaround I use.

by cwyers on Jun 12, 2009 1:38 PM EDT up reply actions   0 recs

Whoops.

That works, but this is the one I use.

by cwyers on Jun 12, 2009 1:39 PM EDT up reply actions   0 recs

I'm having trouble

SELECT yearID, teamIDwinner, teamIDloser FROM BDB.SeriesPost WHERE `yearID` >= 1980

tells me that the Royals and the Yankees made the playoffs in 1980, and

SELECT yearID, lgID, teamID, HR,
@num := @num + 1 rank from BDB.Teams,
(SELECT @num := 0)
d
WHERE `yearID` = 1980
AND lgID = ‘AL’
ORDER BY yearID ASC, lgID ASC, HR DESC

tells me that the Yankees ranked #2 and the Royals ranked #9 in homers that season in the AL. But I can’t create a view of the second statement (because it includes a variable). Since I can’t, I’m lost from there, unless I do the rest of the years manually.

by RFK on Jun 15, 2009 10:42 PM EDT up reply actions   0 recs

If anybody is still paying attention to this,

I found another way to rank, so that a view can be created.

CREATE VIEW view_name AS
SELECT yearID,lgID, teamID, HR, (
SELECT count( * ) + 1
FROM Teams b
WHERE a.HR < b.HR
AND yearID = 1980
AND lgID = ‘AL’
) AS rank
FROM Teams a
WHERE `yearID` = 1980
AND lgID = ‘AL’
ORDER BY rank ASC

This again tells me that the Yankees ranked #2 and the Royals ranked #9 in homers that season in the AL. Then, if I run this query:

SELECT a.yearID, a.teamIDwinner, b.rank as rankw, a.teamIDloser, b.rank as rankl,a.round
FROM SeriesPost a, view_name b
WHERE a.yearID = 1980
AND a.teamIDwinner = b.teamID

…I get an answer, but it’s only half-right. Also, it’s just for 1980, not every season since. So I guess I have some more work to do.

by RFK on Jun 17, 2009 2:15 PM EDT up reply actions   0 recs

Still paying attention!

Not a lot of time to play with it, though.

Beyond the Boxscore // Calling BJ Upton lazy is lazy.

by Sky Kalkman on Jun 18, 2009 2:04 PM EDT up reply actions   0 recs

Comments For This Post Are Closed


User Tools

We use numbers and stuff.
Community Guidelines
Why be a member?
Start posting on Beyond the Box Score »

Join SB Nation and dive into communities focused on all your favorite teams.

FanPosts

Community blog posts and discussion.

Recent FanPosts

Leopold_butter_scotch_southpark_small
Using the TVC
Small
Determining Batted Ball Rates using Pitch Type and Location
Small
a new xBABIP calculator
Img587561916661595
Top 15 high school MLB draft prospects
Small
PZR-based Win Values 2001-2006
Small
The "30 parks on a budget" challenge
Sunflower_small
World Series Simulation, Game #6
Small
JT20 Dynasty League
E52205a2_small
New Look
Sth70021_small
Exploring Hit f/x, Albeit Badly

+ New FanPost All FanPosts >

FanShots

Quick hits of video, photos, quotes, chats, links and lists that you find around the web.

Recent FanShots

Primer on BaseRuns
Cool Baseball Infographics
ESPN's Jerry Crasnick on defensive metrics
I’m also a follower, since Brian Bannister’s on our team, of sabermetric st...
Top Ten Baseball-Reference.com's Sponsorships
Primer on Linear Weights
JC Bradbury on "Hot Stove Myths"
Everyone Should Learn to Throw a Cutter
Criminals of WAR
Ten statisticians you should know about

+ New FanShot All FanShots >

BtB on Twitter

Main Feed: @BtBScore

Tommy B: @tommy_bennett
Sky: @BtB_Sky
Dan: @dturkenk
Harry: @harrypav
Jinaz: @jinazreds
Jack: @jh_moore
Erik: @Erik_Manning
Tommy R: @trancel
Justin: @justinbopp

Subscribe to BtB via Email

Enter your email address:

Delivered by FeedBurner

BtB Goes Social


Managers

Nando_small R.J. Anderson

Limes_125_small Sky Kalkman

E52205a2_small Tommy Bennett

Editors

Face_small Harry Pavlidis

Rawlings_baseball_bigger_small Dan Turkenkopf

770insig_small Jeff Zimmerman (TucsonRoyal)

Aviles_small Justin Bopp

Authors

Banny_small erik

Raysring1_small Tommy Rancel

Jinaz-reds-avatar_small JinAZ

Jmlogo_small Jack Moore

1753738656_110919ebe9_o_small vivaelpujols

1_small Graham

Baseball_small Mike Rogers

Redcap_small SFiercex4

Small Patrick Clark

Walter_album_small Walter Fulbright