Saberizing a Mac #3: Adding Win Shares & Other Data to the BDB
Based on the lack of comments in the previous post, I'm going to guess people had no trouble installing the Baseball Databank database. If you did, feel free to ask questions in the comments of that post. To follow the process I personally used to get it done, read this comment. All Saberizing a Mac posts can be found here.
This next challenge was suggested by Jeff (TucsonRoyal). The Baseball Databank is awesome, but what if there's additional information from other files that we want to add to the database? How do we do that? Well, let's figure it out, starting with Win Shares data provided by Dave Studeman. There are two .csv files in there, both of which include BDB ids, allowing us a way to hook into the data that's already in the BDB.
Let's tackle the total WS file first, then the WS above average. We have some choices to make. Do we want this as a separate table in the database, or do we want to add fields to an existing table? I'm not sure both are even an option with this WS data, but it's an interesting question to discuss.
As a challenge, let's see if everyone can get the data imported and then write a SQL query to find the ten oldest teams since 1980, where team age is defined as the weighted average of each player's age and their total Win Shares. As an example, to find the 2008 World Champion Phillies' weighted age, you'd multiply Chase Utley's 2008 age by his WS, add in Ryan Howard's 2008 age times his WS, add in the same thing for everybody else, then divide by the total team Win Shares. (I've chosent his definition because it requires us to use WS data, and weights team age by the players who are actually providing value to the team. You could also weight by PAs or IPs to find the team age based on playing time instead of performance.)
Calculating age can be a bit tricky, and I'll refer people once again to Colin's two primers on SQL. Colin also has some additional sample code available here. He's offered to help folks with their own queries, which can be something else that happens in the comments of this post. If you challenge yourself to come up with a query to answer an interesting question, but are struggling, ask for help. If you succeed, share the code.
Once you've got the Win Shares data incorporated, try adding some of the data Tom Tango's made available here, like primary defensive position and wOBA. And if you find any other data we can add, please share it.
Get your celebratory age-appropriate beverages chilled... set... go!
(And yes, this is something PC-users can play along with, too, although your results won't be as aestheticlly pleasing and you can't share in our yuppy attitudes.)
0 recs |
27 comments
|
Comments
My answer
Do we want this as a separate table in the database, or do we want to add fields to an existing table?
Definitely separate tables. Otherwise you have major troubles when either piece of data is updated.
If you want to get the data in one easily accessible “table” try creating a view. It doesn’t have any performance improvement over a straight SQL statement, but it saves you having to write the SQL each time.
by Dan Turkenkopf on Jun 10, 2009 9:20 PM EDT reply actions 0 recs
Sounds like good advice.
But what do you know about this sort of thing?
How is a view different from a query?
Beyond the Boxscore // Calling BJ Upton lazy is lazy.
by Sky Kalkman on Jun 10, 2009 9:32 PM EDT up reply actions 0 recs
A view is simply a query that the database remembers.
And without getting too far into the normal forms and whatnot – it’s generally more logical to introduce a new table for these sort of things.
by cwyers on Jun 11, 2009 2:50 AM EDT up reply actions 0 recs
We could get into a normalization discussion though and really confuse Sky
What’s your feeling on the tradeoffs between normalization and performance? :)
by Dan Turkenkopf on Jun 11, 2009 7:45 AM EDT up reply actions 0 recs
You both are definitely not normal.
Beyond the Boxscore // Calling BJ Upton lazy is lazy.
by Sky Kalkman on Jun 11, 2009 8:45 AM EDT up reply actions 0 recs
When it comes down to it, I prefer performance.
The Retrosheet schema I use is not very normalized at all, and it doesn’t bother me any. (I don’t even think it follows the facts table setup of a data warehouse, but I’m not an expert here.)
by cwyers on Jun 11, 2009 11:24 AM EDT up reply actions 0 recs
Me neither, really
I’ve never bothered to normalize my retrosheet tables either.. they’re basically all 100 or so fields laid out in a single table. It suits my purposes so I haven’t spent a lot of time to make it better.
by Dan Turkenkopf on Jun 11, 2009 6:01 PM EDT up reply actions 0 recs
This first step has worked for me so far
In the Pro Sequel BDB database I created a table from the Query view:
CREATE TABLE `BDB`.`2008_Historical_WinShares` (
`playerID` VARCHAR ( 10 ) NOT NULL ,
`yearID` VARCHAR ( 6 ) NOT NULL ,
`Age` VARCHAR ( 6 ) NOT NULL ,
`Multiteam?` VARCHAR ( 6 ) NOT NULL ,
`Multi-Same Team?` VARCHAR ( 6 ) NOT NULL ,
`stint` VARCHAR ( 5 ) NOT NULL ,
`teamID` VARCHAR ( 6 ) NOT NULL ,
`lgID` VARCHAR ( 4 ) NOT NULL ,
`Pos` VARCHAR ( 5 ) NOT NULL ,
`WinShares` VARCHAR ( 10 ) NOT NULL ,
`BatWS` VARCHAR ( 10 ) NOT NULL ,
`FieldWS` VARCHAR ( 10 ) NOT NULL ,
`PitchWS` VARCHAR ( 10 ) NOT NULL ,
`TotWS` VARCHAR ( 10 ) NOT NULL
) ENGINE = MYISAM
…then I imported the 2008_Historical_WinShares.csv file (File>Import…). I just went with the default settings.
(At least the data is there; I don’t claim to know what I’m doing. Your mileage definitely may vary.)
by RFK on Jun 11, 2009 2:01 AM EDT reply actions 0 recs
Sweet.
Can you run a query using the table?
Beyond the Boxscore // Calling BJ Upton lazy is lazy.
by Sky Kalkman on Jun 11, 2009 8:46 AM EDT up reply actions 0 recs
Yes, here are the top 50 total Win Share years for the SF Giants, for example:
SELECT playerID
, yearID
, TotWS
, teamID
FROM 2008_Historical_WinShares
GROUP BY playerID
HAVING teamID = “SFN”
ORDER BY TotWS Desc
LIMIT 50;
What the heck, it’s something!
by RFK on Jun 11, 2009 11:47 AM EDT up reply actions 0 recs
Why do you group by playerID? To combine stints with the same team in the same year?
Just a safety net type thing? Other?
Beyond the Boxscore // Calling BJ Upton lazy is lazy.
by Sky Kalkman on Jun 11, 2009 2:31 PM EDT up reply actions 0 recs
Players who play multiple positions in a year, I'd wager.
by cwyers on Jun 11, 2009 4:20 PM EDT up reply actions 0 recs
If you want to know the truth
I created the table from the MAMP/phpMyAdmin/Operations/Create new table on database tool. It winds up giving you the code. Copy and paste, baby!
by RFK on Jun 11, 2009 5:32 PM EDT up reply actions 0 recs
And
I just aped Colin’s second code from the historic November 16, 2008 “Building a sabermetrician’s workbench, part II”. Colin is my Zeus!
by RFK on Jun 11, 2009 5:38 PM EDT up reply actions 0 recs
What's the ENGINE = MYISAM part do?
Beyond the Boxscore // Calling BJ Upton lazy is lazy.
by Sky Kalkman on Jun 11, 2009 2:20 PM EDT up reply actions 0 recs
MySQL uses several different storage engines.
That just tells it which one to use. MyISAM is probably the best engine choice for the vast majority of the things that you’d want to use MySQL for (and by you I mean a sabermetrician).
by cwyers on Jun 11, 2009 4:19 PM EDT up reply actions 0 recs
So when creating a table, you need that at the end?
Beyond the Boxscore // Calling BJ Upton lazy is lazy.
by Sky Kalkman on Jun 11, 2009 4:30 PM EDT up reply actions 0 recs
Not really.
Although I can’t tell you what your install will default to (mine will make a table MyISAM by default). It’s not a bad idea to do so, however.
by cwyers on Jun 12, 2009 12:36 AM EDT up reply actions 0 recs
Challenge I'm working on...
For all teams to make the playoffs since, oh, 1980, where did they rank within their league in total homeruns?
Beyond the Boxscore // Calling BJ Upton lazy is lazy.
by Sky Kalkman on Jun 11, 2009 10:14 AM EDT reply actions 0 recs
FWIW I'm working on this, too
But it might be a few days before I have time to get it together. (At least it’s easier than the Ten Oldest Teams Since 1980 Challenge.)
by RFK on Jun 12, 2009 10:30 AM EDT up reply actions 0 recs
Rank is tricky...
…as MySQL is missing some functionality in that regard. Here’s the workaround I use.
by cwyers on Jun 12, 2009 1:38 PM EDT up reply actions 0 recs
Whoops.
That works, but this is the one I use.
by cwyers on Jun 12, 2009 1:39 PM EDT up reply actions 0 recs
I'm having trouble
SELECT yearID, teamIDwinner, teamIDloser FROM BDB.SeriesPost WHERE `yearID` >= 1980
tells me that the Royals and the Yankees made the playoffs in 1980, and
SELECT yearID, lgID, teamID, HR,
@num := @num + 1 rank from BDB.Teams,
(SELECT @num := 0)
d
WHERE `yearID` = 1980
AND lgID = ‘AL’
ORDER BY yearID ASC, lgID ASC, HR DESC
tells me that the Yankees ranked #2 and the Royals ranked #9 in homers that season in the AL. But I can’t create a view of the second statement (because it includes a variable). Since I can’t, I’m lost from there, unless I do the rest of the years manually.
by RFK on Jun 15, 2009 10:42 PM EDT up reply actions 0 recs
If anybody is still paying attention to this,
I found another way to rank, so that a view can be created.
CREATE VIEW view_name AS
SELECT yearID,lgID, teamID, HR, (
SELECT count( * ) + 1
FROM Teams b
WHERE a.HR < b.HR
AND yearID = 1980
AND lgID = ‘AL’
) AS rank
FROM Teams a
WHERE `yearID` = 1980
AND lgID = ‘AL’
ORDER BY rank ASC
This again tells me that the Yankees ranked #2 and the Royals ranked #9 in homers that season in the AL. Then, if I run this query:
SELECT a.yearID, a.teamIDwinner, b.rank as rankw, a.teamIDloser, b.rank as rankl,a.round
FROM SeriesPost a, view_name b
WHERE a.yearID = 1980
AND a.teamIDwinner = b.teamID
…I get an answer, but it’s only half-right. Also, it’s just for 1980, not every season since. So I guess I have some more work to do.
by RFK on Jun 17, 2009 2:15 PM EDT up reply actions 0 recs
Still paying attention!
Not a lot of time to play with it, though.
Beyond the Boxscore // Calling BJ Upton lazy is lazy.
by Sky Kalkman on Jun 18, 2009 2:04 PM EDT up reply actions 0 recs

by 












BtB on Facebook
















