As sabermetric-minded individuals here, most of us like to go back and look how awards could have played out knowing what we now know. Heck, you may have even seen a post or two regarding this argument for this past season's AL MVP. Using WAR, there have been plenty of MVPs (Juan Gonzalez in '96, Andre Dawson in '87, etc.) and Cy Young winners (Bob Welch in '90, any reliever) who did not measure up to that standard.
A recent example that may not go to the forefront of your brain is the 1993 NL CYA race (the AL vote was worse, but this is more interesting). While Greg Maddux was the runaway WAR leader in '92, '94, and '95, he lagged behind this particular season at 5.5 WAR. Looking at ERA/FIP, Maddux's 2.36/2.85 line looks quite good, with Jose Rijo (2.48/2.93) as his only competition. However, 14-game winner Rijo finished fifth behind 20-game winners Bill Swift, Tom Glavine, and John Burkett.
Looking deeper, there still aren't many differences between Maddux and Rijo's 1993 seasons. Using B-R's new pitching WAR system, each pitcher faced the same quality opponents, right around 4.55 R/9. Each pitched in parks that averaged out to a park factor of about 99, and Maddux only threw 10 more innings than Rijo that season (267-257). One may wonder how Rijo finished 3.4 WAR ahead of Maddux that year.
The last piece of the puzzle is defensive support, and these two teams couldn't be more different. Maddux's Braves had a team TotalZone of +82, with guys like Otis Nixon, Mark Lemke, Rafael Belliard, and others making great plays behind him. Only Jeff Blauser and Greg Olson had negative ratings on the team. This led to a half-run boost in support, lowering Maddux's expected average level to 4.14 R/9.
Meanwhile, Rijo's Reds had a team -52 mark and a cast to support that number. While there were no double-digit negatives, guys like Juan Samuel and Kevin Mitchell (whose defensive claim to fame is over-running a route and catching a deep flyball with his bare hand) were some of the many who were close to -10 figures. Only Joe Oliver and Reggie Sanders provided above-average defense on the team. This led to a 0.3-run dip in support, raising Rijo's expected average level to 4.97 R/9.
In the 19 years since '93, there have been only seven pitcher seasons that have eclipsed Rijo's 8.9 rWAR, and every one of them was a CYA winner. We all know that his 14-9 record is what held him back from even coming close to challenging Maddux, and it's possible that the defensive disparity may not have been that large. We also know that Rijo was at least as good as Maddux that season, and the voting point margin should not have been 119-8 in favor of Maddux. It's clear that the BBWAA is not yet completely on board with WAR and all the advanced metrics, but I would guess that in 20 years, we will be seeing these types of voting orders at least resemble WAR leaderboards.