Tuesday, April 7, 2009

Thus Spake PECOTA

With our armchair analysis in the books, it's time to wheel out the big analytical guns. We've run the rosters of each team through the PECOTA projection system put out by Baseball Prospectus and come up with something resembling a objective crack at a predicted order of finish. But before we get to the results, let's lay out our methodology and some problems with the system up front. Do us a favor and read all this before you flip out about the predictions, OK?

We took each team's best-projected hitter at each position, even if a team plans to start someone else, in order to take the human element out of the process. No effort was made to account for the production of replacement or bench hitters, because it all just got too complicated, and the whole point here was to use something other than our subjective opinion to predict the league.

For pitchers, we did our best to projected the starting 9 plus the best bench pitcher, to better reflect how teams actually use their staffs in this league. We should also say that these standings are incredibly close in places; one change in a closer job can (and has) moved the predictions around pretty good.

Now, here are the three major problems with all this.

Problem: The system chokes on injury and risk.

Take A-Rod. PECOTA sensibly takes a bite out of his projected production to account for his existing injury. But he still has the best projected full-season line of any 3B on his owner's roster. So A-Rod's line goes into the projection. However, by doing so the projection entirely omits the production the team will get from whatever random guy they plug in until A-Rod is healthy. This means that team's counting stats will be understated, and its rate stats might be overstated. This hurts more for stud offensive players, because each has 4 counting stats and 1 rate stat, than it does for pitchers, who have 2 real counting stats and 2 rate stats.

Similarly, PECOTA takes a bite out of the projection of players who are big injury risks going forward. This especially hurts pitchers, as they tend to have larger risk profiles. So if a guy has a 60% chance of blowing out his arm, PECOTA docks about 60% of his counting stats. Again, while that approach makes some sense, it doesn't factor in replacement performance.

Along those same lines, the system probably underrates stars-and-scrubs teams by assuming that they'll roll with the same scrubs all year rather than sort through the waiver wire for a guy having a fluke year.

The point is that the system underrates the chances of high-risk, high-reward teams. So bear that in mind when you see the projected standings.

Problem: The system does not factor in owner performance.

Good owners can add value in two ways. They can manage well, taking the guys on their roster and mixing and matching them to best effect. Or they can GM well, adding to their roster through pickups and trades. Since we have no idea what will happen on either of those fronts, the predictions just take the rosters as given. The projections show what the likely outcome would be if we all set our lineups on Opening Day and then came back in October to see who won. If you don't like your prediction, well, go manage your way up.

Problem: PECOTA likes what PECOTA likes.

PECOTA has a good reputation. However, it isn't perfect. It has its darlings (like Matt Wieters and A.J. Burnett) and bete noires (like Ichiro, who PECOTA seems to think will be crushed by a falling piano at some point this year, and David Ortiz, who it thinks is cooked). Projections are only as good as the inputs used to generate them; if you think PECOTA is dumb, you'll think the projections are dumb.

With all that in mind, we present the PECOTA-projected Wankdorf standings for the upcoming season:

Pitching Hitting Total
It's Enrico Palazzo 47 41.5 88.5
Wu Tang Financial 46 42 88
Unenviable Position 44.5 41 85.5
Evil League of Evil 35 38 73
Mission Accomplished 26.5 46.5 73
Recalcitrant Cobblers 33.5 34 67.5
The Loose Bowels 34 33 67
wormcheese mousebird 23 32.5 55.5
The Spam Avengers 29.5 24 53.5
Elbow Your Funicular? 25 25.5 50.5
Le Dupont Torkies 32.5 16 48.5
Aroids Anonymous 14.5 15 29.5

If you don't like them, go to Russia. Or just call Baseball Prospectus and yell at them. IEP, pressure's on you.

Edit--by popular request, here are the raw totals PECOTA came up with.

Chad 84 86 1130 3.89 1.29
Ang 86 13 1069 4.12 1.37
Sahil 79 80 1004 4.13 1.32
Will 74 109 919 4.08 1.35
Jake 77 70 1012 3.73 1.31
Alex 71 87 1014 3.95 1.31
Jon 76 90 1073 3.76 1.27
Corey 66 91 912 4.20 1.37
Scot 80 86 1038 3.91 1.31
Elders 66 100 1047 3.96 1.30
Andrew 91 41 1160 3.84 1.28
Tucker 66 143 932 3.85 1.31
Chad 751 227 749 104 0.358
Ang 790 196 750 106 0.366
Sahil 756 178 715 138 0.353
Will 712 196 749 69 0.352
Jake 754 187 739 98 0.358
Alex 741 153 664 130 0.353
Jon 765 188 678 125 0.365
Corey 719 150 648 107 0.350
Scot 757 193 730 88 0.360
Elders 735 190 736 82 0.361
Andrew 773 187 757 82 0.361
Tucker 682 184 708 78 0.353


El Angelo said...

As my wife can tell you in frightening detail, an IEP is a plan developed by school districts for children with learning issues. I'm not implying anything here, but Andrew may want to change his name to Nordberg next season if we're going to abbreviate.

Corey said...

PECOTA scores work like golf scores, right?

Andrew said...

Good catch on the name. As someone who works with at-risk kids who have special needs, there's quite an irony in my team's acronym.