Wednesday, September 23, 2009

A Silly Thought Experiment and Concomitantly Modest Proposal

As we've noted elsewhere, the "official" GRBG league predictions as generated by Baseball Prospectus's PECOTA data were less than razor sharp crossbow accurate. But, as noted in the prediction post, there were three problems with our use of PECOTA. One of those problems--PECOTA's inability to factor in the performance of owners--clearly contributed to the predictions' shortcomings.

We here at the GRBG wondered whether there was a way to separate out managerial performance and see how PECOTA would have done based on the rosters as drafted. As we wondered this, our eyes wandered to our Yahoo! notification stream, which informed us for the 200th consecutive day that Troy Percival was on the injured list, yet in our starting lineup. And then we remembered:

The Death Ship.

Because of various wacky mishaps with the draft system (and a certain owner's apparent inability to read a watch), on draft day the league created an alternate reality. That reality contained slightly warped copies of all our teams, with rosters as they stood after the completion of most of the draft. The bizarro league was then sealed up and left to play itself out without material interference from us. It became, essentially, a control group. So did PECOTA do any better predicting the bizarro world? Here are the predictions vs the actual standings:

PECOTABizarro StandingsReality
It's Enrico PalazzoWu Tang FinancialLe Dupont Torkies
Wu Tang FinancialAroids AnonymousAroids Anonymous
Unenviable PositionRecalcitrant CobblerMission Accomplished
Evil League of Evilfakeleague (Ang)Recalcitrant Cobbler
Mission AccomplishedLe Dupont TorkiesFlaccid Funiculi
Recalcitrant CobblerThe Loose BowelsWu Tang Financial
The Loose BowelsThe Spam Avengerswormcheese etc.
wormcheese etc.wormcheese etc.Unenviable Position
The Spam AvengersJake!Wrong League (Scot)Evil League of Evil
Flaccid FuniculiUnenviable PositionThe Spam Avengers
Le Dupont TorkiesDraft? (Will)It’s Enrico Palazzo
Aroids AnonymousIt’s Enrico PalazzoThe Loose Bowels


Well, PECOTA did seem to come closer to the control group than it did to manager-affected reality, so maybe there's some value there after all. More interestingly, the gaps between the projections, control group, and reality give some interesting insights into who has done the best job as an owner this year (hint: not me). We'll cover that in a later post. But I'll tell you what: we should definitely make the Death Ship league an annual tradition so we can track this stuff.

No comments: