The Decline of the Baseball Prospectus

I've been a fan of the Baseball Prospectus for several years, but this past offseason has sorely tested my BP fandom - perhaps to the breaking point.

Now, the Prospectus is really two things in one: It's a baseball "annual" book, full of articles on the game, the teams and the players (for instance, here's the 2003 edition). And it's a Web site, with daily and weekly columns by the authors of the book. But the two are closely related, and my disappointment is reflected in both the Web site, and the 2003 edition of the book.

The Prospectus emerged (I understand) from a group of baseball fans who were developing new theories about baseball player evaluation and how to assemble a winning team. Disciples (figuratively speaking) of Bill James, they took a statistical (or, to use the notional jargon, "sabermetric") approach to baseball analysis, and after a while found that there seemed to be a market for their thoughts, so they started publishing the book and put up the Web site.

Their work has interested me essentially because they develop and present systems and tools to use for analysis and that they examine the usefulness of these systems and tools as applied to actual players, teams and seasons. Reporting of facts (raw stats, transactions, etc.) is a secondary consideration, largely because there are already so many good sources for the facts (ESPN, for instance).

My disillusionment with the site has been caused by the near-simultaneous development of two things: First, they decided to start charging for most of the Web site content ($39.95 per year), and second, the quality of the content has gone south in a big way during this off-season. By which I mean, they're no longer focusing on presenting an examining systems and tools for player and team evaluation. (Whether the quality has truly "gone south" is obviously a matter of opinion, but this being my journal, these are my opinions.)

So what's happened to the content? Well, it seems like the Prospectus emphasis is now on the following areas:

1. Injury Reporting: New Prospectus columnist Will Carroll is writing a column called "Under the Knife", and wrote a series of injury reports during the offseason (for instance, this one about the Boston Red Sox). What I've read of these has struck me as being in essence factual reporting, with a little bit of intuitive analysis (such as, so-and-so pitcher's mechanics are lousy, so he's at risk of injuring himself unless he somehow fixes them).

   It took me only a few of these injury reports to decide that they were blindingly tedious, and not really more insightful than what I can glean by just reading team injury reports available elsewhere. A Prospectus injury column might be interesting if it did some quantitative analysis ("Here are all the pitchers we could find since 1980 who tore their labrums, and here's how they performed after returning from surgery/rehab"), but Carroll's columns lacked that sort of rubber-hits-the-road analysis.

2. PECOTA Player Evaluations: The brainchild of new BP columnist Nate Silver, PECOTA uses a method similar in spirit to Bill James' Similarity Scores (described in his classic book Whatever Happened to the Hall of Fame?), but uses statistics with more sabermetric value. PECOTA compares active players to similar players of bygone years to estimate how the active players are likely to develop in the future. It's certainly a worthwhile idea, and the generalities of the method are described in the 2003 Prospectus book. But ultimately I was frustrated by PECOTA for two reasons:
   1. Silver doesn't describe the system in detail. The book doesn't explain which measures are used, why, and how much weight is given to each measure. Without this information, one can't independently evaluate the system, rendering its value to me as questionable at best.

   2. The single-stat approach to measuring player performance doesn't ring true to me, because player performance - especially looking forward - is multi-dimensional. Moreover, PECOTA results are presented as an estimate the chance that a player will substantially improve ("Breakout"), Improve, or substantially decline ("Collapse"). But these numbers depend (by Silver's admission) to some degree on whether a player is performing toward an extreme of the performance spectrum, so the numbers don't really mean the same thing for all players. In other words, the numbers don't have an intuitive meaning, which is a problem I also have with Equivalent Average (EQA). It's not a valueless system, but it just doesn't feel right to me.

   PECOTA has been presented by the Prospectus both abruptly (it's used throughout the book and on the Web site, to some extent replacing their previous use of EQA, and seems to have surfaced only earlier this year) and authoritatively, as if it's the new standard, without, I feel, sufficient justification to treat it as such. Because of this, I feel its use this year significantly diminishes the quantitative analysis value of the Prospectus.

   (And, as a cheap shot, I think PECOTA is a really stupid name. Besides just sounding silly, it ostensibly stands for "Pitcher Empirical Comparison and Optimization Test Algorithm", which is a nearly-content-free name that one strongly suspects was concocted just to fit a neat acronym. Or would have been, had the acronym been neat. But I digress.)

3. The Business of Baseball: The Prospectus has been devoting an increasing amount of time over the last year-plus to examining the financial and political elements of baseball. While there's some information of interest here - particularly Doug Pappas' articles analyzing the finances of the 30 MLB teams - it again strikes me as essentially basic factual reporting and qualitative-not-quantitative analysis. While I've never entirely agreed with the Prospectus attitude that performance evaluation vastly outweighs the game's revenue imbalances, that stance at least both tied in to player performance as well as being something which could be objectively evaluated (e.g., Oakland's continued excellence). Analyzing the Collective Bargaining Agreement doesn't hold my interest.

   Overall, I'd be interested in reading 4 or 5 articles like Pappas' per year on their site. I think they're interested in doing a lot more than that.

• Detroit Tigers: Nearly the entire article is spent re-hashing the last five-to-ten years of the franchise's sorry history, rather than focusing on the details of what happened in 2002 and where they're going in 2003. All of which is old, old news. Yeah, I realize every Prospectus is going to be someone's first, and I feel for them having to write an entry on Detroit every year, but we readers would have been better served by one paragraph of, "Not much has changed since last year. Dave Dombrowski might yet pull them out of the mud, but it'll be a few years. Check back with us then," than with recycling old news in new words.

• Chicago Cubs: The first third of the article rips ex-manager Don Baylor a new one, although in lamentably general terms. It reaches what may be the book's nadir in its paragraph about Bruce Kimm, who "yammered about guts and determinism and manliness and professionalism and letting the players play and the pitchers pitch and the sun rise and the moon set and the girl from Ipanema". And telling us this has what, exactly, to do with winning or losing baseball games? Does being a chatterbox mean you're not a good manager?

• Kansas City Royals: Another raking-over-the-coals of GM Allard Baird, this time with the twist that Baird professes to be striving to focus on skills and performance rather than tools and reputations, except that his actions bely his words. Is saying the right things and doing the wrong things any better than him saying and doing the wrong things? Maybe. But it'll be news when he starts doing the right things. Not as bad as the Detroit article, but far from insightful.

• Pittsburgh Pirates: The National League mirror image of the Detroit entry, dwelling yet again on the failures of the Cam Bonifay era. This entry was even lamer than the Detroit entry because the Pirates have made some positive steps forward in the least year - like fleecing the White Sox for two solid starting pitchers - and the article makes hardly any mention of this.

To be fair, my glum attitude towards other elements of the Prospectus may have colored my outlook towards the book's team articles this year. But still.

So what's going on here? A year ago I was pretty happy with the Prospectus content. Well, not being an insider I obviously don't know for certain, but here's my conjecture:

First, I think they probably reached a point where maintaining their Web site was costing them a substantial amount of money - more than they were willing to spend for a mere hobby out of their pockets - and they realized they could probably charge for the content and gross enough to cover their costs. I don't think it's a matter of greed, but one of pragmatism. However, by going to a pay-for-content format, it was a small step to the next point, which has already been under way for a little while.

That point is that I think many of the Prospectus authors aspire to become baseball professionals. One of their staff went to the Blue Jays organization a year-plus back. The presentation of PECOTA suggests to me that Silver (and company?) are hoping to leverage it into some sort of gig down the road, as well. This isn't really a bad thing, but it's different from the Prospectus as we've known it in the past, which seemed to emphasize disseminating information and theories, which anyone could test out and criticize given the time and inclination.

And I think that's where they've basically lost me. They're not about the cool theories and the analysis of the theories anymore - or at least, they don't seem to be from where I stand. I think they strongly believe that they're just building on what they've done before and taking it to a new level, but I think they've shed some essential components of what's made the Prospectus enjoyable and valuable in the past. At least to me as a sabermetrically-inclined fan.

A few years ago, their "Transaction Analysis" feature was my least favorite Prospectus column, because it was mostly factual reporting with a bunch of snarky comments, and the occasional dollup of analysis. Amusing, but little more. Prior to launching the pay system, it had become one of the more enjoyable columns for me. But it hadn't really changed much. The site around it had changed.

Sadly, I've decided not to subscribe to Baseball Prospectus Premium. I'll read the free articles as they come along (though articles like "Deep Fried Twinkies" make me despair that any of these will be worth the effort), but the confluence the shift in emphasis with the increase in price pretty convincingly persuaded me to look elsewhere for my sabermetric education and entertainment.

I wish the Prospectus gang well on their future endeavors (I'll probably keep up with their annual books so long as they're worth my while), but I think the roller-coaster ride ends here for me. Time to bail out and go buy a corn dog.