BAD STATS

James Joyner | Tuesday, November 18, 2003

Chris Lawrence explains why the statistical methods employed by the BCS college football selection process are flawed. It’s Social Science 101, really:

A fundamental problem is that truly ordinal data is treated as metric in the formula. Your age, height, and weight are metric data: differences in age have real meaning. If I’m 27 and my cousin is 3, the difference in our age–24 years–is a meaningful quantity. By contrast, poll rankings aren’t metric. LSU is #3 in the AP poll, and Ole Miss is #15. 15-3=12. Twelve doesn’t tell us much of anything about the difference between LSU and Ole Miss; it just tells us that there’s a difference. Missouri is #27. 27-15=12. Treating this difference as metric makes an invalid assertion: that the difference in quality between LSU and Ole Miss is the same as the difference between Ole Miss and Missouri.

This problem repeats itself throughout the BCS formula. Means of rankings in polls and computer rankings are taken. These means are added together. The strength of schedule component–which is a key component of many computer rankings–starts as metric data, then is converted to a ranking and arbitrarily scaled. . . then added to the means. Losses–which are metric–are then subtracted. Finally, an ad hoc adjustment is made for so-called “quality wins”–an adjustment one would hope that is incorporated in the polls and computer rankings anyway. Then the rankings are reported with these bizarre totals attached, apparently because totals look cool (I guess they got the idea from the AP and ESPN polls, who report the sorta-kinda metric Borda count in addition to the rankings).

About the Author: James Joyner is the publisher of Outside the Beltway and the managing editor of the Atlantic Council. He's a former Army officer, Desert Storm vet, and college professor with a PhD in political science from The University of Alabama. He lives just outside the Beltway in Alexandria, Virginia.

Follow James on FriendFeed | Twitter | Digg

• None Found

• Caption Contest Winners
• The Political Economy of the U.S. Mortgage Default Crisis
• The Democrats and the Shock Doctrine
• Obama and Europe
• Supreme Court to Decide on Obama Citizenship
• International Implications of the Drop in Oil Prices (Updated)
• Obama’s Bipartisan Cabinet
• More Votes Found - Franken Hurt
• Caption Contest
• OTB Latenight - Paul Simon

Tags | Sports

| | Permalink | Send TrackBack

• Backcountry Conservative linked with BCS Critique

Comments are Closed