Back in ancient times when I worked at esteemed weekly newsmagazine U.S. News & World Report, I always loathed the annual college rankings report.
Like all cash cows, however, the college guide was a sacred cow, so I just shut up about its obvious statistical absurdities and inherent mendacity. As a lesson in the evils of our times, it is perhaps inevitable that the college guide is now the only thing left of U.S. News.
A story in today's New York Times reports that Claremont McKenna college has now been caught red handed submitting phony data to the college guide to boost its rankings.
But the real scandal, as usual, is not the occasional flagrant instance of outright dishonesty but the routine corruption that is shot through the whole thing.
In fact, there's much to be said for just making up numbers as Claremont McKenna did: because far worse are the things almost all colleges actually do purely to produce favorable statistics that will bump them up in the U.S. News rankings. Lying is bad; living the lie is much worse.
The contortions colleges routinely go through to game the system as decreed by U.S. News almost defy belief.
To increase selectivity (one of the statistics that go into U.S. News's secret mumbo-jumbo formula to produce an overall ranking), many colleges deliberately encourage applications from students who don't have a prayer of getting in. To increase average SAT scores, colleges offer huge scholarships to un-needy but high scoring applicants to lure them to attend their institution. (The Times story mentioned that other colleges have been offering payments to admitted students to retake the test to increase the school average.)
One of my favorite bits of absurdity was what a friend on the faculty at Case Law School told me they were doing a few years ago: because one of the U.S. News data points was the percentage of graduates employed in their field, the law school simply hired any recent graduate who could not get a job at a law firm and put him to work in the library.
Their other tactic was pure genius: the law school hired as adjunct professors local alumni who already had lucrative careers (thereby increasing the faculty-student ratio, a key U.S. News statistic used in determining ranking), paid them exorbitant salaries they did not need (thereby increasing average faculty salary, another U.S. News data point), then made it understood that since they did not really need all that money they were expected to donate it all back to the school (thereby increasing the alumni giving rate, another U.S. News data point): three birds with one stone!
(I gather the new Case law dean has put an end to these shenanigans.)
As someone who knew a little math, what really drove me bonkers about the college guide was:
(a) the logical absurdity of adding together completely unrelated statistics to produce a single measure of merit—the key point being that you can produce an astonishing range of different results depending on the relative weight each component factor is assigned. And there is simply no logical, a priori basis for establishing such a weighting objectively. Do SAT scores count 30% of the total score? 32.2%? 18.78234%? (How about zero?) It's the classic apples + oranges – bananas/kumquats = fruit salad approach to statistics, and is completely meaningless.
(b) the fact that the entire exercise was designed to emphasize noise over signal: tiny, random fluctuations from year to year would result in regular changes in the final rankings. Even within its own absurd methodology no one ever dared broach the question of the actual statistical significance of the differences between the "No. 1" school and say the No. 5 school. In fact, there was pretty clearly none. It is of course ridiculous to think that when Harvard, Stanford, Yale, whoever changed places from one year to the next in the final rankings this reflected any actual sudden change in the underlying quality of the schools. But the only way to keep selling the damned guide each year was to make sure things kept changing from year to year.
There's a special place in hell for the perpetrators of this, where I hope the gods of mathematics and reason are devising some exquisite tortures for them—perhaps in the form of endlessly reading Introduction to Statistics and doing the same problem sets over and over through eternity . . .
