|
|
|
|
  t
the John Jay College Theater in midtown Manhattan, this past spring’s
attraction was Mnemonic. It is a play built on memory, “one of
the last great mysteries,” according to a snippet of dialogue.
Even before the stage action unfolds, audience members are challenged
to remember where they were two hours ago, two weeks ago, a month
ago, ten years ago.
As it happens, those audience members stepped out of the
theater to see two huge memory aids: twin blue-and-white banners proclaiming
that John Jay College is “Ranked #1 in America by U.S. News &
World Report” for its graduate program in criminal-justice policy.
That’s a memory that’s meant to endure.
Last fall, Jim Gray and other senior
administrators at Duke’s Fuqua School of Business huddled together
just before the unveiling of business-school rankings from BusinessWeek.
Gray, Fuqua’s associate dean for marketing and communications,
recalls meeting in the dean’s conference room as the results
came through over an Internet connection. “All the schools were
logging into this site. It was funny: The connection wouldn’t
work at first because all the schools were flying into this thing
precisely at six o’clock. But we finally got in, and they did
this big countdown—counting down from number twenty-five to number
one. And once they got past number seven or six and we weren’t
yet on the list, we knew that something good was going to happen.”
About 200 students were gathered in a classroom nearby.
As the countdown persisted—and as a lustrous Fuqua ranking became
more and more likely—the excitement kept building. Fuqua ended
up at number five. When that word finally came, the students were
“yelling, screaming, high-fiving”; several of them ran up
to fetch then-dean Rex Adams ’62. Adams climbed onto a bench
and made some appropriately enthusiastic remarks. And the next day,
Gray says, it was back to the serious-minded business of business
education.
Those who want to keep reveling in the world of rankings
can consult Stuart Rojstaczer’s College Ranking Service (www.rankyourcollege.com).
Rojstaczer, a Duke geology professor, unveiled the mock website in
July. The rambling rationalization on the site says, “Through
elaborate meta-analysis that took place over several years at a cost
equal (in 1994 dollars) to the Manhattan Project, we identified 629
independent factors (cabalists on our staff note that this number
is the numerical equivalent of the words ‘Torah’ and ‘life’
combined, and believe we have identified the Holy Grail, so to speak,
of higher education) contributing to the quality of a college.”
Bragging that “our rankings are not static,” the website
advises users to hit the refresh button on their web browser. That
causes the “Mighty Max” computer program to recalculate
the rankings. In fact, the site uses a program to sort colleges randomly;
in the course of less than a minute, Duke, Dartmouth, and Carnegie
Mellon changed places at the top of the list.
|
| "A
process that students should be able to do for themselves, U.S.
News & World Report does for everyone. And then it's assumed
that there is objective truth in the outcome." |
|
Rojstaczer told The Chronicle of Higher
Education that “there is no rational basis to numerically ranking
American universities and colleges.” Rational or not, college
rankings are one aspect of the number-one—or at least a prevalent—cultural
phenomenon. Amazon.com ranks book preferences and Consumer Reports
ranks refrigerators. An international rankings system sorts out the
expertise of Scrabble players. In a paint-by-the-numbers variation,
an artist has produced a work with the elements that, so his polling
data tell him, rank at the top of the public’s preferences: a
landscape with water, mountains, animals, lots of blue, and the figure
of George Washington. The Van Cliburn competition produces the top-ranked
classical pianist—prompting the complaint from a New York Times
columnist that “ranking pianists as if they were Olympic athletes
is inherently inartistic.”
Why the preoccupation with college rankings? According
to a 1997 study by UCLA’s Graduate School of Education and Information
Studies, “Choosing a college is an intangible, expensive purchase
perceived to be fraught with risks, and parents and students may be
using national rankings as impartial sources of reliable information.
The more uncertain the decision, the greater the likelihood that consumers
consult ratings information in an attempt to lower their risks.”
To the extent that they validate a decision steeped in ambiguity,
resorting to rankings can even be emotionally soothing.
The UCLA study found that most
students don’t find rankings to be important. At the same time,
“Users of rankings (those citing them as somewhat or very important)
are more likely to have frequently asked a teacher for advice in high
school, more likely to be high-achieving students, and more likely
to aspire to doctoral, legal, and medical degrees.” That is,
rankings have a special appeal to the brightest and most ambitious
students. “The students who are using the rankings are precisely
those students who have fine-tuned perceptions of what’s important
in choosing a college and who already know, and act on, notions of
which institutions are ‘best.’ Newsmagazine rankings are
merely reinforcing and legitimizing those students’ status obsessions.”
Rules for the Game
In a late-summer series on the agony of actors
vying for roles, National Public Radio’s Susan Stamberg
came to an unsurprising conclusion: “The goal of every
actor is never to have to audition again.” And it’s
not a stretch to say that at least one goal of every college
is never to have to be ranked again.
 That
goal is hardly within reach, a fact that frustrates Arthur J.
Rothkopf, president of Lafayette College in Pennsylvania for
the last eight years. During that time, he’s led the most
conspicuous effort to “fix” the ratings system. It
hasn’t been, he’s quick to acknowledge, a stunning
success.
In the summer of 1995, Rothkopf wrote an opinion
piece in The Chronicle of Higher Education lamenting that “Although
many college administrators have reservations about the value
of rankings as a measure of the quality of academic programs,
we know that the college-going public takes them very seriously.”
And he referred to troubling reports that some schools were
“cooking” the statistics they provided for the rankings.
Every institution that receives federal student
aid, noted Rothkopf, provides the U.S. Department of Education
with certified financial statements each year. Professional
audits also ensure compliance with the regulations of the National
Collegiate Athletic Association. Why not apply the same auditing
practices, he wondered, when colleges file reports used in rankings?
Rothkopf—a former tax attorney and deputy secretary
of the U.S. Department of Transportation—wrote: “While
we may continue to resist being compared and ranked statistically,
the filter provided by a standardized reporting process would
at least correct a major weakness in the present system, which
permits the reporting of data not scrutinized by unbiased outside
examiners. At a time when the pressure on colleges to be accountable
to the public for their cost and educational quality has reached
an all-time high, we simply cannot afford to be less than fully
candid and accurate in reporting statistical data.”
A few months after the Chronicle column was published,
Rothkopf convened a forum at Lafayette to explore how colleges
might collect and verify the data they give to publishers. Participants
included representatives of guidebook and magazine publishers,
leaders of
higher-education associations, college presidents, and campus
researchers. They formed two groups, one to develop a standardized
way of reporting institutional data and the other to create
and enforce methods for verifying them.
 |
 |
The first group produced the
“Common Data Set”; it was later reviewed by registrars,
admissions officers, researchers, and financial-aid administrators
at their professional meetings. The form, refined from year
to year, is still used by U.S. News & World Report and others
in the business of ranking. And so one problem—the complaint
by colleges that they were being asked for essentially the same
information in different ways—was alleviated.
That was the easy part, Rothkopf says. The verification
issue was never resolved. Schools, and their accrediting agencies,
seem to have “no enthusiasm for the idea” of auditing
for the sake of ranking, he says. And it bothers him—a
lot—that the fudging of data persists. “What started
me in this effort was the thought that higher education should
be doing better, that we should be held to high standards of
accuracy. It’s not only embarrassing, but there are real
consumer-fraud issues if colleges are reporting inaccurate data.”
Rothkopf worries about more subtle but equally pernicious
outgrowths of rankings. For example, he says, in order to improve
their “yield” rates or to make themselves look more
competitive, some schools will turn down a qualified student
who doesn’t seem likely to enroll. “The problem is
that colleges are resigned to the rankings, and some are adjusting
their behavior to try to get higher rankings.” |
To Peter Cary, special-projects editor
at U.S. News & World Report, rankings transcend status concerns.
With so much information, much of it conflicting, college-bound students
can find themselves “utterly lost” without “an assessment
of comparative educational quality.” Rankings, he says, can narrow
college choices, but shouldn’t define the ultimate choice. In
forum after forum, he says, he stresses to students and parents that
“one would be crazy to apply to only the number-one school in
our rankings.”
Though a growing phenomenon, college rankings aren’t,
strictly speaking, a new phenomenon. They’ve been around for
more than 200 years, says Ted Fiske, former education editor of The
New York Times. His Fiske Guide to Colleges, which evaluates schools
by several criteria but doesn’t rank them, is considered a standard.
(Fiske, now an education consultant, lives in Durham with his wife,
Helen Ladd, a Duke public policy professor.) In the 1870s, the U.S.
Bureau of Education published lists of colleges by type. And in 1886,
Fiske says, it singled out twelve schools as having “achieved
more than national distinction.”
Fiske says that colleges themselves brought on the rankings
frenzy. By the late 1970s, they were discerning a shift from a seller’s
to a buyer’s market. With a demographic downturn, enrollments
were dwindling and competition for tuition-paying students was intensifying.
So colleges became marketing-minded. “One result of the new professionalism
in college advertising is that promotional brochures are beginning
to look like cigarette ads,” Fiske wrote in a 1979 Atlantic Monthly
story. He went on to argue that “the most obvious problem”
with the newly spirited grab for students was “the abuse of simple
truth, a virtue with which colleges have often presumed to identify
themselves in the past.”
Today he says, “One thing colleges didn’t count
on as they became so savvy about marketing is that Americans know
how to be consumers.” Quite inadvertently, colleges, in their
self-promoting mode, “created a market for people to come in
on the side of consumers, to sort out the overload of propaganda.”
Fiske is concerned that college rankings obscure deeper
issues of campus “flavor and character.” He says, “It’s
inappropriate to say what’s the best college—the issue is
what’s the best college for a particular individual. And criteria
that can be quantified are not necessarily the important ones in making
decisions about colleges. Colleges are like people, and matching a
student to a college is like a marriage. You want to find a place
that coincides with your needs and desires. There’s no way you
can quantify the sort of people who go there or quantify whether you’ll
want them as lifelong friends.”
U.S. News sparked the modern rankings trend in 1983 with
its “best colleges” list, originally compiled from a survey
of college presidents. And it’s received the brunt of the criticism.
Reed College, for one, publicly questioned the methodology and usefulness
of the magazine’s rankings from the beginning. Reed president
Steven Koblik told U.S. News that its project wasn’t credible,
and said the college would not be returning any of the magazine’s
surveys. “Higher education isn’t a commodity like cars or
refrigerators,” he insists. “There aren’t twenty-five
colleges in this country that are best for everyone.”
In 1996, Stanford’s then-president, Gerhard Casper,
wrote to U.S. News’ editor complaining about the “specious
formulas and spurious precision” behind rankings. Casper noted,
“Universities change very slowly—in many ways more slowly
than even I would like. Yet, the people behind the U.S. News rankings
lead readers to believe either that university quality pops up and
down like politicians in polls, or that last year’s rankings
were wrong but this year’s are right (until, of course, next
year’s prove them wrong).” And he disputed the validity
of particular indicators, observing, for example, that a college could
improve its “predicted” graduation rate by “offering
a cream-puff curriculum and automatic A’s.”
The annual college-rankings issue is an automatic winner
for U.S. News: It sells twice as many copies as a typical run of the
magazine, and is reportedly a bigger seller than the Sports Illustrated
swimsuit issue. During the month that the rankings are released, the
U.S. News website gets some 40 million page views, says Peter Cary.
As special-projects editor, he has a purview that extends to the magazine’s
college guide, graduate-school guide, online education guide, and
website.
Those rankings are part of a larger social reality—a
reality that may not be so quick to separate cars from colleges. Claudia
Buchmann, an assistant professor of sociology at Duke, says Americans
like to believe “that things can be ordered in a hierarchy, that
one thing can be quantitatively better than another. There are really
big problems when we do that, especially because people tend to buy
into these things wholeheartedly.” The ready-made hierarchy of
colleges is presented as a tool to help students make decisions. In
fact, she says, it can be an impediment to making the important decisions
that should come from a careful reasoning process.
Buchmann finds it odd that college rankings are seen as
being no less meaningful, and no less appropriate, than product rankings.
But she says they show a familiar American preoccupation with prestige.
“It used to be that the level of education meant something; you
graduated from high school and that meant something, and the elite
went on to higher education. Then the credential of a college degree
was diminished because higher education was so plentiful. But the
prestige of the degree became the distinguishable factor.” In
a culture where prestige does count, she says, “The college a
person goes to matters because of the social networks it affords—access
to an elite that can help in getting a good job.”
In his elite position as Duke’s director of undergraduate
admissions, Christoph Guttentag says he regularly gets unsolicited
college mailings. They’re clearly meant to impress him as someone
invited to rate peer institutions. U.S. News, notably, surveys admissions
directors, provosts, and college presidents for “reputational”
assessments. In sounding out its fellow liberal-arts colleges on rankings,
Alma College discovered a couple of years ago that 84 percent of the
voters were unfamiliar with some of the schools they were asked to
rank, and one-quarter were just guessing. (U.S. News asks individuals
not prepared to evaluate a school to mark “don’t know.”)
There’s little guessing about the fact that colleges
are rankings-sensitive. And from the vantage point of U.S. News, that’s
not a bad thing. “A number of schools have made public pronouncement
that they want to improve themselves in the rankings,” says U.S.
News’ Cary, because they believe the magazine’s ranking
system provides legitimate pointers to “improving academic quality.”
Ohio State University is guided by a so-called 20-10 Plan: By the
year 2010, twenty of its programs should rank in the top twenty, and
ten of those should be in the top ten. The benchmark ranking systems
are the National Research Council and U.S. News. The lead item in
a June “Update from the President’s Office” at the
University of Georgia reports that “For the first time, the University
of Georgia was ranked among the top twenty public universities by
U.S. News & World Report. We now have our sights set on the top
fifteen.”
Caltech’s website points out that the school was
“ranked the number-one university in the U.S. by U.S. News &
World Report in September 1999.” It had leapt from ninth place
the year before. Wagner College, in its employment notices, accents
its U.S. News standing in the “top tier in the Northeast.”
When it’s not possible to make such boasts, the consequences
can be unpleasant. Two years ago, a senior vice president at Hobart
and William Smith Colleges resigned under pressure after a self-inflicted
rankings wound. She had failed to submit updated information that
U.S. News uses to compile its annual survey. As a result, said Hobart’s
president at the time, the college suffered a “profoundly disturbing”
fall from the second to the third tier in the rankings of liberal-arts
colleges.
The quest for higher rankings has had even more extreme
implications. In 1995, Wall Street Journal education writer Steve
Stecklow reported that some colleges were inflating the data they
supplied. In reporting SAT scores, a school in Florida, for example,
was lopping off the bottom-scoring 6 percent of students, thereby
lifting the average about 40 points. A northeastern school was excluding
both international students and remedial students, who together represented
about 20 percent of the freshman class. The practice boosted the school’s
SAT average by about 50 points. Another school excluded the verbal
SAT scores, but not the math scores, of about 350 international students.
The reason: Foreign students often have trouble with English and tend
to do poorly on verbal SATs, but many score better than U.S. students
in math.
• continues
on page two.
|
|
