(‘Knowing what I know now, I actually wish we’d withdrawn from the rankings earlier,’ Ms. Gerken said)
By- Banibrata Datta, FTFMI
(Special Editorial)
The revolt against the survey, started by Yale Law School, was decades in the making. Yale Law School Dean Heather Gerken floated the idea past a small circle of colleagues. She had a sleepless night and queasy morning. And then, on Nov. 16, she started the revolt.
Like many university administrators, Ms. Gerken had tried for years to get U.S. News & World Report to rethink its law-school rankings. The problem for Ms. Gerken wasn’t Yale Law’s score—it had been No. 1 for more than 30 years straight. She worried about the broader effect on schools and their priorities.
“The U.S. News rankings are profoundly flawed,” Ms. Gerken said in a letter that day. And with that, Yale Law pulled out. Within three months, more than 40 law schools—about 20% of the programs that U.S. News ranks—said they would also end their cooperation and no longer share data with the publication, including 12 of the top 14. A wave of medical schools, led by No. 1 Harvard Medical School, followed. At the undergraduate level, the Rhode Island School of Design (No. 3 among regional universities in the North) and Colorado College (No. 27 in the latest measure of national liberal-arts colleges) withdrew last month.
The rebellion, which has thrown into tumult the most famous source of college rankings for generations of would-be students, was decades in the making. College presidents, deans and institutional research officers said they have regularly raised concerns with the company’s chief data strategist in letters, on phone calls, at conferences and during in-person meetings at the company’s Washington, D.C., headquarters. The rankings, they argued, were opaque, favored the wealthiest schools and promoted practices that didn’t benefit students. They warned against simplifying something as complex as an education into an ordinal rank.
School administrators said U.S. News wasn’t doing enough to address their concerns, and their frustrations continued to build. Yet few took action, spooked by the experience of Reed College in Oregon, whose position plummeted after it stopped sharing its data with U.S. News in 1995. Ms. Gerken broke the dam.
“I think if anybody other than Yale had pulled out, it wouldn’t have made a difference,” said Georgetown Law Center Dean William Treanor. “When Yale pulls out, it’s clear it’s not sour grapes.”
The move has led to a moment of crisis for U.S. News, once a publisher of a weekly news magazine that competed with Time and Newsweek. It began ranking colleges 40 years ago and graduate schools soon after. The rankings quickly grew into the definitive guide for prospective students and families desperate to simplify an otherwise complicated calculation of which school was the best choice.
The publication, which stopped its monthly print magazine in 2010, capitalized on the lists’ popularity and eventually added rankings for hospitals, cars, cruise lines and even states.
U.S. News & World Report Chief Executive Eric Gertler said the company provides valuable consumer journalism to help people navigate major life decisions.
Mr. Gertler said education rankings and related products “represent significantly less than half of our revenue” but declined to share detailed figures. Those related products include badges that schools buy to adorn websites and billboards and customizable online profile pages. It also sells subscriptions to Academic Insights, a tool that provides access to the data behind the rankings, which allows schools to work out how they can improve. Each of those add-ons can run more than $10,000 a year, according to several schools.
The commercial success of the U.S. News rankings led other publications to release their own lists. The Wall Street Journal ranks undergraduate colleges using data pulled mainly from the federal government and student surveys.
As the influence of the U.S. News rankings grew, schools began studying how changes in admissions decisions, fundraising strategy and even the size of seminar classes might boost their positions. Butler University in Indianapolis, the University of Houston and Lewis & Clark College in Portland, Ore., referenced the publication in strategic plans and connected institutional success with better rankings. The University of Florida held a press conference when it broke into the top five public universities in 2021.
U.S. News, owned by real-estate billionaire Mortimer Zuckerman, defended its process, and said it would continue ranking schools regardless of their cooperation.
“Those that describe our rankings as gimmicks don’t understand our rankings,” Mr. Gertler said. “There are billions of pieces of data within our rankings. We spend enormous resources, we have an expert data team, we analyze data to make sure we’re providing important information to our consumers.”
Bard College in Annandale-on-Hudson, N.Y., cooperated with U.S. News data requests, even though more than 20 years ago President Leon Botstein called the rankings a “catastrophic fraud” and “corrupt, intellectually bankrupt and revolting.”
Dr. Botstein said he stayed put partly because the school performed better than he expected, hovering among the top 40 liberal-arts colleges for years. Also, he said, he didn’t think one school’s departure would be able to topple Goliath. Bard pulled out this year.
“Now that there is a critical mass of institutions that are willing to walk away from it, we are willing to walk with them,” Dr. Botstein said.
Critics say that there are myriad ways for schools to goose the numbers. Inaccurate data, including submissions with transposed numbers and efforts to creatively tally alumni donations or research expenditures, are unearthed every few months and detailed on U.S. News’s own website.
In 2021, the former dean of Temple University’s business school was convicted on fraud charges related to a scheme in which he submitted inflated student data. He was sentenced to more than a year in prison.
Last year, the University of Southern California’s Rossier School of Education said a report it had commissioned by a law firm found administrators omitted information from the school’s submission to boost placement dating back to at least 2013. Columbia University admitted in September to reporting incorrect figures in multiple categories. The school dropped to No. 18 from No. 2 after U.S. News turned to other public sources for data, rather than to the school, in the latest ranking.
Mr. Gertler said U.S. News flags significant year-over-year changes in schools’ data and expects schools to report accurate information. “Is this the kind of educational system that exists—where universities that are being relied upon to inculcate values to the future generations of students cannot be trusted to submit accurate data?” he asked.
Even when the information is accurate, some critics say the rankings are corrupted by how the U.S. News data team determines how to weigh each factor.
At a January presentation to law schools, Robert Morse, U.S. News’s chief data strategist, disclosed that he didn’t commit to a particular mathematical model until after receiving schools’ data, according to Ian Ayres, an economist and Yale Law School professor who attended the event. Once that information was in hand, Dr. Ayres said, the team ran simulations giving various factors different weights to see the potential outcomes before deciding on a final method.
Dr. Ayres said that approach violates fundamental social-science research standards in which the methodology is specified ahead of time to prevent anyone from reverse-engineering a preferred result.
“Our expert data team is always modeling to determine the impact of new metrics and data outliers,” Mr. Gertler said. “We never adjust our methodology to prioritize one school over another in our rankings.” U.S. News declined to make Mr. Morse available for an interview.
Among the most pervasive concerns about the ranking is that it discourages schools from trying to keep costs in check by awarding points for per-student expenditures. (The Journal’s undergraduate ranking historically considered per-student spending as a factor, but won’t do so in its next installment.)
Critics question priorities in law-school rankings, where last year 11.25% of a score came from students’ LSAT and GRE scores. Only 3% was tied to the share of graduates who passed the bar exam, a prerequisite for practicing law.
Dr. Treanor from Georgetown Law said U.S. News has repeatedly declined to share the algorithms behind some of its calculations and schools feel it necessary to subscribe to the Academic Insights tool to try to crack the code. “They make their money in not being transparent,” he said.
Mr. Gertler said U.S. News publishes its methodology and any adjustments to weightings each year.
One of the most controversial elements of the ranking calculation is a peer reputation survey, in which administrators are asked to rate competitors. Response rates last year were 28% for medical schools and 69% for law schools. The survey is the sole basis for ranking clinical psychology Ph.D. programs—for which the most recent response rate was 18%.
The response rate for undergraduate programs fell to 46% in 2008 from 70% in 2006 after a group of liberal-arts colleges pledged to boycott the questionnaire. It has continued to drop, and hit 34% last year.
Many presidents, provosts and deans say it’s impossible to make informed judgments about the teaching quality and overall reputation of hundreds of other institutions.
The surveys have given rise to a cottage industry. University marketing teams send out postcards, gift baskets, branded Moleskine notebooks dotted with details about the school and candy made on campus, hoping to be top-of-mind—or at least somewhat familiar—come survey time. Some plaster the walls of airports in cities where presidents are meeting for conferences, and wrap Uber and Lyft cars with their logos and sponsor rides to transport attendees.
After Yale and other schools started pulling out, U.S. News embarked on a flurry of Zoom meetings in December with more than 100 law school deans aiming to better understand and possibly stem the exodus. Some deans likened the experience to pulling back the curtain to reveal the not-so-great and powerful Oz.
“They not only lacked knowledge of some of the very important concerns the deans had raised with them repeatedly, but they seemed unable to articulate the justification for their policies as well,” Ms. Gerken said. “Knowing what I know now, I actually wish we’d withdrawn from the rankings earlier.”
In January, U.S. News said it would tweak its law-school rankings, including putting less weight on reputation surveys and counting graduates with school-funded public-interest legal fellowships the same way they would other employed graduates. The departures continued.
Mr. Gertler adopted a more forceful tone last month, writing in a Wall Street Journal op-ed that elite schools object to their work “because our rankings are something they can’t control and they don’t want to be held accountable by an independent third party.”
Ahead of a meeting earlier this month co-hosted by Harvard and Yale’s law schools to discuss alternatives to the U.S. News rankings, Mr. Gertler took out a full-page ad in the Boston Globe featuring an open letter to Education Secretary Miguel Cardona, who spoke at the event. The letter urged him to demand that schools make their statistics available to the public in a common data set.
Presidents and deans say they already disclose many of those numbers to their accreditors and the federal government. Law schools must submit information on admissions, class sizes, financial aid, attrition and more to the American Bar Association.
Mr. Gertler said inundating prospective students with numbers isn’t a solution. “Students need to go to one destination to be able to see comprehensive, accurate information from an independent third party that they trust,” he said.
U.S. News said Tuesday that its new graduate-school rankings would be released next month. School officials say there could be more high-profile departures among undergraduate programs ahead of the next college ranking, slated to come out in September.
About 16% of undergraduate programs ranked by U.S. News didn’t provide the publication with data last year, up from 7% in 2015. Most of them hover near the bottom of the lists.
Administrators at multiple schools in the middle of the pack say they’d be more inclined to pull out if one of the top colleges, such as Princeton University (No. 1 among national universities for 21 of the past 23 years) or Williams College (No. 1 among national liberal-arts colleges for the past 20 years straight), went first.
