U.S. News Ranked Columbia No. 2, but a Math Professor Has His Doubts
“A professor identified several data discrepancies that Columbia University provided to U.S. News & World Report, renewing the debate over the value and accuracy of college rankings.
Everyone knows that students buff their résumés when applying to college. But a math professor is accusing Columbia University of buffing its own résumé — or worse — to climb the all-important U.S. News & World Report rankings of best universities.
Michael Thaddeus, who specializes in algebraic geometry at Columbia, has challenged the university’s No. 2 ranking this year with a statistical analysis that found that key supporting data was “inaccurate, dubious or highly misleading.”
In a 21-page blistering critique on his website, Dr. Thaddeus is not only challenging the rating but redoubling the debate over whether college rankings — used by millions of prospective college students and their parents — are valuable or even accurate.
Columbia said it stood by its data. Officials said there was no accepted industry standard for the data that goes into college rankings — every rankings project does it differently — and they strove to meet the technical requirements as set by U.S. News. But, they said, the university was not necessarily defending the process.
The dispute has seized the education world, and university officials are in the awkward position of trying to defend themselves against the sleuthing of one of their own tenured faculty, while not alienating him or his colleagues.
“I think the majority of institutions would be happy if the rankings went away,” said Colin Diver, a former president of Reed College, who has a book coming out about college rankings.
“But as long as the rankings are taken seriously by applicants, they’re going to be taken seriously by educators.”
This year, Columbia rose to No. 2 from No. 3, surpassed only by Princeton in the No. 1 spot and tied with Harvard and M.I.T.
Dr. Thaddeus notes that Columbia was ranked 18th in 1988, a rise that he suggests is remarkable.
“Why have Columbia’s fortunes improved so dramatically?” he asks in his analysis.
He does not question that in some ways, Columbia has gotten stronger over the years, he said in a Skype interview this week from Vienna, where he is on sabbatical. But some of the statistics immediately aroused his suspicion because they did not conform to his own observations as a professor in the classroom.
Searching further, he said he found discrepancies with other sources of data that he believes made undergraduate class sizes look smaller than they are, made instructional spending look bigger than it is and made professors look more highly educated than they are.
Columbia officials said that the numbers could be sliced in different ways, including in ways that would be even more favorable to the university, and that the public data sources Dr. Thaddeus used were not always the final word. Asked about Dr. Thaddeus’s analysis, U.S. News & World Report did not address the details, but said that it relied on schools to accurately report their data.
University officials said its rise had hardly been as precipitous as Dr. Thaddeus suggested. From 1988 to 1989, the university vaulted 10 places, to eighth from 18th, in large part because the rankings relied more on data, and less on a survey of reputation among university presidents. It has ranked in the top five for a decade, Columbia said.
High school students and their parents rely on the rankings as a putatively objective way of judging which colleges to apply to. College presidents fear that a drop in rankings will tarnish the reputation of their school and deter top students from applying. Critics of the rankings say that the criteria that go into them — such as class size and instructional spending — can be manipulated, and that the very act of rating schools has produced conformism in the race to the top.
“To say, as U.S. News does, we’re going to rank 392 institutions from one to 392 is just absurd, and it forces all the wonderful diversity in that group into a single template,” Mr. Diver said.
The formula that U.S. News has developed, he said, tends to reward wealth and reputation. Twenty percent of the ranking is based on the reputation of a school among other college administrators, which becomes “an echo chamber,” Mr. Diver said. “Their endowments are at the top, their alumni giving is at the top, and their spending per student at is at the top,” he said. “They’re the richest.”
The rankings have driven colleges to make relatively benign changes in culture, but there has also been some fraud, Mr. Diver said. “There’s been repeated evidence of not just gaming the system,” he said, but also “outright misrepresentation, outright lying.”
Last year, a former dean of Temple University’s business school was found guilty of using fraudulent data between 2014 and 2018 to improve the school’s national rankings and increase revenue. The school’s online M.B.A. program was ranked best in the country by U.S. News & World Report in the years that he falsified data.
Over the years, other schools like Iona College, Claremont McKenna College and Emory University have been found to have falsified or manipulated data.
When Mr. Diver was dean of the University of Pennsylvania law school, he said, the school increased the weight it gave to LSAT scores in admissions because test scores were important in the U.S. News rankings.
“I talked to lots of my fellow deans at the hypercompetitive schools and they all did it,” he said. “Is that corrupt? No, it’s not corrupt. It means that you’re often going to put on the wait list or reject wonderful applicants who have terrific life stories.”
Dr. Thaddeus has not done a systematic analysis of universities other than Columbia, but he does have a bigger agenda.
He believes that all rankings are “inherently suspect,” he said, because they are based on information from the institution being ranked.
There is little, if any, independent monitoring. “Who has the power to audit the books of these organizations that make the reports?” he said. “It’s frightening. There’s almost nothing.”
Dr. Thaddeus found discrepancies in important criteria that go into the ratings: class size, the proportion of faculty with the highest degree in their field, the percentage of faculty who are full-time, the ratio of students to faculty and the amount of spending on instruction. Those categories make up about one-fifth of the ranking formula used by U.S. News.
Columbia claimed that 100 percent of its faculty had “terminal degrees,” the highest in their field; Harvard, for instance, claimed 91 percent, he said.
By poring through the 958 full-time faculty members of Columbia College listed on its website (the only public list he could find), Dr. Thaddeus came up with 69 people (he has since corrected it to 66) whose highest degree, if any, was a bachelor’s or master’s degree (not including a master of fine arts) or a degree that was not in the field that they were teaching.
They include distinguished scholars like the writer Orhan Pamuk, who won the Nobel Prize, but received a Bachelor of Arts from Istanbul University.
“Columbia would surely be a lesser place without them,” Dr. Thaddeus wrote.
Columbia officials said that Dr. Thaddeus was fixated on the Ph.D., but that in many fields — like writing — that might not be the relevant degree. The 100 percent figure was rounded up, officials said, and they believed they were allowed some leeway in deciding what constituted a terminal degree for particular fields.
Dr. Thaddeus himself is listed by Columbia as having a D.Phil., the equivalent of a Ph.D., from Oxford. He says that after successfully defending his thesis, he never bothered to attend the ceremony where the degree would have been formally conferred, but would count himself in the column of those holding terminal degrees.
Of the top-ranked universities, Columbia scored the best in the percentage of classes with under 20 students — 82.5 percent, Dr. Thaddeus said. But by looking at the directory of classes, he calculated that the correct figure was probably between 62.7 percent and 66.9 percent.
Columbia officials said the directory of classes Dr. Thaddeus relied on was not an official record of enrollment used by the registrar.
Using publicly available data, Dr. Thaddeus also disputed Columbia’s reported 6-to-1 ratio of students to faculty, calculating that using the U.S. News methodology, it should be between 8 to 1 and 11 to 1. Columbia officials said that if they included all part-time faculty the ratio would be even lower than 6 to 1, but they believed they were complying with the spirit of what U.S. News wanted.
On the financial front, Dr. Thaddeus found that Columbia’s claim that it spent $3.1 billion annually on instruction was “implausibly large” — more than Harvard, Yale and Princeton combined. He said that Columbia appeared to be including patient care in the spending, something he noted that New York University, for instance, does not do, to its detriment in the rankings.
Columbia officials essentially confirmed that point of his analysis, saying that the university educates close to 4,000 full-time equivalent students in medical fields, and that instruction and care often happen at the same time.
Dr. Thaddeus, who has taught at Columbia for 24 years, has made a hobby of provoking his employer. He said he had been radicalized by the experience of being chair of the math department from 2017 to 2020, when he discovered how secretive the university was. Since then, he has challenged the administration on subjects like administrative bloat, the management of its endowment and, now, the rankings.
“I’m a gadfly in that I’m willing to espouse causes that other people haven’t taken up,” he said, adding that when it comes to the rankings, “People don’t like talking about it, just like they don’t like talking about cheating on exams.”
No comments:
Post a Comment