Six months following the exposure of misreported admissions numbers at Emory University, Dean of Admission John Latting is pleased with the changes he's seen in the Admissions Office. A new Data Advisory Committee (DAC) has been implemented. And perhaps most importantly, he said, valuable lessons have been learned.
"I think we've come away from the episode with priorities put in perspective in a healthy way," Latting said. "When we're reporting a success it should be real and demonstrated and verifiable in some way. And then, we can celebrate those successes because they're real."
At Emory, though, the University community first reacted to the misreported data during the summer with surprise and confusion. In an Aug. 17 school-wide email, University President James W. Wagner announced the results of an internal investigation, conducted with the help of an outside firm, Jones Day.
The investigation revealed that officials in the Office of Admission before Latting's time and in the Office of Institutional Research had submitted misrepresented admissions numbers to various external audiences through the Common Data Set, which organizations that rank institutions, like U.S. News& World Report, use in compiling statistics.
Latting had discovered that for years, Emory submitted the SAT and ACT scores for admitted students, rather than the requested scores for enrolled students, which were lower numbers. A similar situation occurred for entering students' high school class ranks.
Including Emory and Claremont McKenna College (Ca.), which revealed a similar circumstance last spring, five schools have misreported data in the past year, the other three being Tulane University's Freeman School of Business (La.), the George Washington University (GWU) (D.C.) and Bucknell University (Pa.). Each of the five situations involves misrepresented test scores or high school class ranks, or both, according to U.S. News.
Brian Kelly, the editor and chief content officer at U.S. News, said in an interview with the Wheel that five instances in a 12-month period is a slight increase from what the organization has seen in the past. Usually, he mentioned, there's one to two incidents like these a year.
"We don't know if there's any particular significance; each case seems to be individual," Kelly said, noting that five is a very small percentage of the approximately 1,500 schools the organization ranks. "I'm speculating, but I think one of the good results is that people are paying more serious attention to the importance of this kind of data."
Meanwhile, schools don't only submit these numbers to U.S. News; they are also reported to the federal government and are often published on the institutions' websites.
"I would just stress though that people tend to put the onus on us here and say, 'this is a problem with U.S. News,'" Kelly said. "It's first and foremost a problem for the university."
Bucknell, located in Lewisburg, Pa., is the most recent instance of misreported data. The school's President John Bravman wrote in a university statement that from 2006 through 2012, Bucknell omitted many SAT and ACT scores from its submitted data, most of which were lower.
Similar to Emory, Bravman wrote, the misrepresented information came from "enrollment management leadership no longer with the university." Bucknell's President for Enrollment Management Bill Conley informed Bravman of the problematic numbers.
"These numerical omissions, as relatively small as they were, violated the trust of every student, faculty member, staff member and Bucknellian they reached," Bravman wrote. "What matters is that important information conveyed on behalf of our university was inaccurate. On behalf of the entire university, I offer my sincerest apology to all Bucknellians for these violations of the integrity of Bucknell."
Of the undergraduate schools that misreported data, U.S. News has made only GWU an "unranked" school for the 2013 rankings. It will retain that status until the 2014 edition is released in the fall.
Kelly said that unlike the other schools, including Emory, GWU was removed from the rankings list because of a policy that eliminates a school if it moves more than one place higher due to misrepresented data.
"We feel like we don't have confidence in the validity ... so we move them into an 'unranked' category," Kelly specified, adding that rankings are an "important part of our publication."
Tulane's business school was also moved into the "unranked" category in U.S. News' 2013 graduate school rankings.
Emory's undergraduate ranking stayed at No. 20 for 2011-2013 after the University submitted corrected data for those years following the investigation.
Moving Forward
Kelly, the U.S. News editor, and Latting, Emory's dean of admission, both have plans in mind for moving forward following these investigations.
"There are a lot of reasons that this kind of data needs to be accurate," Kelly said. "And if there are people either making mistakes or willfully fudging the numbers, it's a pretty serious problem."
Kelly said U.S. News "does double check and cross check some of this data." But given the recent circumstances, he said, the news organization is exploring the possibility of having high-level executives at schools that submit information for rankings – such as a president or provost – verify data before submitting it to the Common Data Set and other independent organizations.
"It's the school's responsibility to get the data right, and ultimately that responsibility would lie with the president," Kelly said. "If I were a president, I would want to have confidence that the people working for me were doing their jobs properly and not fooling with the numbers."
And in the aftermath of Emory's own misreported data scandal during the summer, Latting said, the University "spent a good deal of time" gathering information about where Emory reports this information.
"We went back to every single one of them and made sure they understood the changes and what the most accurate information would look like," he said.
As part of the Provost's Corrective Action Plan, the DAC – led by Professor of Pedagogy in the Department of Psychology Nancy Bliwise – is working to establish best practices in data reporting across the University. The committee aims to collect information on what types of data reporting occur across the University and the processes by which data reporting takes place.
Bliwise wrote in an email to the Wheel that the DAC met last week and currently has a series of meetings scheduled for the upcoming weeks. She wrote she will have more information about the committee's work by March.
As part of the corrective action plan, the University will hire a data analyst specifically for the Office of Admission. Latting said potential candidates have been interviewed and hopes to complete the search process soon.
Do Rankings Matter?
U.S. News has been publishing rankings for about 30 years, Kelly said, for institutions ranging from national universities to hospitals, law schools, business schools and many others. He said he feels the U.S. News rankings are a positive way to "objectively evaluate important institutions" and that these rankings have become more important and available with the rise of the Internet.
"It allows people to make comparisons for things that are otherwise difficult to compare," Kelly told the Wheel.
Latting agreed that today, college rankings "matter more and more" to applicants and parents. But, he said, the systems of auditing these numbers are poor.
"It's not a good combination," Latting said. "I hope in years ahead we get to a better place: either improving the auditing of these numbers or attaching less importance to the rankings – one of the other, or both."
Students and alumni have expressed mixed opinions on the importance of rankings from organizations like U.S. News.
College senior Ross Slutsky wrote in a comment on the Wheel's Facebook page that he is "still furious" about the falsified admissions numbers that were revealed during the summer.
"[I] feel that I came here under false pretenses about the quality of the student body," he wrote.
He noted that because he does pay attention to the U.S. News rankings – and given the number of schools that have come forward – there is a "good chance that the reporting mechanisms are flawed and unreliable."
Nicolas Sobredo ('09B), on the other hand, wrote that basing an opinion on one source's college rankings can be misleading, especially because there are so many organizations that provide rankings for higher education institutions.
He added that while rankings can be useful in helping individuals compare different schools, it shouldn't be a "tell-all make-or-break factor."
"At the end of the day, it doesn't matter which university you attend," he said. "What matters is what you do with what you're taught."
– By Jordan Friedman
jordan.m.friedman@emory.edu
Updated Feb. 22 at 11:30 a.m.
