AUSTIN — Eight years ago, leaders of the University of Texas set out to measure something few in higher education had thought to question — how much their students learn before graduation.
An unsettling answer emerged: arguably, not very much.
That conclusion is based on results from a 90-minute essay test given to freshmen and seniors that aims to gauge gains in critical thinking and communication skills.
The Texas flagship and a few hundred other public universities have joined a growing accountability movement in higher education, embracing this test and others like it that attempt, for the first time, to quantify collegiate learning on a large scale.
But the results have triggered a wave of rancor. Some college leaders are outraged that four years of learning might now be reduced to a single score. Lackluster results have seeded fresh doubts about the country’s vaunted system of higher education.
“Oh, it’s hit us in the gut,” said Andrew Hacker, a Queens College political scientist and authority on college teaching.
The Collegiate Learning Assessment, launched in 2000, has brought rare scrutiny to higher education. Until now, colleges have been largely exempt from the accountability movement sweeping through public elementary and secondary schools yielding the No Child Left Behind law and other initiatives.
In a landmark study published last year, sociologists Richard Arum and Josipa Roksa used the test to measure collegiate learning in the nation. Using data drawn from a sampling of public and private colleges, they shook the academic world with a finding that 36 percent of students made no significant learning gains from freshman to senior year.
“I think it’s extremely troubling,” said Margaret Spellings, the former U.S. education secretary and a longtime advocate of accountability in education. “And God bless Richard Arum for taking this on.”
But a chorus of college leaders reject that this test — or any other — can affirm or refute the essential value of college. They view the CLA as a rough gauge of student learning, grossly inadequate to measure an entire institution.
“I think it’s a very worthwhile attempt,” said William E. “Brit” Kirwan, chancellor of the Maryland state university system. “I don’t think it should be seen as the final answer.”
From time to time, accountability advocates have proposed requiring colleges to show the value they add to the quest for knowledge as a condition of receiving federal aid. But higher education lobbyists and their allies in Congress have “vigorously opposed” attempts to impose a No Child-style system on academia, Spellings said. Perhaps the biggest fear among college presidents is that published test scores might be put to ill use by the collegiate ranking industry.
Yet a voluntary system of accountability is underway. Two groups representing more than 500 public colleges have pledged to give the CLA or one of its rival tests — the Collegiate Assessment of Academic Proficiency or Proficiency Profile — and to publish results by the end of this year. So far, 144 schools have posted test results, including Frostburg State University in western Maryland. Many schools have not participated in the testing, including the universities of Virginia and Maryland.
Teresa Sullivan, president of U-Va., said the University of Michigan gave the CLA when she was provost there. Freshmen scored so high, she said, there was no way for seniors to score higher. She believes U-Va. students would hit the same ceiling.
“If there is no way to improve, why would you invest your money in this?” she said.
The CLA is an essay exam that tests students on skills colleges avow to teach. Responses are judged on use of language, organizational structure and persuasive heft. Students might be asked, for example, to assail the logic in this proposition: Couples should not wed in June, because many failed marriages begin as June weddings.
The University of Texas, one of the nation’s top research universities, was among the first to give the CLA and is using the results to improve instruction. Testing began in 2004 in Austin, under a state mandate.
Last year, UT freshmen scored an average 1261 on the assessment, which is graded on a scale similar to that of the SAT. Seniors averaged 1303. Both groups scored very well, but seniors fared little better than freshmen, according to score reports The Washington Post obtained through a public records request.
“The seniors have spent four years there, and the scores have not gone up that much,” said Arum, a New York University sociologist and co-author of the 2011 book “Academically Adrift.” He reviewed UT’s results at the request of The Post. The school was not among the 24 unnamed colleges in Arum’s study.