Published July 21, 2006
Steps ranging from better software to more training — and even providing pencils and erasers at test centers — could improve the reliability of scoring the SAT exam, a consultant's report says.
The report, commissioned by the College Board and released Thursday, says the scoring system for the college entrance test has improved since more than 4,000 SAT tests taken last October were given incorrectly low scores. On the whole, scores are reliable, it says.
But the report by Booz Allen Hamilton identifies a series of continuing risks — such as scanners affected by debris or misinterpreting erased marks — and suggests a range of mostly technical steps to provide further safeguards. Overall, it paints a picture of a less-than-infallible exam, noting several areas where current controls fall short of providing perfect reliability.
The College Board and Pearson Educational Measurement, which scores most of the exams, had previously blamed the October errors on the misreading of "marginal marks" and on answer sheets that expanded because of humidity. Some of the recommendations would address those problems, including additional "anchor marks" on the sheets, which reveal whether they have expanded.
In the long run, the report suggests the College Board consider moving the SAT online, something the organization says it has discussed in the past and will consider again, though it has security concerns.
The report was delivered to the College Board, which owns the SAT, in late May. But it then backed off a pledge to make the report public, citing litigation on behalf of students whose tests has been mis-graded. It changed course after receiving a subpoena from Sen. Kenneth LaValle, chairman of New York's state Senate Higher Education Committee.
Robert Schaeffer, a College Board critic with the group FairTest, attacked the report for failing to provide any new insight into what went wrong with the October exams.
"After all the noise and all the promises, they still haven't answered those questions," he said. "It's going to be another arena where they're answered — presumably the courts."
But College Board spokeswoman Chiara Coletti said the organization had already determined that humidity and problems with so-called "marginal marks" were to blame for the October errors. She said the report was commissioned "to determine if what we put in (as a remedy) was effective and if we needed to do anything else."
"We're very pleased with the report because it does confirm our improvements were effective," she said.
Coletti said some recommendations are already under consideration, and called it a "probability" that test centers would soon provide students with proper pencils and erasers to try to head off smudging problems.