NEW YORK – When Peter Braunstein was put on trial last year for a twisted Halloween torture attack, his lawyers used a visual aid to suggest that his actions were the product of mental illness.
It was a scan of the defendant's brain. A doctor testified that the patterns it revealed indicated that Braunstein, accused of donning a firefighter's costume and imprisoning a woman for 13 hours, suffered from schizophrenia.
The New York trial was one of a growing number of instances in which cutting-edge neuroscience has found its way into U.S. courts.
Brain scans have emerged as potentially powerful tools in battles over defendants' sanity. More defense attorneys are seeking scans showing brain damage or abnormalities that might have made it difficult for their clients to control violent impulses.
And experts say there is much more to come — including a few things that seem the stuff of science fiction. Within years, brain scans might be capable of serving as reliable lie detectors.
Similar tests could potentially show whether a plaintiff in a personal injury case is really in pain, or faking it for sympathy, and brain images might even help jurors assess the reliability of a witness's memory.
However, some question whether the legal community might be moving too fast to embrace unproven technology.
"There is a danger here that the cart can get ahead of the horse if we're not careful," said Dr. Marcus E. Raichle, a pioneering researcher of neurology and radiology at the Washington University School of Medicine in St. Louis.
While the potential of brain imaging is huge, he said, it may yet be a leap to claim that scans could be used to accurately detect lies, or say conclusively that a brain abnormality caused a specific person to become violent.
"As a general statement, we are probably not ready to have this in front of a jury," Raichle said. "It is probably premature, but that hasn't prevented it from happening."
Figuring out just what types of neuroscience are ready for the courtroom is one of the goals of a $10 million Law & Neuroscience Project funded by The John D. and Catherine T. MacArthur Foundation.
The effort, which began this past fall, has brought together legal scholars and top scientists to examine the proper role of neuroscience in the legal system.
One panel is investigating what brain research reveals about drug addiction, with an aim toward improving court-mandated treatment programs, and is considering whether addiction can diminish a person's capacity for rationality or ability to conform to law.
Another group is studying research suggesting that brain defects could be responsible for some types of behavior previously chalked up to poor moral character.
"How personally responsible are we for our actions?" said the project's director, Michael S. Gazzaniga, a professor at the University of California at Santa Barbara and director of the SAGE Center for the Study of the Mind.
Similar questions have been weighing on judges.
The U.S. Supreme Court reviewed a legal brief citing brain research as part of its deliberation in Roper v. Simmons, the case in which the court banned the death penalty for children under age 18.
Brain scans were introduced as evidence last year in the trial of John Couey, a Floridian who kidnapped and murdered a 9-year-old girl. The defense said a frontal-lobe asymmetry hindered Couey's control of sexual urges.
Defense lawyers wanted to introduce brain scans last year at the trial of Lisa Montgomery, who strangled a pregnant stranger in Missouri, then stole the dying woman's baby by carving it from her womb with a knife. The baby survived.
Neither approach worked. Couey was sentenced to death. A federal judge refused to allow brain scan testimony at Montgomery's trial. Montgomery also was convicted and sentenced to death.
Richard Henderson Jr., a Florida man who beat four relatives to death, fared slightly better. A jury convicted him, but rejected a call for the death penalty after seeing brain scans the defense said indicated a possible brain disorder.
University of Pennsylvania Law School professor Stephen Morse, a member of the MacArthur Foundation research group, said cases involving neuroscience are still relatively rare — and for good reason.
Things like foolproof lie detectors based on brain scans are still "more popular imagination than reality," he said, although some companies have begun marketing such devices.
And there is some danger, he said, that jurors could be blinded by science that can't reliably answer legal questions of personal responsibility.
"When people see pictures of the brain, they tend to fall prey to what I call 'the lure of mechanism.' They tend to think that we are all machines," he said.
The brain scans didn't prevent Braunstein from being convicted.
Prosecutors had their own medical experts testify that the type of brain scan introduced by the defense couldn't reliably indicate whether he had schizophrenia.
The jury voted to convict. The judge gave him 20 years.
One of his lawyers, Celia Gordon, said jurors concluded they couldn't acquit Braunstein, even if he truly was mentally ill. Under New York law, an acquittal would have set him free.
"I think that was a problem for everyone involved, even for the defense," she said. "We thought he should be in a hospital."