The Detroit Free Press made an astonishing discovery last month. The city's former crime lab had been abandoned. As the paper reported, "Thousands of rounds of live ammunition, sealed evidence kits and case files -- some containing Social Security numbers of rape and assault victims" sat unattended in an old elementary school building, accessible to anyone who happened upon them.
The lab was closed in 2008 after another investigation revealed habitually sloppy analysis among the lab's workers, and an error rate as high as 10 percent, a jaw-dropping figure considering that those analysts' testimony can send someone to prison. The city expected the results of that investigation could have reopened thousands of cases and subject Detroit to hundreds of millions of dollars in damages.
Detroit is far from the only jurisdiction to suffer from embarrassing crime lab ineptitude. Scandals have plagued state crime labs in North Carolina, California, Virginia, Illinois, Maryland, West Virginia and Mississippi; the city crime labs in Houston, Cleveland, Chicago, Omaha, Oklahoma City, Washington and San Francisco; the county lab in Nassau County, New York; and even at the FBI and Army crime labs.
What's going on? Most of these scandals were exposed after DNA testing cleared someone who was convicted based on testimony from crime lab analysts. DNA testing, which is actually grounded in solid science, is showing that forensic analysis isn't as certain or as scientific as it is often claimed to be. It's also showing us that forensics is plagued by bias, both intentional and unintentional, and that bias is caused by poorly structured incentives that often reward crime lab workers for helping win convictions, not for sound analysis. The Innocence Project estimates that bad forensic science contributed to about half of wrongful convictions that were later exposed by DNA testing.
Consider the scandal in North Carolina, uncovered last year after a state investigation and follow-up series by the Raleigh News & Observer. The initial investigation found at least 230 cases in which crime lab workers failed to turn over potentially exculpatory evidence, including three cases that resulted in the defendant's execution.
The News & Observer follow-up found even more, often stunning, bias at the lab, including training manuals that taught workers to consider defendants and their attorneys as the enemy. Many lab workers' performance reviews were actually written by prosecutors. In one case, two blood-spatter specialists were caught on video high-fiving one another after running through multiple experiments until they found one that supported the prosecution's theory of a case.
It isn't difficult to see how having experts who present themselves in court as objective, unbiased analysts report directly to prosecutors or police agencies could present some problems. But that's exactly what's happening. According to a 2009 report on forensic science by the National Academy of Sciences, more than half the crime labs in the U.S. report directly to a law enforcement organization. In some cases, this can lead to overt pressure from police officers and prosecutors to produce desirable results. But most of the time the bias is more subtle, and unintentional.
But that doesn't mean it isn't cause for concern. Bias can creep into an analyst's work in a number of ways, including in the choice of which tests to run, how to records the results of those tests, how to interprets the results, and what the analyst later remembers about the entire process when testifying at trial.
In most government crime labs, there's no check on these problems, referred to collectively as "cognitive bias." The state lab is often the only lab to test crime scene evidence. Even when defendants are given money to conduct their own testing, the labs they choose are often seen by jurors as "hired guns," despite the fact that similar biases also exist in government labs.
There are other problems. Though often presented in court as "science", many forensic disciplines don't incorporate basic scientific principles such as peer review and double-blind testing into their analyses.
For example, giving an analyst a crime-scene fingerprint and the fingerprint of the suspect and asking if they're "a match" can produce very different results from the more scientific method of presenting the analyst with multiple prints (some of which are relevant to the case, some of which aren't), then asking if any two are a match.
One 2006 study by researchers at the University of Southampton in the U.K. found that the error rate of fingerprint analysts doubled when they were first given some information about the case.
In a 2007 report for the Reason Foundation, Roger Koppl, an economist at the Fairleigh Dickinson University, argued that the best way to address these problems is to begin including private crime labs in criminal investigations and prosecutions. [Disclosure: the Reason Foundation publishes Reason magazine, this reporter's previous employer.]
The word privatization often conjures up images of no-bid contracts and crony capitalism. But the idea here isn't to simply sign state crime labs over to private companies. For several years in just such a system in Mississippi, up until 2009, prosecutors were contracting most of the state's autopsies out to a single private-practice medical examiner, because they knew he'd give him the results they needed to win convictions. The results were disastrous.
Instead, Koppl's plan would use competition to remedy the incentive and cognitive bias problems that occur when analysts who are supposed to be objective work for and report to the same government agencies that then use their results to try to put defendants in prison.
Under Koppl's plan, a city or state would create a position of "evidence handler." The evidence handler's job would be to distribute the testable evidence in a case to the appropriate crime lab. Under a fully privatized system, the evidence handler would distribute it to one of a rotating series of private labs. Under a partially-privatized system, there would still be a state lab, but under both systems, in every third case or so, the evidence would be sumbitted to a second or third lab for verification. The original lab would not know when it was being checked by other labs.
This system, which Koppl calls "rivalrous redundancy," flips the incentive problem upside down. For the individual crime lab worker, the incentive is no longer to please prosecutors or police, but to do the most thorough, sound, objective analysis possible. For the private labs, the incentive is to catch the state labs -- or another private lab -- making a mistake. When there's conflict over test results, a third or fourth lab could come into the mix.
Private labs also have an incentive to protect themselves from liability. It's nearly impossible to sue the government. Individual crime lab workers employed by state or local governments are protected by qualified immunity, making it difficult to sue them as well. Private labs don't have such protections, so they're more likely not only to be careful, but to preserve evidence in the case of litigation.
Dallas County, Texas, for example, began sending crime scene evidence to a private lab in the early 1980s. It's one of the few cities in the country to do so. When Craig Watkins was elected as the county's district attorney in 2006, he began to actively seek out cases in which an innocent person may have been sent to prison.
State crime labs often destroy biological evidence after a defendant has exhausted his appeals. It isn't difficult to understand why: Preserving the evidence is expensive, and really only serves the purpose of possibly revealing mistakes somewhere down the line. But the private lab in Dallas had different incentives, which is why it preserved the biological evidence in cases going back 25 years.
In a 2008 interview, Watkins said that preservation was key to helping him discover a number of wrongly convicted people.
"I don't think there was anything unique about the way Dallas was prosecuting crimes," Watkins said. "It's unfortunate that other places didn't preserve evidence, too. We're just in a unique position where I can look at a case, test DNA evidence from that period, and say without a doubt that a person is innocent. They can't do that in other places."
The system Koppl proposes would likely be more expensive than the system most cities and states use now, though if they eliminated the state lab altogether, it's possible the difference would be small. But Koppl estimates that his reforms could be implemented and maintained for years for the cost of a few wrongful convictions -- which this system would go a long way toward preventing.