When the Atlanta Education Fund hired me to independently analyze the test results in the Atlanta Public Schools cheating scandal, I found significant irregularities. But the data alone, while indicating highly unusual gains and losses, weren't enough to prove that anyone cheated. That conclusion required a motivated governor, dogged investigators, and teachers willing to spill the beans. That same level of influence and determination will be necessary to fix the systemic problems that appear to be underlying the recent spate of cheating scandals in Atlanta and elsewhere around the nation.
I'm a numbers guy, and a few figures stuck in my mind as I followed this scandal. During Superintendent Beverly Hall's tenure, 90 percent of the district's principals either resigned or were fired from their jobs, many under what the Atlanta Journal-Constitution described as "increasingly tough performance targets for every school that would become progressively more difficult to hit." Hall received hundreds of thousands of dollars in performance bonuses -- rewards that were based largely on test scores. Similarly, the AJC reported that if a school met 70 percent of its target, all of its employees were given bonuses ranging from hundreds of dollars to more than a thousand dollars.
Just as with the raw test scores, these numbers, by themselves, don't prove anything. But they do suggest that the stakes -- both in terms of rewards and consequences -- were high enough to drive people to cheat. Common sense tells us that educators weren't changing students' answers because they wanted little Joey and Jenny to feel better about themselves. Jobs were on the line. Money and glory were in the offing. One need only look to Wall Street, Las Vegas, or professional sports to see that any time meaningful consequences are involved, some folks are going to try to con the system.
When you have this type of high stakes accountability, the professional testing standards dictate that, to be fair, you must have multiple measures. What constitutes adequate multiple measures is up for debate, but nobody's going to argue that Atlanta's single test, administered once a year, satisfies this requirement.
Cheating is only one of the unintended consequences of too few or inadequate measures. It's far easier to cheat on a single measure than it is to cheat on several of them. And even after you take adequate steps to control cheating, you still need to make sure that what is being measured is a perfect target, because that's the only thing people are going to focus on. Imagine if your doctor based her assessment of your overall health on just one measure -- say, your weight -- while ignoring other metrics. If you're even one pound overweight, you fail your physical, with no regard to your blood pressure, heart rate, or any other indicators. In the Atlanta case, the only academic vital signs tested were students' math and reading abilities, and then only once a year.
Some people are taking these unintended consequences as a call for the end of high stakes testing: You take away the stakes, you eliminate the cheating. Others are calling for national standards for test security to make it much harder for people to cheat. What we really need is some common sense and the aforementioned determination on the part of our educational and political leaders to address the problems.
To do high stakes testing right involves three simple steps: First, you need to set a good target. If you're going to hold people accountable, you better make sure that the measures communicate what you want. Second, accountability should be symmetric. If you're going to hold kids accountable, you should hold teachers accountable. If you're going to hold teachers accountable, you need to hold kids accountable. When education works symmetrically, kids and teachers roll up their sleeves toward a common goal of excellence in student achievement. Third, the system of accountability should be fair: You have to give people an adequate opportunity to succeed. Kids have to be taught the content that is being tested. Teachers have to be provided with the resources they need. Put simply, fair means that you have a chance to succeed if you apply yourself and take it seriously.
Using this straightforward approach as a starting point for thinking about Atlanta's high stakes tests, the school district appears to have come up short on all three counts. It didn't have a good target in the form of adequate multiple measures. The teachers and administrators seem to have been held far more accountable than the students. And given the flagrant cheating, the system was hardly fair.
Which leads me to one final number: 51,283. That's how many children the Atlanta Public School system betrayed when educators erased students' wrong answers instead of teaching them the right ones.