THE BLOG

Data Drunk -- Let's Take the Ass Out of Assessment

Aug 23, 2013 | Updated Oct 23, 2013

Truth -- or the passionate pursuit of truth -- should always be a guiding principle in the educating of our children. So we ought never delude ourselves about whether we are effectively teaching our students.

Without some means of measurement, a belief in our effectiveness can too easily be manufactured. Those of us concerned with teaching our students to think and helping them develop the skills and appreciation for pursuing truth would therefore be hypocrites to disdain data as a valid tool.

But many of us have come to be wary of the manner in which a certain kind of mass-produced data gets presented -- worshipped even -- as truth.

Not all data is equal. Not even close. The poorest performing Mutual Funds can find a 6- or 12-month period in which it showed phenomenal growth.

I have worked at a school that looked awful on paper but did great work luring young men and women out of gangs and despair and into the world of ideas. I still hear from many of those students thanking me and my colleagues for saving their lives.

I've also worked at a school that looked great on paper but wasn't doing nearly as much for its students as it should have.

At that under-performing model school, the principal retired and a few of the teachers were invited to be part of interviews for prospective replacements. Almost all the candidates expressed admiration for our school, based entirely on our API and other hard numbers and said they would keep us on the same course and that future goals would be to continue to raise those numbers.

What an incredible lack of vision. What a complete lack of imagination or understanding about what makes a school worthwhile.

Understandable, I suppose, to be so infatuated with hard data -- since their administrative superiors have apparently made that a requirement. Seems like everyone who doesn't actually work with children is prone to that infatuation.

Reminds me of one of my first jobs as a teenager working in the mail room in the headquarters of the Motel 6 Corporation. I remember the reverence with which we were expected to treat the weekly vacancy reports. We fed them into the photocopier, stapled the copies, and then neatly stacked them for distribution at the executive meeting. Each of us had signed a pledge promising we wouldn't smuggle copies out and show them to anyone. Each week we would shred the old reports so that, God forbid, the competition would never see the numbers. I remember at the time thinking I couldn't imagine anything less inspiring than page after page of numbers representing how many motel rooms didn't get rented -- but I understood the urgency. What could be more important to the Motel 6 Corporation? Empty rooms didn't pay my salary or anyone else's.

Those responsible for the quality of instruction inside our classrooms and those responsible for the quality of entire schools and school districts also need objective numbers to guide them and I suppose that the totality of all these numbers might have some meaning. They might help us understand which districts and schools and teachers are effective and which are not.

But measuring student learning isn't like measuring the vacancy rate of a motel -- or the gross sales of a retailer or the degree to which productivity in a factory is affected by a new policy. In education, the infatuation with data tends to produce expedient and pedantic solutions to often complex challenges.

2013-08-23-holy_grail_0.jpgMultiple choice tests reduce education to a guessing game about a narrow set of facts and ideas. And, by the way, multiple choice tests are not always right. They are sometimes factually incorrect (I've seen it) and they are sometimes so reductionist that they insult the intelligence of our brightest students and even tend to penalize the most analytic thinkers.

These tests celebrate a kind of mediocrity, the mediocrity of the correct and the incorrect. The very belief in a a single right answer -- in that narrow idea about knowledge and understanding -- is a bad habit of which I try hard to break my students. In that sense, the multiple-choice tests themselves are anti-educational. A necessary evil, at best.

But where we really get stupid about this -- where we really put the ASS in assessment -- is insisting that teachers need such test results in order to provide quality instruction for our students.

I am not suggesting that we ought not assess our students' abilities and then consider their weaknesses and the gaps in their knowledge when we organize and plan our lessons. Of course we should. But that is what effective teachers have been doing for generations.

We know how to do that -- we assess our students all the time -- and any teacher who does not, or cannot figure it out, needs to be taught how. But effective teaching doesn't stop there. It barely begins there.

Motivate students and they will learn. Inspire them and they will go further than expected and ultimately take an active role in their intellectual lives. I have seen that happen for more than twenty years and I am talking about children growing up in the inner-city with all the challenges that means.

Stop treating our students like statistics and stop treating their minds like generators of data and they can become so much more.

Illustration by Carly Strauss.