This article was published in OCPA's Perspective magazine View Issue

A growing body of research has been finding that rising test scores in school don’t correspond with improved long-term life outcomes. The latest news is a major meta-analysis combining data from 39 prior analyses to find that increases in reading and math scores have no important impact on high school graduation or college attendance. This builds on a relatively new but impressive series of findings that scores don’t do a good job of predicting educational attainment, employment, income levels or incarceration rates later in life.

As an education researcher, I feel a little like an engineer hearing that the coefficient of gravitation has been cut in half as an energy-saving measure, or a mathematician getting the news that for the sake of simplicity, Pi will henceforth be rounded down to 3. We’ve spent a generation building our discipline—and education reform ideas—on the assumption that rising scores mean better education. If they don’t, we have to rethink everything.

Part of the reason this is such a surprise is that, at an earlier stage, we tested the tests, and they passed. A generation ago, the touchy-feely crowd—who object on principle to setting any objective standards in education—insisted that standardized tests don’t work. So we did a lot of studies like this one, confirming that reading tests really do measure whether people are literate, and math tests whether they are numerate.

We always knew that a standardized test couldn’t capture everything. But what the tests are supposed to measure, they do measure. A math test can’t tell which of the 10 math whizzes in your school might be the next Einstein. But it can tell you which 10 are the whiz kids. And if it can tell you that, it can definitely tell you whether your school is letting some kids slip so far through the cracks that they can’t add and subtract.

Unfortunately, it turns out that what tests measure isn’t, for the most part, what we need to be measuring in order to set educational standards. And if that’s the case, we really don’t know what we should be measuring. Which means our efforts to set high standards and hold schools accountable need to be radically rethought.

Education in America has been moving in a technocratic direction since the post-World-War-II era. The desire to set higher standards and leave no child behind was admirable; what was misguided was the idea that “higher standards” meant improving competencies that could be measured on tests. Our effort to set higher standards actually lowered standards, by reducing the standard of what counts as a good education to mere literacy and numeracy.

You can get a sense of how limited our education vision is by noticing that we naturally wonder which math whiz might be the next Einstein, but we don’t ask which kid in drama club might be the next Christopher Nolan or which kid in debate club might be the next Daniel Patrick Moynihan. Of course the point is not that we care only about geniuses. The point is that our vision of what constitutes “genius” sets the standard for what constitutes excellence for the ordinary student.

We won’t set high standards with the narrow tool of test scores alone. It takes a broad vision to know what education is, and qualitative human judgment to know when schools are providing it. The future of school accountability is the people at large—not a specialist expert class—empowered to use their full human judgment to evaluate schools that they know personally. In other words, school choice and other forms of local control.