A new paper adds to the growing body of research pointing out just how difficult it is to tell teacher prep programs apart based on outputs.

In a clever re-analysis of previous research evaluations of teacher prep programs in six states—New York, Louisiana, Missouri, Washington, Texas, and Florida—researchers Paul von Hippel and Laura Bellows answer three questions:

• How big are the real differences across all programs in a state?
• Are the programs statistically different from each other?
• How much are any differences due to measurement error?

If you looked at any group of 100 teacher prep programs in a state to see if they are statistically different from the average, you would expect to find that five are indeed different from the average—even though they are all actually identical. In other words, programs that appeared better or worse than average are not really different, but appear to be because of statistical noise.

By correcting for this phenomenon, the researchers find that the number of programs reported in previous studies as being better or worse than the average drops quite substantially from eight percent to just two percent.

Not only do few programs stand out, the average differences are quite negligible, ranging from only .01 to .03 of a standard deviation (as measured by their graduates' contribution to student achievement).

They make a logical conclusion that using teacher value-added measurements to evaluate and make decisions with respect to teacher prep programs is not productive, except for the rare extreme outliers.

The statistical challenges outlined by the authors should not be ignored, but what may be the more intriguing question is why there is so little variation.

From our decades-long study of 1,500 programs, the lack of variation can be explained by teacher education's general unwillingness to embrace what is research based and, perhaps even more importantly, reject what is not. If programs by and large do not adhere to what is evidence-based in their coursework and clinical experiences, then it won't really matter much that some programs appear to be doing a 'better' job than others.

What could drive more variation among programs is missing. No matter how well managed or organized a program may be, or how mesmerizing professors may be, or even how many clinical opportunities are provided—none of those advantages (which some programs obviously have) are going to lead to measurably better programs when research-based practices are ignored.

Until many more teach what is grounded in strong research, the typical teacher prep program will be average—indistinguishable from the rest.