Taking pot shots at the poor quality of most education research can be sport to those of us who are better at tapping away at keyboards than slam dunking. It's so easy that most credible researchers won't engage, which earns our respect, but it sure makes them a lot less fun… (shades of the Kerry campaign, but I digress). To his great credit, the highly respected economist, Eric Hanushek, stepped up to the plate in a letter to Education Week in which he reduced to rubble a California professor’s “study,??? which purported to document falling test scores across the country, but was basically a sham. Actually, not just to rubble but to dust. No more mild mannered, cautious economist; Hanushek’s military background resurfaces.
I was thinking about his decision to write that letter in the context of reviewing a paper this week published by Education Policy Analysis Archives entitled, "The Support Gap: New Teachers' Early Experiences in High-Income and Low-Income Schools.??? There are some credible researchers that signed on to this effort, in particular Rutgers' professor Edward Liu who worked on Harvard's Next Generation of Teachers project. He authored an inordinately valuable paper last year on the abysmal hiring practices of far too many schools, a good twin piece to the report by The New Teacher Project.
I don't mean to imply that this latest study (where Liu is listed as only one of five authors) is all junk and should also be subjected to Hanushek's withering words. The authors--Susan Moore Johnson, Susan M. Kardos, David Kauffman, Morgaen L. Donaldson, and Liu--present some important results from surveys given to new teachers working in either low or high income schools. They were asked about three things: 1) their hiring experiences (one of every five teachers in low- income schools were never even interviewed by their principal); 2) the quality of their mentoring experience (if any); and 3) their feelings about the curriculum used in their schools. On curriculum, teachers were asked if they received “insufficient direction??? about how to use the curriculum they were assigned, if they had “insufficient freedom??? to adopt the curriculum to their needs, and if there was too much teaching to the test.
What was so striking about the study wasn't the data it produced but how comfortable the authors felt when it came to unabashedly revealing their own views of the world. Most of the closing discussion in the paper is a blast at scripted curricula and testing, not really supported by the data… but, oh, perhaps that’s nit-picking. It is true that more of the teachers working in the low income schools reported having insufficient freedom to teach than the teachers in the high income schools (7 versus 20 percent). One might have observed that the survey question itself was irrelevant because new teachers shouldn’t feel free to pick and choose from the curriculum until they know what they are doing, especially in schools that are persistently failing. But more troubling is the researchers’ decision to seize on the variance of 7 to 20 percent, while ignoring that the vast majority of teachers-- 4 out of 5--were just fine with the constraints. Maybe these teachers recognized what the researchers could not: there was less freedom in these schools because kids weren't learning.
Research is not supposed to be advocacy. (That's our job!) It doesn’t sit well when trained researchers feel free to make ill-founded, incendiary observations like: "Detailed prescription about what to teach and how to teach, coupled with excessive reliance on test preparation, may generate some short-term gains on test scores, but ultimately, students will not be well-served." Are we now supposed to believe that teaching kids to read is a short term goal?
I also don't understand a strategy which chooses to knock people over the head rather than let facts speaks for themselves. If ed researchers want to persuade the world of a particular view, their work must bear some semblance of objectivity and disinterest. Studies like this one convert only the already converted. The rest of us just find it sporting fun.