Help us put a great teacher in every classroom

JOIN US

Building Better Teachers

NCTQ Teacher Prep Review

FAQ

1. How can universities and schools of education participate in this Review?
2. What if our institution does not wish to participate?
3. Our school of education is in the process of getting or renewing national accreditation. Can't the Review wait until that is over?

1. With what authority does NCTQ act?
2. If an education school meets state standards needed for program approval and is accredited, why do NCTQ's ratings matter?
3. How are NCTQ's standards different from those of accreditation agencies?
4. NCTQ seems more concerned with inputs than with outputs. Isn't the bottom line whether or not an institution is producing effective teachers?
5. Might meeting NCTQ standards cause an education school to violate either state or federal regulations, if not accreditation standards?
6. Given NCTQ's long-standing criticism of education schools, how can it claim to be impartial?
7. How were NCTQ's standards formulated?

1. How thorough is NCTQ's Review methodology?
2. How long does it take NCTQ to rate a school?
3. Why should syllabi be considered useful sources of information?
4. NCTQ likens syllabi to blueprints, implying that items that are not on it can't be added later. Hasn't NCTQ ever heard of "add-ons"?
5. Don't syllabi "belong" to the professors who created them?
6. How can NCTQ be certain that it has an accurate read on a course, as some syllabi are clearer than others?
7. NCTQ clearly looks at a lot of course descriptions. Can a one-paragraph description of a course in a catalog reveal all that much?
8. Are all of NCTQ's standards weighted equally?
9. How can NCTQ have anything to say about the quality of an education school without visiting the campus?
10. Do education schools feel that NCTQ ratings are accurate?

1. Regarding NCTQ's standard on admissions, you hold up as models countries that are pickier about teacher candidates. But those countries don't have the same democratic philosophy as the United States.
2. When NCTQ advocates for higher admission standards, is it essentially saying that only teachers who test well should be allowed to teach?
3. We know a lot of good teachers who didn't score well on the SAT or ACT. Why do you value these tests when they tell us nothing about a teacher's ability to be effective?
4. There are already shortages of some teachers and will soon be more as lots of teachers retire. This isn't the time to raise admissions standards.
5. If we apply NCTQ's admission standards, education schools will not be able to accept as many minority teachers—something critical to their core mission.
6. Regarding NCTQ's standard on reading, why should anyone teach the five components of scientifically based reading instruction when Reading First didn't live up to its billing?
7. What can be wrong with a balanced literacy approach to reading instruction?
8. Regarding NCTQ's standard on elementary training in mathematics, what is so magic about its recommendation of three math courses?
9. Elementary mathematics is pretty basic. Won't anyone who passed his or her math classes in high school know enough math to teach first graders?
10. When it comes to teacher preparation, why does NCTQ place such a high value on content? Don't students need critical thinking/21st century skills?

We have sent introductory letters to university and college presidents as well as to all the deans of schools of education and heads of all departments that have received state approval to train teachers. The letters will ask them to confirm contact and demographic information about these institutions that we have already obtained. In the months ahead, we will be corresponding with the persons designated by the institutions to serve as point people, asking for additional information. These follow-up requests will be sent in stages, so not every institution will receive one at the same time.

If you have further questions about how to participate, please contact us at (202) 347-0487 or go here.

Currently, the public views schools of education as largely interchangeable. And if people happen to hold negative views of teacher preparation programs in general, they will not bother making distinctions for a specific institution, no matter how well designed its program of preparation happens to be. The Review will give the public the information it needs to see that there are, in fact, differences among programs. High-quality programs, therefore, should welcome the opportunity to participate.

Institutions that choose to train teachers for the public schools in publicly approved programs take on a responsibility to be transparent to the public about their requirements and coursework. For that reason, NCTQ and U.S. News & World Report will publish ratings of how well the vast majority of teacher preparation programs meet the standards of the Review, regardless of these programs' degree of participation. For all programs, some of the information needed for the review is publicly available and will be used as the basis of NCTQ's judgment. If a public university chooses not to participate, NCTQ plans to make open records requests of it to gather the documentation the review requires. In many states, the public agencies that approve teacher preparation programs collect the documents that we will need from both public and private colleges and universities, so we will make open records requests of them. And in cases where we cannot obtain documents needed to make ratings, NCTQ will make an estimated rating based on what information is available.

Ultimately, NCTQ believes that all but a few institutions will see that it is both their responsibility and in their interest to work with NCTQ for the Review. This is what NCTQ found when it conducted its Illinois review, for which every education school but one provided NCTQ with the documents necessary. NCTQ and U.S. News & World Report want the Review to be as comprehensive as possible. And the leaders of schools of education surely do not want ratings of their programs that are not based on the fullest examination of their programs, for such ratings may not fully capture their program's strengths. Estimated ratings are of limited benefit to everyone, including the programs themselves.

Actually, NCTQ's advisors who work in schools of education have told us that the preparation for national accreditation will make it much easier to complete the document request for the review. And institutions will be given adequate time to provide NCTQ with the necessary documents needed. While NCTQ does not wish to impose too much of a burden, it believes that the Review is important enough to the public and the institutions themselves to merit the time that will be required of them.

By choosing to prepare public school teachers, education schools and the institutions in which they are housed, whether public or private, are operating in the public arena. As such, they are rightly subject to the scrutiny not just of regulators and accrediting bodies, but of organizations working for the public interest. NCTQ is one such organization devoting the time and resources to evaluate teacher preparation programs. Others are welcome to do the same.

Moreover, NCTQ has a demonstrated track record of conducting high-quality research on crucial policies and practices that affect teacher quality. This track record will give NCTQ's findings about the design quality of education programs credibility. But it also means that NCTQ will make every effort to ensure the validity and reliability of the national review's design and methodology. NCTQ's reputation is too valuable for it to be squandered.

By design, state regulators and accrediting bodies only issue "pass/fail" grades to teacher preparation programs. Unfortunately, this leaves consumers—aspiring teachers and schools which hire teachers—in the dark.

Furthermore, there are clear indications that state standards set a low bar and that national accreditation is of dubious value. Nationwide, a miniscule fraction of programs —39 out of some 7,000 housed in roughly 1,400 schools of education!—have ever been put on probation by a state, and none of them have been shut down. The public may draw the conclusion that the rest are performing satisfactorily, which is just not the case.

As far as national accreditation goes, not a single study has found that an accredited school of education is of higher quality than non-accredited schools. NCTQ's own review of Illinois schools of education found no "value added" by the national accreditation process in the 55 programs it evaluated in accredited education schools. While accrediting agencies have expressed a commitment to utilize the student performance data to drive program improvement from state data systems now under construction, the evidence thus far suggests that right now their standards do not adequately discriminate between programs of varying levels of quality.

Here's just one example (from the many that can be pulled from our various education school reports) of the difference between an NCTQ evaluation and the standard accreditation process. The typical accreditation agency says that, when it comes to admissions, education schools must have multiple standards that are clearly described and well-advertised. The agency does not specify, however, what these standards must be. NCTQ says that the institution should only accept teacher candidates who are in the top half of the college-going population.

NCTQ is absolutely clear about the parameters of our analysis. We are only assessing the fundamentals of a program, those features of a program that are necessary—but not sufficient—to produce well prepared teachers.

The best way to explain this approach is by way of example. If we determine that a program is not requiring its elementary teachers to take a course in reading, it doesn't much matter if it offers a great reading course as an elective or that the professor teaching that course is exceptional. All that matters is that the course is not required.

The measurable effectiveness of teachers an institution produces is of course critically important. Unfortunately, only three states—Louisiana, North Carolina and Tennessee—publicly report the data that show how much impact a given program's graduates have on student learning. For programs in those states (whose number will grow over time), we have developed a standard that will track how much impact their graduates add to student learning. In states where it is not yet possible to apply this standard, we are evaluating whether education schools are at least using other sources of data on the performance and retention of their graduates from surrounding school districts.

Nevertheless, even when such data become more widely available, we will also still need to conduct a standards-based examination. Why? Here are a few reasons:

• A pure value-added approach runs the risk of identifying programs as high-performing when they only look good by comparison to relatively low-performing programs in the same state.
• A value added approach will teach us which are the worst programs—but we won't know how to fix them.
• Programs with tough admission standards are likely to score higher, but they may not be adding significant value.
• NCTQ's standards address the design of teacher preparation programs along the most significant dimensions, not just how well they prepare their teachers to teach reading and mathematics, which is all that student outcome data can tell us.
• It's hard to parse out the impact of districts' own professional development and curricula from the impact of the preparation program.

Asking to use only value-added, "outcome" data to determine teacher preparation quality would be like only using norm-referenced data to ascertain how much students learned. A norm-referenced test compares students only to one another, without indicating what students should be able to do. The Review, like criterion-referenced tests, measures programs against a set of clearly defined performance standards.

This argument, heard often from education schools we studied in Texas, does not hold water. NCTQ is suggesting that institutions exceed state standards, which is certainly not a violation. In the Texas study, in fact, many approved and accredited education schools earned the highest rating on a variety of NCTQ standards

State regulations, it is true, generally do not encourage education schools to meet a higher standard, but states certainly do not prohibit education schools from doing so.

While NCTQ does not defend the status quo, we are deeply committed to high-quality formal teacher preparation. In fact, we consider education schools to be necessary, whereas many education reformers argue that the only solution to improving teacher quality is to attract more talented people to the profession.

Unfortunately, studies that compare teachers who enter the classroom with little or no training with those who have gone through undergraduate or graduate-level teacher preparation find no aggregate difference in performance. Only one conclusion can be reached: Most education schools—but not all—are not adding value. NCTQ believes that teacher preparation, if it is selective and rigorous, can and should add real value.

To the extent that high quality research can inform how teachers should be prepared, NCTQ uses that research to formulate standards. Unfortunately, there is not a large body of research in education that connects to teacher effectiveness. The lack of standards and research supporting almost everything that comprises teacher education was made all too clear in the exhaustive review conducted in 2004 by the American Educational Research Association. That effort reviewed every aspect of teacher preparation and found little to no support for almost all current practices.

In areas where there is strong research evidence—such as effective early reading instruction—our standards are based firmly on that evidence. Our other standards, where research is not as strong, have coalesced from research findings on teacher effectiveness, consultations with expert panels, the best practices of other nations and the highest performing states in the nation, and, most importantly, what superintendents around the country tell us they are looking for in the teachers they hire.

In a word, exhaustive. We base our evaluations on many different sources of data. A key, but far from exclusive, source is course syllabi. We review syllabi provided directly from education schools, not downloaded from websites. We examine all the syllabi we need to make an accurate assessment of what a program teaches. For the Illinois study, NCTQ evaluated as few as four syllabi for an institution for which we rated only one program, but evaluated up to 46 syllabi in larger institutions for which we rated multiple programs.

Beyond review of syllabi, NCTQ has subject-matter experts review and rate textbooks and reading packets put together by instructors for reading and elementary mathematics courses. For reading courses alone, NCTQ has already reviewed over 630 texts.

NCTQ will also look at student teaching placement information and handbooks; graduate and employer surveys; institutional admissions standards and an education school's own admission policy; general education course requirements; course requirements for secondary teachers in their subject area(s); and professional course requirements and descriptions.

Following a similar methodology for the review of Illinois education programs, NCTQ examined 3,205 separate files—and that was just for 111 programs in one state!

NCTQ's analysis of an education school is not a minor undertaking. For each school, the entire preliminary rating requires approximately 40 hours. Each analysis starts with approximately 18 hours of an initial examination of catalogs, websites and syllabi collection. This sets in motion a process for verifying and double-checking scores; the use of expert consultants to examine syllabi and textbooks (with requisite checks on their work); and the work of separate teams conducting survey work.

Syllabi are the basic units of the design of teacher preparation programs. When professors develop their syllabi, they do so not just for the benefit of their students, but also to provide assurances to their departments about the material they intend to cover. In many cases, syllabi must be turned over to states and/or accrediting bodies for approval purposes. We trust in the professionalism of instructors by presuming that their syllabi will accurately reflect the important concepts addressed in their courses.

In effect, a syllabus is like an architect's blueprint for a wing of a building.

It is only because NCTQ is looking for evidence of basic, essential topics that we can make fair use of syllabi for rating course quality.

Here's a useful way to better understand NCTQ's methodology: If a syllabus for an early American history course contains no mention of topics associated with the American Revolution, one might rightfully suspect that the course is deficient, because the Revolution is considered a basic, essential topic. But it wouldn't be as troubling to discover that Benedict Arnold was omitted from the syllabus. The professor might not have thought to list Arnold, and in any case he might end up talking about Arnold in a lecture—not unlike adding a sconce to a set of light fixtures in a building after it was built. But even if the professor doesn't, it would be unfair to assume that the course is deficient as a result of the omission, because Arnold is not a basic, essential topic.

When NCTQ rates a reading course, for example, we look for references to the five components of scientifically based reading instruction, considered to be no less fundamental to the teaching of reading than the American Revolution is to early American history.

NCTQ believes that the tradition of "The professor owns the lecture hall" is antithetical to learning all we should about educator preparation. As with all sciences, the sharing of data and techniques is a necessary condition for creating an evidence-based consensus and extending the boundaries of knowledge. In fact, many colleges already post syllabi on publicly accessible websites. An excellent example is the Massachusetts Institute of Technology's visionary OpenCourseWare, where materials including syllabi, course notes, assessments, projects and videos are available for 1,900 courses.

In any case, it is standard operating procedure for state departments of education, higher education bodies and accrediting bodies to collect these syllabi, for much the same purpose as our own.

When we're not certain, we say so. We mark the syllabus as unclear. If the text is equally ambiguous, the course is rated as unclear. We never give a rating without sufficient evidence. Another point about syllabi is important. Syllabi tend to provide a more ambitious picture of the content of a course than the professor is usually able to achieve. Professors generally overestimate what they will be able to accomplish, not underestimate. For that reason, a methodology relying partially on syllabi could end up rating a course higher than it might actually deserve.

We are using course descriptions only to evaluate elementary content preparation in, for example, biology or American history. The scope of such coursework can reasonably be expected to be clear from a course description.

An example below illustrates how course descriptions enable us to rate a program on whether elementary teacher candidates are required to take an adequate course in ancient world history. Here we were looking for a course that provides a general historical narrative of all major civilizations in ancient times.

Sample description of a course we deem adequate:

World Civilizations I
A study of Western and non-Western cultures from antiquity through the Protestant Reformation. Emphasis on the development of the classical and medieval West and an introduction to major world cultures and religions.

Sample description of a course that may have merit, but which we deem inadequate for this requirement because it does not provide a foundation of understanding world history and addresses too great a timespan.

Western Civilization
Team taught by instructors from a variety of disciplines. Emphasis on conceptual approach to intellectual, cultural, political, economic, social, and technological issues that have formed the spirit of the various ages from ancient Greece to the present.

Some standards are more important than others. Some have been field tested longer than others. Some articulate goals which we know are not widely practiced in education schools, but they represent a direction which we believe education schools should move. For these reasons, not all standards receive the same weight in a study, and some may not be weighted into programs' overall performances at all.

Absent sitting in on courses for an entire year, NCTQ does not believe that campus visits would provide a measurably better perspective than our examination of formal course materials and surveys does. NCTQ is looking at the critical foundations upon which the quality of a program is built, and no amount of engaging classroom interaction can fill in the gaping holes we often find in the content, requirements and sequence of a program.

Campus visits may be seen as essential by education schools to assessing quality because they are a customary feature of the accreditation process. Undoubtedly they do serve a useful purpose in assessing programmatic features that are relevant to accreditation. However, the link between accreditation and program quality is by no means evident, and, by extension, there is no evidence that these site-based visits offer a definitive measure of program quality. In contrast, our analysis is more likely to yield an accurate measure of program quality than are short visits orchestrated by the objects of our study.

Many education schools have claimed that our ratings are in error. However, the interpretation of "error" for schools is clearly different from NCTQ's definition. In some cases, the source of their perception of error is based on a lack of understanding of our standards and the criteria by which we evaluate them. Such perception of error will decrease with greater familiarity and with increased efforts on our part to communicate our standards and indicators. But let's be candid: any effort to rate any set of programs that met with universal approval by the programs under review would immediately and rightly be suspect as being too easy.

Here's an example of the sorts of disputes that arise regarding our ratings. The following comments sent to us by Illinois education schools in response to our evaluation of their programs in Ed School Essentials: A Review of Illinois Teacher Preparation

Comment on rating on graduate admissions standard:
We fail to understand the criteria that a candidate must be in the top 50% of his or her graduating class for graduate students. The candidate has already proved that he or she can graduate from college.

NCTQ response:
Graduation from college cannot be deemed a sufficient criterion for admission to a graduate teacher preparation program.

Comment on rating on elementary mathematics content standard:
Findings do not take into account the additional mathematics coursework required to fulfill the quantitative reasoning general education requirement, or the undergraduate mathematics coursework completed by students in the graduate special education program. Findings also discount mathematics methods courses, despite the fact that several math content topics are indicated in the course syllabi.

NCTQ response:
Our standards for mathematics preparation of elementary and special education teacher candidates are not based on the general audience coursework noted. We do not discount the importance of mathematics methods coursework—it is separately evaluated in another standard.

In the first comment you can see that the rationale for selectivity in admissions for graduate school has been rejected by the education school. Any rating we make on a standard based on selectivity in graduate admissions is likely to be seen as an "error" by this institution. In the second comment, the education school advances the argument that general education mathematics coursework—the type that might be taken by a history or business major to fulfill a graduation requirement—is as relevant to elementary and special education teacher candidates as mathematics coursework designed for them. Keep in mind: knowing, let's say, calculus is not in and of itself of much value to teaching elementary students basic math. That's why we're looking for specific kinds of math coursework. And as for the notion that math content can be adequately covered in methods courses, we simply do not see how this can be true as a general principle. Math methods courses are by their nature pedagical, and cannot provide the depth of knowledge that teachers need.

Our democratic philosophy seems much more alive at the doors of education schools than at the doors of our K-12 schools, with the result that it doesn't have a democratic effect at all. You won't find high-performing school districts willing to take in teachers who were themselves poor students and have demonstrably low academic performance.

Where are those teachers teaching? It is poor and minority children who are assigned the teachers with the weakest academic backgrounds. The notion that academic background shouldn't matter all that much has had disastrous consequences for poor and minority children, the ones who are most in need of a high-quality education. We tend to be okay with letting low performing teachers into the profession as long as they don't teach our own kids.

That's why NCTQ points to other countries. Those which have better education systems than our own, and are more successful at educating minority students who live in poverty, have much higher standards than the United States does for entering the teaching profession. Finland's education programs, for example, only admit the top 10 percent of their high school graduating classes, Singapore's the top third. In many of our nation's teacher preparation programs, by way of contrast, there are virtually no admission standards at all.

Less than five percent of all education schools are housed in institutions considered to be most selective, but at least one quarter of all education schools are housed in institutions which have no standard for admission. We're advocating that the floor be no lower than standards admitting the top 50 percent of the college-going population—still a long way off from the practices of other countries.

Actually, if you look at the body of research on such tests, its overall conclusion is that teachers with higher SAT/ACT scores appear to be relatively more effective. That doesn't mean that an applicant who didn't get a 1400 on the SAT or a 23 on the ACT might not become a great teacher one day. But it does suggest that there should be a floor on the SAT/ACT admission standard. To the extent that SAT and ACT measure intelligence and verbal and academic ability, being smart is generally a benefit in the classroom, as it is in any profession.

That said, NCTQ's standards also give credit to education programs that admit candidates with high GPAs but who for some reason did not test well. So NCTQ does not penalize programs that are using valid means of selecting high-caliber candidates other than standardized tests.

There's little evidence from states which have raised their standards that big teacher shortages ensue. In truth, raising standards makes the profession more attractive to academically talented individuals who are otherwise put off by the profession's low standards. Massachusetts ignored warnings about shortages when it raised its standards to among the highest in the nation and has not experienced any shortages. Likewise, England found that teaching became the most popular profession among undergraduates and graduates after program standards were raised.

In fact, because many people getting teaching degrees never intend to teach and seem to be getting teaching degrees only because it's an easy major, some drop-off in admissions may not mean that fewer teachers are available in classrooms.

While a far lower proportion of the most talented minority students choose to become teachers than do talented white students, highly selective education programs that heavily recruit talent of all colors do succeed in attracting minority teacher candidates. For example, 30 percent of Teach For America teachers are of color.

The long-term strategy to achieve a teaching force that better mirrors the student population is to immediately improve the educational prospects of every child by putting an effective teacher in every classroom. Those effective teachers will be produced by education schools with higher, not lower, admission standards. Among many other things, the fruit of more effective instruction will be many more minority high school graduates qualified to enter teaching, no matter how selective admissions become.

The evaluation of Reading First did not in any way, shape or form show that scientifically based reading instruction is ineffective. The impact study only showed that Reading First schools did not experience greater gains in student reading achievement than control-group schools. One of the primary theories for why this is so—promulgated by, among others, one of the contractors that conducted the impact study—is that there was a high level of spill-over of scientifically based reading instruction into the control schools. The study, in other words, was inconclusive.

Unfortunately, evidence suggests that programs touted as "balanced" are frequently little more than whole language approaches with, at best, some cursory coverage of the five components of scientifically based reading instruction.

A program that attempts to balance a scientific approach with a literature-based approach—that is, one that isn't a whole language program in disguise—would meet NCTQ's reading standard. As long as the program adequately covers the five components without including non-research-based strategies (such as three-cueing systems), NCTQ would not question this approach.

The three-course requirement is not arbitrary, but based on a calculation of how much class time is required to cover the 12 essential topics of elementary mathematics. A two-course requirement means that the institution is either skipping essential topics or treating topics too lightly. A four-course requirement is likely to be inefficient.

Further, the lack of any clear consensus among American institutions about what aspiring elementary teachers need in mathematics is quite problematic. The diffuse requirements compare poorly with the more consistent standards found in other countries. In the United States, institutions require anywhere from no math courses to five. Some institutions require that these courses be designed specifically for the teacher; others insist that teacher candidates take the same courses as other students on the campus. Many institutions fail to delineate the type of course that a teacher should take, viewing all courses as equally valid.

Clearly this absence of a standard must be resolved in order for the United States to improve its standing in mathematics performance with other nations.

We should note that we do make an exception to our three-course standard. Institutions that have a highly selective admission process can probably cover all the topics needed in two courses.

In a word, "no." Teachers cannot rely on the procedural knowledge of mathematics that they acquired as elementary students themselves to be able to successfully teach those topics. Even first grade teachers need to acquire a much deeper understanding of mathematics than is reflected in their ability to get the answers right on a basic mathematics test. They must know how the number system is put together; why the procedures for adding, subtracting, multiplying and dividing are what they are; and that algebra is just generalized arithmetic. Teachers must know elementary concepts at a depth sufficient for instruction, an understanding that can best be acquired through classes specifically designed for future teachers.

They need this knowledge because, in the past, teachers have made mistakes—by failing to understand why students make common errors, or that there are multiple, equally legitimate, ways to arrive at an answer.

For example, American elementary teachers sometimes have trouble teaching children who have emigrated from Mexico. The teachers can't understand the students' procedures, labeling them incorrect even though Mexican schools' methodologies are simply different, not incorrect. But many American teachers have memorized algorithms without understanding how and why they work. As a result, they think the Americans method of computation is the only correct one.

This argument isn't new, and the dichotomy drawn between content and 21st century skills misses the point. Critical thinking is impossible without acquiring substantive knowledge. Someone who knows a lot about baseball is able to think critically about baseball, but that does not mean that he can think equally critically about nuclear physics. Cognitive scientists have proven over and over again that "critical thinking" is largely associated with depth of knowledge of a specific domain of knowledge. It is not, as is commonly understood, a skill unto itself.

The central importance of content knowledge underlies the design of the Common Core Standards, which have now been adopted by 46 states and which will be implemented in the next few years. NCTQ has good reason to believe that teachers who have learned more about the subject matter covered by these standards will be better equipped to help their students master them.

Therefore, elementary and special education teachers need to have a broad understanding of many subjects in order to field and pose questions about the myriad topics that come up day to day in the classroom. They also should know and love a particular subject well, inspiring students to share that passion. We believe that providing teacher candidates with a broader knowledge base is one of the best ways to prepare teachers to handle the long-term demands of their complex jobs.