Assessing the Quality of Math Licensure Tests

This analysis includes all subject-matter math tests currently in use for aspiring teachers to earn an elementary certification. It does not include tests used for additional endorsements, or tests of basic skills or performance assessments. This analysis counts tests as available even when candidates can bypass them through other means (e.g., by completing a teacher prep program). Furthermore, if the state identifies a primary test used for certification yet allows other test options to substitute for that test, all offered tests count in the analysis. A state’s policy counts as requiring all candidates to pass the test if the state does not provide any alternative options to bypass either taking the test or passing the test at the required minimum cut score.

The analysts for this work are experts in mathematics, with experience teaching mathematics as well supporting others in becoming stronger math teachers.

Analysts used official practice tests made available (for free or for purchase) by the state or testing company and official study guides or related information about the content of the tests. Analysts did not use materials developed by third-party vendors (e.g., test-prep or tutoring companies).

Analysts reviewed all available materials and gave assessments credit for any topic that appeared in any data source. For example, if “understanding patterns, relations, and functions” is mentioned in the study guide but not in the sample test, the assessment gets credit for that topic, and the same is true if the topic is mentioned in the sample test but not the study guide. Test materials only need to mention a topic once to earn credit for that topic.

Each content topic (e.g., numbers and operations) receives a rating of adequate or inadequate.

For each subtopic (e.g., compute fluently and make reasonable estimates), analysts look for evidence of all three aspects of mathematical proficiency: procedural fluency, conceptual understanding, and application (defined below). To be adequate in a topic, the test must address an average of half of all subtopics across all aspects (e.g., a test may address more than half of topics in one aspect and less than half of topics in another aspect, and it can still score adequate if it covers at least half of all topics). 

• Meets an average of at least 75% of content subtopics and aspects from each content topic AND
• Does not combine math with other subjects (e.g., does not test math in a subtest that also includes science)

• Earns an adequate score in all four topics (addresses at least half of all subtopics and aspects in each area) AND
• Does not combine math with other subjects (e.g., does not test math in a subtest that also includes science)

• Earns an inadequate score in one or more areas AND/OR
• Combines math in a test or subtest with other topics beyond math (e.g., a subtest includes both math and science)

• Has no coverage in one or more components (addresses 0% of topics in that area)

• Procedural fluency: Whether someone knows mathematical facts, can compute and do the math,¹⁴ and whether someone has skill in carrying out procedures flexibly, accurately, efficiently, and appropriately. ¹⁵

• Conceptual understanding: Whether someone can make sense of the math, can reason about and understand math concepts and ideas,¹⁶ and comprehends mathematical concepts, operations, and relations.¹⁷

• Application: Whether someone can solve a wide range of problems in various contexts by reasoning, thinking, and applying the mathematics they have learned.¹⁸ Examples may include:

• • Strategic competence—ability to formulate, represent, and solve mathematical problems. 
  • Adaptive reasoning—capacity for logical thought, reflection, explanation, and justification.
  • Productive disposition—habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one’s own efficacy.

Numbers and operations

• Numbers and operations (as an umbrella term).
• Understand numbers, ways of representing numbers, relationships among numbers, and number systems.
• Understand meanings of operations and how they relate to one another.

• Compute fluently and make reasonable estimates.
• Relationship between numbers and operations and other aspects of math.

Algebraic thinking

• Algebraic thinking (as an umbrella term).
• Understand patterns, relations, and functions.
• Represent and analyze mathematical situations and structures using algebraic symbols. 
• Use mathematical models to represent and understand quantitative relationships. 
• Analyze change in various contexts. 
• Relationship between algebraic thinking and other aspects of math.

Geometry and measurement

• Geometry and measurement (as an umbrella term).
• Analyze characteristics and properties of two- and three-dimensional geometric shapes and develop mathematical arguments about geometric relationships.
• Specify locations and describe spatial relationships using coordinate geometry and other representational systems.
• Apply transformations and use symmetry to analyze mathematical situations.
• Use visualization, spatial reasoning, and geometric modeling to solve problems.
• Understand measurable attributes of objects and the units, systems, and processes of measurement.
• Apply appropriate techniques, tools, and formulas to determine measurements.
• Relationship between geometry and measurement and other aspects of math.

Data analysis and probability

• Data analysis and probability (as an umbrella term).
• Formulate questions that can be addressed with data and collect, organize, and display relevant data to answer them.
• Select and use appropriate statistical methods to analyze data.
• Develop and evaluate inferences and predictions that are based on data.
• Understand and apply basic concepts of probability. 
• Relationship between data analysis and probability and other aspects of math.

NCTQ also evaluated whether the tests address instruction for specific student groups: English learners, students struggling with math, and advanced math students. Teachers may need to adapt or differentiate instruction to meet students’ specific needs. Five tests address students struggling with math, one addresses teaching English learners and multilingual learners, and none address advanced math students. 

The analysis supporting a range of learners does not affect the overall rating of the test. Instead, NCTQ collected this information so that state leaders can identify the tests that best address the state’s needs.

Mathematical Teaching Practices (unrated)

The National Council of Teachers of Mathematics (NCTM) identifies eight Mathematics Teaching Practices as those that “research indicates need to be consistent components of every mathematics lesson.”¹⁹ Based on input from NCTQ’s expert advisory panel, we analyzed whether the licensure tests address those practices. Eighteen tests (of the 30 total) include a range from two to all eight instructional practices. While we report on whether the tests address these practices, this information does not factor into the licensure tests’ ratings. 

The Mathematics Teaching Practices are:

• Establish mathematics goals to focus learning. Effective teaching of mathematics establishes clear goals for the mathematics that students are learning, situates goals within learning progressions, and uses the goals to guide instructional decisions. 
• Implement tasks that promote reasoning and problem solving. Effective teaching of mathematics engages students in solving and discussing tasks that promote mathematical reasoning and problem solving and allow multiple entry points and varied solution strategies. 
• Use and connect mathematical representations. Effective teaching of mathematics engages students in making connections among mathematical representations to deepen understanding of mathematics concepts and procedures and as tools for problem solving. 
• Facilitate meaningful mathematical discourse. Effective teaching of mathematics facilitates discourse among students to build shared understanding of mathematical ideas by analyzing and comparing student approaches and arguments. 
• Pose purposeful questions. Effective teaching of mathematics uses purposeful questions to assess and advance students’ reasoning and sense making about important mathematical ideas and relationships. 
• Build procedural fluency from conceptual understanding. Effective teaching of mathematics builds fluency with procedures on a foundation of conceptual understanding so that students, over time, become skillful in using procedures flexibly as they solve contextual and mathematical problems. 
• Support productive struggle in learning mathematics. Effective teaching of mathematics consistently provides students, individually and collectively, with opportunities and supports to engage in productive struggle as they grapple with mathematical ideas and relationships. 
• Elicit and use evidence of student thinking. Effective teaching of mathematics uses evidence of student thinking to assess progress toward mathematical understanding and to adjust instruction continually in ways that support and extend learning.

Inquiry and direct Instruction (unrated)

While different research studies may favor either more student-led instructional approaches (e.g., inquiry or project-based learning) or more teacher-led approaches (e.g., direct or explicit instruction) [link to math literature review], the field tends to view both approaches to math instruction as worthwhile. NCTQ’s expert advisory panel agreed that both inquiry and explicit instruction are necessary for effective math instruction. 

Based on input from the expert advisory panel, the analysts applied the following definitions in their analysis:

• Inquiry, discovery, or project-based learning:

• • Inquiry-based learning: a process of discovering new causal relations, with the learner formulating hypotheses and testing them by conducting experiments and/or making observations.²⁰
  • Discovery learning: an action embedded within inquiry-based learning.
  • Project-based learning: a teaching method in which students gain knowledge and skills by working for an extended period of time to investigate and respond to an authentic, engaging, and complex question, problem, or challenge.

• Direct or explicit instruction: 

• • Direct instruction: a teaching method that is structured, sequenced, and led by a teacher, and/or the presentation of academic content to students by the teacher, such as in a lecture or demonstration. In other words, teachers are “directing” the instructional process or instruction is being “directed” at students.²¹
  • Explicit instruction: A teaching method wherein the teacher tells students what they need to do using direct explanation along with sharing and modeling new knowledge.²² Analysts also looked for evidence of explicit strategy instruction, a teaching method that uses visuals and contexts to reveal mathematical relationships that illustrate how and why a strategy or method works.²³ 

While tests’ attention to instructional approaches does not factor into the rating, NCTQ provides the information about these instructional approaches to facilitate states in selecting exams that align with their standards and instructional materials. This analysis found that 16 tests address any elements of inquiry/discovery/project-based learning, and 16 tests address any elements of direct/explicit instruction. Notably, all but one test that addressed inquiry/discovery/project-based learning also addressed direct/explicit instruction, suggesting that tests typically do not promote one instructional approach over another.