Formative Assessment in Mathematics: Current Status and Guidelines for Future Developments

In response to the increased use of formative assessment practices in California's PK-12 mathematics classrooms, the University of California-Davis and Claremont Graduate University join efforts to investigate what formative assessments are in use, how practitioners and students are utilizing these assessments, and how they impact performance on summative state assessments. Four research questions guide this study: (1) What is the current status of formative assessment use in California's PK & K, elementary, middle, and high schools? (2) What are the content and psychometric properties of the current formative assessments? (3) What impact do formative assessment outcomes have on student academic achievement as measured by summative assessments? and (4) Does the use of formative assessment differentially impact performance of subgroups of students, such as ELLs, students of color, economically disadvantaged students, and students with disabilities?

The study consists of three phases and employs a mixed-methods approach to answer the four research questions. To answer research question 1, Phase I administers an online survey to all school principals in California. Frequency and descriptive analyses of data are conducted to obtain an accurate view of the current status of formative assessments. Software is used to qualitatively analyze responses to open-ended questions. Frequencies of categorical responses and mean and standard deviations of Likert-type responses are compared across respondents with different backgrounds and schools with different characteristics. Additional interviews with test publishers, state directors of assessment and research experts will be conducted to round out the perspective. To address research question 2, Phase II conducts a series of psychometric and content analyses of existing assessments to ascertain validity, reliability, and accessibility of assessments. A total of 120 PK-12 schools with the strongest formative assessment instruments and the greatest formative assessment impact on summative assessments participate in these analyses. Sample sizes for this phase, representative of a four-grade span (2nd, 5th, 8th, and 10th grades), were determined through a power analysis for a minimum of 0.25 standard deviation detectable difference with a statistical power of 80% at a 0.05 type I error. Due to the study's focus on traditionally low-performing and underrepresented populations, a stratified random sampling approach in which subgroup identifiers are used as stratification variables is employed. To assess content properties, the study utilizes Webb's (1997) alignment methodology. Reliability of formative assessments is examined using the internal consistency approach (Cronbach alpha). Validity is analyzed through content- and criterion-related validity approaches, including a Multi-Traits/Multi-Methods approach. To answer research questions 3 and 4, a series of path models are created to explore the power of formative assessments in predicting student performance on the State's end-of-year assessments. In addition, an Ex Post Facto design is conducted to investigate whether one or more preexisting conditions have possibly caused subsequent differences in the groups of participants. To document successful practices, the study's Phase III develops case studies of six schools with effective formative assessment practices and demonstrated academic growth using summative measures. Case studies include audiotaped focus group interviews with teachers and administrators, videotaped observations of teacher meetings, and classroom observations.

The main outcome of this study will be a set of research-informed and field-tested conclusions, recommendations, guidelines, and tools for the development and use of new or improved PK-12 mathematics formative assessments. Outcome tools include a blueprint and test specifications based on the California State Mathematics Content Standards for 8th grade, and a field-tested formative assessment prototype focused on key algebra concepts and skills for that grade level. The Center for the Study of Evaluation at the University of California-Los Angeles serves as the project external evaluator.