Technical Assistance

A program’s effectiveness is judged by whether it produces positive outcomes for participants, with the randomized experiment being the gold standard for determining the efficacy of an intervention. However, there are many ways to evaluate program effectiveness, including design-based and implementation methods. The intervention implemented usually differs—to a greater or lesser degree—from the program that was designed, making it unclear whether unfavorable results are due to an ineffective program model or implementation failure.

The objective of this session is to enable researchers to use scenario-based questionnaires in the study and assessment of teaching knowledge and practice. Presenters show examples of items from 15 instruments developed to study teachers’ decisions in instructional situations, their recognition of specific norms of mathematics instruction, and their dispositions toward professional obligations of teaching. After participants peruse the items and instruments, they are walked through the steps taken by presenters to develop, pilot, revise, and implement this set of instruments.

This session provides applied training in the principles and practice of designing and analyzing studies to address questions of moderation and mediation. Presenters outline the motivation and critical value of complementing “what works”-type questions with explanatory investigations that address for whom a program works, under what conditions it works, and how it works for different groups. A practical outline of the concepts of moderation and mediation is presented through several STEM case studies.

This session provides an accessible introduction to the principles and practice of measuring latent traits in STEM studies as well as their implementation in common statistical software. Presenters first outline the motivation for latent traits and the practical challenges to measuring them in STEM studies. They then introduce case studies focused on measuring several different types of latent constructs associated with STEM education in different contextual settings.

The purpose of this session is to train researchers in how to plan efficient and effective studies in multilevel settings (e.g., students nested within classrooms and schools) using several common designs.