Technical Assistance

Fidelity of Implementation

Day
Fri

In this session, Hulleman discusses the advantages of assessing fidelity of implementation, elaborates a procedure for systematically assessing fidelity, and invites participants to discuss applications to their work.

Date/Time
-
Presenters

A program’s effectiveness is judged by whether it produces positive outcomes for participants, with the randomized experiment being the gold standard for determining the efficacy of an intervention. However, there are many ways to evaluate program effectiveness, including design-based and implementation methods. The intervention implemented usually differs—to a greater or lesser degree—from the program that was designed, making it unclear whether unfavorable results are due to an ineffective program model or implementation failure.

Using Scenario-based Questionnaires to Investigate Teaching Knowledge and Practice at Scale

Day
Thu

Presenters use illustrations from a project that used scenario-based, graphics-enhanced online questionnaires to introduce participants to the use of these techniques to study teaching knowledge and practice.

Date/Time
-

The objective of this session is to enable researchers to use scenario-based questionnaires in the study and assessment of teaching knowledge and practice. Presenters show examples of items from 15 instruments developed to study teachers’ decisions in instructional situations, their recognition of specific norms of mathematics instruction, and their dispositions toward professional obligations of teaching. After participants peruse the items and instruments, they are walked through the steps taken by presenters to develop, pilot, revise, and implement this set of instruments.

Designing and Analyzing Studies to Detect Moderation and Mediation

STEM Categorization
Day
Thu

This session provides practical training in the concepts and application of moderation and mediation within the context of STEM education.

Date/Time
-

This session provides applied training in the principles and practice of designing and analyzing studies to address questions of moderation and mediation. Presenters outline the motivation and critical value of complementing “what works”-type questions with explanatory investigations that address for whom a program works, under what conditions it works, and how it works for different groups. A practical outline of the concepts of moderation and mediation is presented through several STEM case studies.

Measuring Latent Constructs

Day
Thu

The session details the measurement of latent constructs and provides practical training on the use of confirmatory factor analysis and latent trait models.

Date/Time
-

This session provides an accessible introduction to the principles and practice of measuring latent traits in STEM studies as well as their implementation in common statistical software. Presenters first outline the motivation for latent traits and the practical challenges to measuring them in STEM studies. They then introduce case studies focused on measuring several different types of latent constructs associated with STEM education in different contextual settings.

Exploring Variation in Discourse and Interactional Analyses

STEM Categorization
Day
Thu

Presenters engage the audience in various theories and methods for discourse and interactional analysis, exploring the ways in which these approaches shape researchers’ claims.

Date/Time
-

Designing Multilevel Studies

STEM Categorization
Day
Wed

Using practical training and STEM study examples, presenters provide training on planning efficient and effective impact studies in multilevel settings.

Date/Time
-

The purpose of this session is to train researchers in how to plan efficient and effective studies in multilevel settings (e.g., students nested within classrooms and schools) using several common designs.