AIR Methodology Webinar Series Designed for the DRK-12 Program


Systematic Literature Reviews and Meta-Analytic Techniques

  • Webinar 1: An Introduction to Systematic Literature Reviews (September 3, 2020)
    Dr. Emily Tanner-Smith from the University of Oregon and Dr. Jeff Valentine from the University of Louisville will introduce the logic underlying systematic reviews. They will then share common challenges and provide practical advice for identifying the kinds of research questions that are suitable for systematic review, developing and assessing literature search strategies, and evaluating the credibility of the studies included in a review. Tools and resources will be shared to help the DRK-12 community apply those approaches.
    Slides | Webinar Recording | Chat Archive
  • Webinar 2: An Introduction to Meta-Analytic Techniques (held September 28, 2020) 
    In the second part of this series, Drs. Tanner-Smith and Valentine will discuss how researchers can use meta-analytic procedures to combine quantitative data from multiple studies, statistically synthesize findings, and interpret meta-analytic results. They will discuss model choice in meta-analysis, describe the basics of computing and meta-analyzing study effects, illustrate how dependencies in effect sizes can be addressed, and discuss how publication bias tests are used (and misused).
    Slides | Webinar Recording | Chat Archive

Social Network Analysis

  • Webinar 1: Social Network Analysis - An Introduction (March 26, 2020)
    Researchers Kyle Fagan, PhD, and Ben Kalina from the American Institutes for Research will provide an introduction to social network analysis, including an overview of key considerations for datacollection, analysis, and network visualization. They will review the kinds of research questions that can be answered with social network analysis, present the merits and limitations of this method, and share tools and resources to help the DRK-12 community apply this method to their projects.
    Slides | Webinar Recording
  • Webinar 2: Applying Social Network Analysis to STEM Education Research (April 20, 2020)
    In the second part of this series, a panel of two STEM education researchers, Dr. Maria Gonzalez-Howard from the University of Texas at Austin and Dr. Susan Yonezawa from the University of California San Diego, will each offer an overview of their research, including the STEM research questions studied using social network analysis and findings identified as a result. Dr. Gonzalez-Howard will discuss her research that uses social network analysis to explore patterns that occurs between classroom members when they engage in the scientific practice of argumentation. Dr. Yonezawa will discuss her research on the San Diego Math Network, and how she measured the impact of the Network on its members and tracked the health of the network over time. The webinar will include a facilitated discussion on challenges panelists faced when using this method and the strategies they applied to overcome these challenges.
    Slides | Webinar Recording

 Measuring Program Implementation

  • Webinar 1: Measuring Program Implementation (October 1, 2019) 
    Dr. Hulleman reviewed the importance of developing a logic model and using systematic measurement approaches to examine program implementation. He reviewed the merits and limitations of different methods for measuring implementation in research and development projects and shared tools and resources to help the DRK-12 community develop and refine their implementation measurement plans. A preview of Webinar 2 encouraged participants to return for part two of the discussion, during which a DRK-12 principal investigator (PI) or co-PI served as a discussant to share an authentic example of approaches and challenges for measuring implementation in his/her project.
    Slides | Webinar Recording
  • Webinar 2: Developing a Program Implementation Measurement Framework (October 10, 2019)
    In the second part of this series, Dr. Hulleman demonstrated how to develop an implementation measurement framework, aligned to a program logic model. The measurement framework provided a foundation for systematically examining implementation. Dr. Hulleman discussed practical considerations, such as reporting burden and availability and quality of data. Then he invited the DRK-12 discussant to share challenges encountered in measuring program implementation and receive feedback on how best to identify appropriate measurement indicators and thresholds for the key components of programs they are studying.
    Slides | Webinar Recording

Evidence Standards for Evaluating Math and Science Programs (March 20, 2019)
What are evidence standards for evaluating math and science programs? Join experts from the American Institutes for Research (AIR) for an engaging webinar about evidence standards. Presenters will discuss the Every Student Succeeds Act (ESSA) Tiers of Evidence and the What Works Clearinghouse (WWC) evidence rating system and how these evidence standards relate to evaluating math and science programs. The presenters will discuss how participants can use these resources to strengthen their existing and proposed study designs. This webinar is designed to be interactive and there will be time for audience participation and questions.
Webinar Recording  | Resources: