Science

An Empirical Investigation of Neural Methods for Content Scoring of Science Explanations

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out.

Author/Presenter

Riordan, B., Bichler, S., Bradford, A., King Chen, J., Wiley, K., Gerard, L., & Linn, M.C.

Short Description

We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.

An Empirical Investigation of Neural Methods for Content Scoring of Science Explanations

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out.

Author/Presenter

Riordan, B., Bichler, S., Bradford, A., King Chen, J., Wiley, K., Gerard, L., & Linn, M.C.

Short Description

We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.

How Does an Adaptive Dialog Based on Natural Language Processing Impact Students from Distinct Language Backgrounds?

This study takes advantage of advances in Natural Language Processing (NLP) to build an idea detection model that can identify ideas grounded in students’ linguistic experiences. We designed adaptive, interactive dialogs for four explanation items using the NLP idea detection model and investigated whether they similarly support students from distinct language backgrounds. The curriculum, assessments, and scoring rubrics were informed by the Knowledge Integration (KI) pedagogy.

Author/Presenter

Holtman, M., Gerard, L., Li, W., Linn, M.C., Steimel, K., & Riordan, B. 

Year
2023
Short Description

This study takes advantage of advances in Natural Language Processing (NLP) to build an idea detection model that can identify ideas grounded in students’ linguistic experiences.

How Does an Adaptive Dialog Based on Natural Language Processing Impact Students from Distinct Language Backgrounds?

This study takes advantage of advances in Natural Language Processing (NLP) to build an idea detection model that can identify ideas grounded in students’ linguistic experiences. We designed adaptive, interactive dialogs for four explanation items using the NLP idea detection model and investigated whether they similarly support students from distinct language backgrounds. The curriculum, assessments, and scoring rubrics were informed by the Knowledge Integration (KI) pedagogy.

Author/Presenter

Holtman, M., Gerard, L., Li, W., Linn, M.C., Steimel, K., & Riordan, B. 

Year
2023
Short Description

This study takes advantage of advances in Natural Language Processing (NLP) to build an idea detection model that can identify ideas grounded in students’ linguistic experiences.

Development of a Questionnaire on Teachers' Beliefs, Preparedness, and Instructional Practices for Teaching NGSS Science with Multilingual Learners

The limited availability of research instruments that reflect the vision of the Next Generation Science Standards (NGSS) restricts the field's understanding of whether and how teachers are making instructional shifts called for by the standards. The need for such instruments is particularly urgent with teachers of multilingual learners (MLs), who are called on to make shifts in how they think about and enact instruction related to both science and language.

Author/Presenter

Scott E. Grapin

Courtney Plumley

Eric Banilower

Alycia J. Sterenberg Mahon

Laura Craven

Kristen Malzahn

Joan Pasley

Abigail Schwenger

Alison Haas

Okhee Lee

Lead Organization(s)
Year
2024
Short Description

The limited availability of research instruments that reflect the vision of the Next Generation Science Standards (NGSS) restricts the field's understanding of whether and how teachers are making instructional shifts called for by the standards. The need for such instruments is particularly urgent with teachers of multilingual learners (MLs), who are called on to make shifts in how they think about and enact instruction related to both science and language. The purpose of this study was to develop and gather validity evidence for a questionnaire that measures elementary teachers' beliefs, preparedness, and instructional practices for teaching NGSS science with MLs.

Contextual Resources Supporting the Co-evolution of Teachers' Collective Inquiry and Classroom Practice After the Grant Ended

We explored how various contextual resources accumulated over multiple years operated together to facilitate a team of high school teachers' sustained and agentive learning after a 4-year research–practice partnership (RPP) grant concluded. Specifically, we examined constellations of resources that promoted the co-evolution of the teachers' collective inquiry in the professional learning community (PLC) and classroom instruction, focused on supporting students' scientific explanations.

Author/Presenter

Soo-Yean Shim

Jessica Thompson

Lead Organization(s)
Year
2024
Short Description

We explored how various contextual resources accumulated over multiple years operated together to facilitate a team of high school teachers' sustained and agentive learning after a 4-year research–practice partnership (RPP) grant concluded. Specifically, we examined constellations of resources that promoted the co-evolution of the teachers' collective inquiry in the professional learning community (PLC) and classroom instruction, focused on supporting students' scientific explanations.

Understanding Variation in Integrated STEM Practice as Measured by the STEM Observation Protocol (STEM-OP)

To better understand integrated STEM education, this work explored scores on the STEM Observation Protocol (STEM-OP), a newly developed observation protocol for use in K-12 science and engineering classrooms. The goals of this work were to better understand how integrated STEM might look throughout an integrated STEM unit and identify limitations of the instrument when examining daily scores and full unit implementation scores. The work takes a mixed methods approach to first examine what scores may be typically seen with daily and unit implementations.

Author/Presenter

Emily A. Dare

Elizabeth A. Ring-Whalen

Year
2024
Short Description

To better understand integrated STEM education, this work explored scores on the STEM Observation Protocol (STEM-OP), a newly developed observation protocol for use in K-12 science and engineering classrooms. The goals of this work were to better understand how integrated STEM might look throughout an integrated STEM unit and identify limitations of the instrument when examining daily scores and full unit implementation scores.

Understanding Variation in Integrated STEM Practice as Measured by the STEM Observation Protocol (STEM-OP)

To better understand integrated STEM education, this work explored scores on the STEM Observation Protocol (STEM-OP), a newly developed observation protocol for use in K-12 science and engineering classrooms. The goals of this work were to better understand how integrated STEM might look throughout an integrated STEM unit and identify limitations of the instrument when examining daily scores and full unit implementation scores. The work takes a mixed methods approach to first examine what scores may be typically seen with daily and unit implementations.

Author/Presenter

Emily A. Dare

Elizabeth A. Ring-Whalen

Year
2024
Short Description

To better understand integrated STEM education, this work explored scores on the STEM Observation Protocol (STEM-OP), a newly developed observation protocol for use in K-12 science and engineering classrooms. The goals of this work were to better understand how integrated STEM might look throughout an integrated STEM unit and identify limitations of the instrument when examining daily scores and full unit implementation scores.

Novice Elementary Teachers’ Science Teaching: Instructional Planning and Discourse

Teaching science to elementary school students is complex, and teacher preparation programs must support preservice teachers’ learning to communicate science content and practices with children. Science teachers bring the experiences they have as students to their teacher preparation, and these experiences may support or conflict with what they are learning about effective teaching. In this study, we present stories of two first-year teachers who attended a STEM-focused teacher preparation program in the U.S.

Author/Presenter

Sarah J. Carrier

James Minogue

Aimee B. Fraulo

Year
2024
Short Description

Teaching science to elementary school students is complex, and teacher preparation programs must support preservice teachers’ learning to communicate science content and practices with children. Science teachers bring the experiences they have as students to their teacher preparation, and these experiences may support or conflict with what they are learning about effective teaching. In this study, we present stories of two first-year teachers who attended a STEM-focused teacher preparation program in the U.S.

Engaging Elementary Students in Science Practice: Strategies for Helping Children Plan Investigations

This article presents a tool that teachers can use to support children in planning science investigations. Using an extended example from a second-grade investigation into seed dispersal, we describe strategies for structuring conversations that anchor investigations in phenomena and provide opportunities for students to be involved in making decisions about what materials to use in an investigation, how to use materials, and what to look for or count as evidence.

Author/Presenter

Annabel Stoler

Eve Manz

Lead Organization(s)
Year
2024
Short Description

This article presents a tool that teachers can use to support children in planning science investigations. Using an extended example from a second-grade investigation into seed dispersal, we describe strategies for structuring conversations that anchor investigations in phenomena and provide opportunities for students to be involved in making decisions about what materials to use in an investigation, how to use materials, and what to look for or count as evidence.