Educational Technology

Multidimensional Science Assessment: Design Challenges and Technology Affordances

Contemporary views on what students should learn increasingly emphasize that students need to acquire more than a base of knowledge; they need to acquire the skills and abilities to use such knowledge in dynamic and flexible ways. To be most effective, learning environments need assessments that are aligned to these perspectives. Using a principled design framework can help guide assessment development toward such targets. Even when using a framework, however, thorny design challenges may arise.

Author/Presenter

Brian D. Gane

Diksha Gaur

Samuel Arnold

Daniel Damelin

Lead Organization(s)
Year
2024
Short Description

In this paper, we describe three challenges (conflict between multiple dimensions of science proficiency, authentic data, and grade-appropriate graphing tools) that we faced when designing for a specific Next Generation Science Standard, and the theoretical and design principles that guided us as we ideated design solutions. Through these designs we maintained alignment to our multidimensional assessment targets, a critical component of our larger assessment validity argument.

Multidimensional Science Assessment: Design Challenges and Technology Affordances

Contemporary views on what students should learn increasingly emphasize that students need to acquire more than a base of knowledge; they need to acquire the skills and abilities to use such knowledge in dynamic and flexible ways. To be most effective, learning environments need assessments that are aligned to these perspectives. Using a principled design framework can help guide assessment development toward such targets. Even when using a framework, however, thorny design challenges may arise.

Author/Presenter

Brian D. Gane

Diksha Gaur

Samuel Arnold

Daniel Damelin

Lead Organization(s)
Year
2024
Short Description

In this paper, we describe three challenges (conflict between multiple dimensions of science proficiency, authentic data, and grade-appropriate graphing tools) that we faced when designing for a specific Next Generation Science Standard, and the theoretical and design principles that guided us as we ideated design solutions. Through these designs we maintained alignment to our multidimensional assessment targets, a critical component of our larger assessment validity argument.

Can Generative AI and ChatGPT Outperform Humans on Cognitive-Demanding Problem-Solving Tasks in Science?

This study aimed to examine an assumption regarding whether generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We examine the performance of ChatGPT and GPT-4 on NAEP science assessments and compare their performance to students by cognitive demands of the items. Fifty-four 2019 NAEP science assessment tasks were coded by content experts using a two-dimensional cognitive load framework, including task cognitive complexity and dimensionality.

Author/Presenter

Xiaoming Zhai

Matthew Nyaaba

Wenchao Ma

Lead Organization(s)
Year
2024
Short Description

This study aimed to examine an assumption regarding whether generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We examine the performance of ChatGPT and GPT-4 on NAEP science assessments and compare their performance to students by cognitive demands of the items.

Can Generative AI and ChatGPT Outperform Humans on Cognitive-Demanding Problem-Solving Tasks in Science?

This study aimed to examine an assumption regarding whether generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We examine the performance of ChatGPT and GPT-4 on NAEP science assessments and compare their performance to students by cognitive demands of the items. Fifty-four 2019 NAEP science assessment tasks were coded by content experts using a two-dimensional cognitive load framework, including task cognitive complexity and dimensionality.

Author/Presenter

Xiaoming Zhai

Matthew Nyaaba

Wenchao Ma

Lead Organization(s)
Year
2024
Short Description

This study aimed to examine an assumption regarding whether generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We examine the performance of ChatGPT and GPT-4 on NAEP science assessments and compare their performance to students by cognitive demands of the items.

Teachers’ Use and Adaptation of a Model-based Climate Curriculum: A Three-Year Longitudinal Study

Foregrounding climate education in formal science learning environments provides students with opportunities to develop critical climate-related knowledge and skills. However, research has shown many challenges to teaching and learning about Earth’s climate and global climate change (GCC). This longitudinal study aims to establish how secondary science teachers, over time, implement model-based climate curricula in support of students’ climate and GCC education by utilizing EzGCM. The model (EzGCM) is a data-driven, computer-based climate modeling tool use to explore global climate data.

Author/Presenter

Kimberly Carroll Steward

David Gosselin

Devarati Bhattacharya

Mark Chandler

Cory T. Forbes

Year
2024
Short Description

Foregrounding climate education in formal science learning environments provides students with opportunities to develop critical climate-related knowledge and skills. However, research has shown many challenges to teaching and learning about Earth’s climate and global climate change (GCC). This longitudinal study aims to establish how secondary science teachers, over time, implement model-based climate curricula in support of students’ climate and GCC education by utilizing EzGCM. The model (EzGCM) is a data-driven, computer-based climate modeling tool use to explore global climate data.

Teachers’ Use and Adaptation of a Model-based Climate Curriculum: A Three-Year Longitudinal Study

Foregrounding climate education in formal science learning environments provides students with opportunities to develop critical climate-related knowledge and skills. However, research has shown many challenges to teaching and learning about Earth’s climate and global climate change (GCC). This longitudinal study aims to establish how secondary science teachers, over time, implement model-based climate curricula in support of students’ climate and GCC education by utilizing EzGCM. The model (EzGCM) is a data-driven, computer-based climate modeling tool use to explore global climate data.

Author/Presenter

Kimberly Carroll Steward

David Gosselin

Devarati Bhattacharya

Mark Chandler

Cory T. Forbes

Year
2024
Short Description

Foregrounding climate education in formal science learning environments provides students with opportunities to develop critical climate-related knowledge and skills. However, research has shown many challenges to teaching and learning about Earth’s climate and global climate change (GCC). This longitudinal study aims to establish how secondary science teachers, over time, implement model-based climate curricula in support of students’ climate and GCC education by utilizing EzGCM. The model (EzGCM) is a data-driven, computer-based climate modeling tool use to explore global climate data.

Supporting Secondary Students’ Understanding of Earth’s Climate System and Global Climate Change Using EzGCM: A Cross-Sectional Study

Global climate change (GCC) is one of the greatest challenges of our age and a highly significant socio-scientific issue (SSI). Developing secondary students’ understanding about the Earth’s climate and GCC is critical for empowering future citizens and a key focus of the Next Generation Science Standards (NGSS Lead States, 2013).

Author/Presenter

Silvia-Jessica Mostacedo-Marasovic

Amanda A. Olsen

Cory T. Forbes

Year
2023
Short Description

Global climate change (GCC) is one of the greatest challenges of our age and a highly significant socio-scientific issue (SSI). Developing secondary students’ understanding about the Earth’s climate and GCC is critical for empowering future citizens and a key focus of the Next Generation Science Standards. In this cross-sectional study, we investigated secondary students’ evidence-based reasoning about GCC grounded in a curricular intervention involving the use of a data-driven, computer-based global climate model—EzGCM—over 3 years with four teachers who adapted the module in their own courses.

Supporting Secondary Students’ Understanding of Earth’s Climate System and Global Climate Change Using EzGCM: A Cross-Sectional Study

Global climate change (GCC) is one of the greatest challenges of our age and a highly significant socio-scientific issue (SSI). Developing secondary students’ understanding about the Earth’s climate and GCC is critical for empowering future citizens and a key focus of the Next Generation Science Standards (NGSS Lead States, 2013).

Author/Presenter

Silvia-Jessica Mostacedo-Marasovic

Amanda A. Olsen

Cory T. Forbes

Year
2023
Short Description

Global climate change (GCC) is one of the greatest challenges of our age and a highly significant socio-scientific issue (SSI). Developing secondary students’ understanding about the Earth’s climate and GCC is critical for empowering future citizens and a key focus of the Next Generation Science Standards. In this cross-sectional study, we investigated secondary students’ evidence-based reasoning about GCC grounded in a curricular intervention involving the use of a data-driven, computer-based global climate model—EzGCM—over 3 years with four teachers who adapted the module in their own courses.

Student Outcomes of Teaching About Socio-scientific Issues in Secondary Science Classrooms: Applications of EzGCM

Science education literature has highlighted socio-scientific issues (SSI) as an effective pedagogy for teaching science in a social and political context. SSI links science education and real-world problems to engage students in real-world issues, making it ideal for teaching global climate change (GCC). Additionally, technological advances have created a unique opportunity for teaching climate by making previously inaccessible computer-based computational models and data visualizations accessible to the typical K-12 learning environment.

Author/Presenter

Kimberly Carroll Steward

David Gosselin

Mark Chandler

Cory T. Forbes

Year
2023
Short Description

Science education literature has highlighted socio-scientific issues (SSI) as an effective pedagogy for teaching science in a social and political context. SSI links science education and real-world problems to engage students in real-world issues, making it ideal for teaching global climate change (GCC). Additionally, technological advances have created a unique opportunity for teaching climate by making previously inaccessible computer-based computational models and data visualizations accessible to the typical K-12 learning environment. Here, we present the findings from the 2020–2021 school year pre-/post-implementation of a 3-week, model-based climate education curriculum module (EzGCM).

Student Outcomes of Teaching About Socio-scientific Issues in Secondary Science Classrooms: Applications of EzGCM

Science education literature has highlighted socio-scientific issues (SSI) as an effective pedagogy for teaching science in a social and political context. SSI links science education and real-world problems to engage students in real-world issues, making it ideal for teaching global climate change (GCC). Additionally, technological advances have created a unique opportunity for teaching climate by making previously inaccessible computer-based computational models and data visualizations accessible to the typical K-12 learning environment.

Author/Presenter

Kimberly Carroll Steward

David Gosselin

Mark Chandler

Cory T. Forbes

Year
2023
Short Description

Science education literature has highlighted socio-scientific issues (SSI) as an effective pedagogy for teaching science in a social and political context. SSI links science education and real-world problems to engage students in real-world issues, making it ideal for teaching global climate change (GCC). Additionally, technological advances have created a unique opportunity for teaching climate by making previously inaccessible computer-based computational models and data visualizations accessible to the typical K-12 learning environment. Here, we present the findings from the 2020–2021 school year pre-/post-implementation of a 3-week, model-based climate education curriculum module (EzGCM).