Assessment

The Dimensionality of the Epistemic Orientation Survey and Longitudinal Measurement Invariance for the Short Form of EOS (EOS-SF)

The dimensionality of the epistemic orientation survey (EOS) was examined across four occasions with item factor analysis (IFA). Because of an emphasis on the knowledge generation of epistemic orientation (EO), four factors were selected and built into a short form of EOS (EOS-SF) including knowledge generation, knowledge replication, epistemic nature of knowledge, and student ability. To track the stability of the factor structure for each factor of EOS-SF, longitudinal invariance models were conducted. Partial measurement invariance was obtained for each of the four factors of EOS-SF.

Author/Presenter

Chenchen Ding

Gavin Fulmer

Lesa Hoffman

Brian Hand

Jee K. Suh

Lead Organization(s)
Year
2023
Short Description

The dimensionality of the epistemic orientation survey (EOS) was examined across four occasions with item factor analysis (IFA). Because of an emphasis on the knowledge generation of epistemic orientation (EO), four factors were selected and built into a short form of EOS (EOS-SF) including knowledge generation, knowledge replication, epistemic nature of knowledge, and student ability.

Uncovering Middle School CS Students’ Understanding of Variables and Control Structures: A Cognitive Think-Aloud Approach

This poster presents findings on middle school students’ understanding of core computer science (CS) concepts, such as variables and control structures, using cognitive think-aloud interviews with eight students. Each student worked on 16-22 formative assessment tasks designed to assess understanding on the ‘Algorithms and Programming’ middle school CS standards. Our study describes students’ interpretations of the CS concepts and discusses potential factors influencing student interpretations. Significance and next steps are described.

Author/Presenter

Hui Yang

Satabdi Basu

Daisy Rutstein

Arif Rachmatullah

Carol Tate

Christopher Ortiz

Eliese Rulifson

Lead Organization(s)
Year
2023
Short Description

This poster presents findings on middle school students’ understanding of core computer science (CS) concepts, such as variables and control structures, using cognitive think-aloud interviews with eight students.

Exploring Middle School Students’ Understanding of Algorithms Using Standards-aligned Formative Assessments: Teacher and Researcher Perspectives

‘Algorithms’ is a core CS concept included in the K-12 CS standards, yet student challenges with understanding different aspects of algorithms are still not well documented, especially for younger students. This paper describes an approach to decompose the broad middle-school ‘algorithms’ standard into finer grained learning targets, develop formative assessment tasks aligned with the learning targets, and use the tasks to explore student understanding of, and challenges with, the various aspects of the standard.

Author/Presenter

Satabdi Basu

Daisy Rutstein

Carol Tate

Arif Rachmatullah

Hui Yang

Christopher Ortiz

Lead Organization(s)
Year
2023
Short Description

‘Algorithms’ is a core CS concept included in the K-12 CS standards, yet student challenges with understanding different aspects of algorithms are still not well documented, especially for younger students. This paper describes an approach to decompose the broad middle-school ‘algorithms’ standard into finer grained learning targets, develop formative assessment tasks aligned with the learning targets, and use the tasks to explore student understanding of, and challenges with, the various aspects of the standard.

Designing Standards-aligned Formative Assessments to Explore Middle School Students’ Understanding of Algorithms

'Algorithms’ is a core CS concept included in the K-12 CS learning standards, yet student challenges with understanding and using algorithms are still not well documented. This paper describes an approach to decompose the broad middle-school ‘algorithms’ standard into finer grained learning targets, develop formative assessment tasks aligned with the fine-grained learning targets, and use the tasks to explore student understanding of and challenges with the various concepts underlying the standard.

Author/Presenter

Satabdi Basu

Daisy Wise Rutstein

Arif Rachmatullah

Carol Tate

Hui Yang

Christopher Ortiz

Lead Organization(s)
Year
2023
Short Description

'Algorithms’ is a core CS concept included in the K-12 CS learning standards, yet student challenges with understanding and using algorithms are still not well documented. This paper describes an approach to decompose the broad middle-school ‘algorithms’ standard into finer grained learning targets, develop formative assessment tasks aligned with the fine-grained learning targets, and use the tasks to explore student understanding of and challenges with the various concepts underlying the standard.

Technology-based Innovative Assessment

This section presents an overview of critical developments in technology-driven, classroom-based innovative assessment practices. It uses a framework organized around cognitive constructs, assessment functionality, and automaticity to review the technological developments of innovative assessments and identify how they have been advanced to meet researcher and practitioner needs.

Author/Presenter

Xiaoming Zhai

Eric Wiebe

Year
2023
Short Description

This section presents an overview of critical developments in technology-driven, classroom-based innovative assessment practices. It uses a framework organized around cognitive constructs, assessment functionality, and automaticity to review the technological developments of innovative assessments and identify how they have been advanced to meet researcher and practitioner needs.

Technology-based Innovative Assessment

This section presents an overview of critical developments in technology-driven, classroom-based innovative assessment practices. It uses a framework organized around cognitive constructs, assessment functionality, and automaticity to review the technological developments of innovative assessments and identify how they have been advanced to meet researcher and practitioner needs.

Author/Presenter

Xiaoming Zhai

Eric Wiebe

Year
2023
Short Description

This section presents an overview of critical developments in technology-driven, classroom-based innovative assessment practices. It uses a framework organized around cognitive constructs, assessment functionality, and automaticity to review the technological developments of innovative assessments and identify how they have been advanced to meet researcher and practitioner needs.

Technology-based Innovative Assessment

This section presents an overview of critical developments in technology-driven, classroom-based innovative assessment practices. It uses a framework organized around cognitive constructs, assessment functionality, and automaticity to review the technological developments of innovative assessments and identify how they have been advanced to meet researcher and practitioner needs.

Author/Presenter

Xiaoming Zhai

Eric Wiebe

Year
2023
Short Description

This section presents an overview of critical developments in technology-driven, classroom-based innovative assessment practices. It uses a framework organized around cognitive constructs, assessment functionality, and automaticity to review the technological developments of innovative assessments and identify how they have been advanced to meet researcher and practitioner needs.

Technology-based Innovative Assessment

This section presents an overview of critical developments in technology-driven, classroom-based innovative assessment practices. It uses a framework organized around cognitive constructs, assessment functionality, and automaticity to review the technological developments of innovative assessments and identify how they have been advanced to meet researcher and practitioner needs.

Author/Presenter

Xiaoming Zhai

Eric Wiebe

Year
2023
Short Description

This section presents an overview of critical developments in technology-driven, classroom-based innovative assessment practices. It uses a framework organized around cognitive constructs, assessment functionality, and automaticity to review the technological developments of innovative assessments and identify how they have been advanced to meet researcher and practitioner needs.

Iterative Cognitive Interview Design to Uncover Children’s Spatial Reasoning

Cognitive interviews play an important role in articulating the intended construct of educational assessments. This paper describes the iterative development of protocols for cognitive interviews with kindergarten through second-grade children to understand how their spatial reasoning skill development aligns with intended constructs. We describe the procedures used to gather evidence of construct relevance and improved alignment to task-based interview items through multiple pilot rounds before conducting cognitive interviews.

Author/Presenter

Elizabeth R. Thomas

Robyn K. Pinilla

Leanne R. Ketterlin-Geller

Cassandra Hatfield

Year
2023
Short Description

Cognitive interviews play an important role in articulating the intended construct of educational assessments. This paper describes the iterative development of protocols for cognitive interviews with kindergarten through second-grade children to understand how their spatial reasoning skill development aligns with intended constructs.

Connecting Classroom Assessment with Learning Goals and Instruction Through Theories of Learning

In this report section, we discuss the importance of aligning classroom assessments with learning goals and instructional practices to both shape and evaluate students’ learning opportunities. We describe a plausible solution for improving alignment by integrating theories of learning in the design of classroom assessments. We discuss ways in which the specification of theories of learning as learning progressions can improve alignment between classroom assessments and instruction by focusing on the content, task design, and data generated from classroom assessments.

Author/Presenter

Leanne R. Ketterlin-Geller

Christopher J. Harris

Year
2023
Short Description

In this report section, we discuss the importance of aligning classroom assessments with learning goals and instructional practices to both shape and evaluate students’ learning opportunities.