Teachers' Use of Standards-based Instructional Materials

This study explores the ways middle school mathematics teachers implement standards-based curriculum materials in urban schools. It takes the view that instructional materials are cultural tools and examines how teachers use these tools to plan and implement the curriculum in their classrooms. The study is using a mixed methods approach that combines surveys of teachers in 30 schools in the Newark Public Schools district and closer observations of teachers in selected case schools.

Lead Organization(s): 
Award Number: 
Funding Period: 
Sat, 09/01/2007 to Fri, 08/31/2012
Project Evaluator: 
Ashaki Coleman

Mathematics Instruction Using Decision Science and Engineering Tools

A collaboration among educators, engineers, and mathematicians in three universities, this project is creating, implementing, and evaluating a one-year curriculum for teaching a non-calculus, fourth-year high school mathematics course and accompanied assessment instruments. The curriculum will draw on decision-making tools that include but go well beyond linear programming, to enhance student mathematical competence (particularly solving multi-step problems), improve students' attitudes toward mathematics, and promote states' adoption of the curriculum (initially NC and MI).

Project Email: 
Award Number: 
Funding Period: 
Sat, 09/15/2007 to Tue, 08/31/2010
Project Evaluator: 
Dr. Shlomo S. Sawilowsky
Full Description: 

Mathematics INstruction using Decision Science and Engineering Tools (MINDSET) is a collaboration among educators, engineers, and mathematicians at three universities to create, implement, and evaluate a new curriculum and textbook to teach standard mathematics concepts using math-based decision-making tools for a non-calculus fourth-year mathematics curriculum that several states now require and others may require in the near future. MINDSET has three goals: (1) enhancement of students’ mathematical ability, especially their ability to formulate and solve multi-step problems and interpret results; (2) improvement in students’ attitude toward mathematics, especially those from underrepresented groups, thereby motivating them to study mathematics; and (3) adoption of the curriculum initially in North Carolina and Michigan, then in other states.

Using decision-making tools from Operations Research and Industrial Engineering, we will develop a fourthyear high school curriculum in mathematics and support materials to teach standard content. Through a multi-state, multi-school district assessment, we will determine if a statistically significant improvement in students’ mathematical ability—particularly in multi-step problem solving and interpretation of results—and in motivation and attitude toward mathematics has occurred. Participating teachers will receive professional training, help to create a knowledge-based online community for support, and in-person and online technical assistance. Through extensive data collection and analysis, we will determine if this infrastructure is sustainable and sufficiently flexible to be reproduced and used by others.

Quality Cyber-enabled, Engineering Education Professional Development to Support Teacher Change and Student Achievement (E2PD)

In this project, a video and audio network links elementary school teachers with researchers and educators at Purdue to form a community of practice dedicated to implementing engineering education at the elementary grades. The research plan includes identifying the attributes of face-to-face and cyber-enabled teacher professional development and community building that can transform teachers into master users and designers of engineering education for elementary learners.

Lead Organization(s): 
Award Number: 
Funding Period: 
Mon, 09/15/2008 to Tue, 08/31/2010
Project Evaluator: 
Rose Marra, University of Missouri-Columbia

Learning Progressions for Scientific Inquiry: A Model Implementation in the Context of Energy

The project has had three major areas of focus:  (1) Offering professional development to help elementary and 6th grade teachers become more responsive teachers, attending and responding to their students' ideas and reasoning; (2)  Developing web-based resources (both curriculum and case studies) to promote responsive teaching in science; and (3) research how both teachers and students progress in their ability to engage in science inquiry. 

Lead Organization(s): 
Award Number: 
Funding Period: 
Tue, 01/01/2008 to Mon, 12/31/2012
Project Evaluator: 
Lawrence Hall of Science

Chemistry Facets: Formative Assessment to Improve Student Understanding in Chemistry

This project implemented a facets-of-thinking perspective to design tools and practices to improve high school chemistry teachers' formative assessment practices. Goals are to identify and develop clusters of facets related to key chemistry concepts; develop assessment items; enhance the assessment system for administering items, reporting results, and providing teacher resource materials; develop teacher professional development and resource materials; and examine whether student learning in chemistry improves in classes that incorporate a facet-based assessment system.

Partner Organization(s): 
Award Number: 
Funding Period: 
Sat, 09/15/2007 to Wed, 08/31/2011
Project Evaluator: 
Heller Research Associates
Full Description: 

Supported by research on students' preconceptions, particularly in chemistry, and the need to build on the knowledge and skills that students bring to the classroom, this project implements a facets-of-thinking perspective for the improvement of formative assessment, learning, and instruction in high school chemistry. Its goals are: to identify and develop clusters of facets (students' ideas and understandings) related to key high school chemistry concepts; to develop assessment items that diagnose facets within each cluster; to enhance the existing web-based Diagnoser assessment system for administering items, reporting results, and providing teacher resource materials for interpreting and using the assessment data; to develop teacher professional development and resource materials to support their use of facet-based approaches in chemistry; and to examine whether student learning in chemistry improves in classes that incorporate a facet-based assessment system.

The proposed work builds on two previously NSF-funded projects focused on designing Diagnoser (ESI-0435727) in the area of physics and on assessment development to support the transition to complex science learning (REC-0129406). The work plan is organized in three strands: (1) Assessment Development, consisting of the development and validation of facet clusters related to the Atomic Structure of Matter and Changes in Matter and the development and validation of question sets related to each facet cluster, including their administration to chemistry classes; (2) Professional Development, through which materials will be produced for a teacher workshop focused on the assessment-for-learning cycle; and (3) Technology Development, to upgrade the Diagnoser authoring system and to include chemistry facets and assessments.

Anticipated products include: (1) 8-10 validated facet clusters related to the Atomic Structure of Matter and Changes in Matter; (2) 12-20 items per facet cluster that provide diagnostic information about student understanding in relation to the facet clusters; (3) additional instructional materials related to each facet cluster, including 1-3 questions to elicit inital student ideas, a developmental lesson to encourage students' exploration of new concepts, and 3-5 prescriptive lessons to address persistent problematic ideas; and (4) a publically-available web-based Diagnoser for chemistry (, including student assessments and instructional materials.

Integrating Computing Across the Curriculum (ICAC): Incorporating Technology into STEM Education Using XO Laptops

This project builds and tests applications tied to the school curriculum that integrate the sciences with mathematics, computational thinking, reading and writing in elementary schools. The investigative core of the project is to determine how to best integrate computing across the curriculum in such a way as to support STEM learning and lead more urban children to STEM career paths.

Project Email: 
Award Number: 
Funding Period: 
Sat, 08/01/2009 to Sun, 07/31/2011
Project Evaluator: 
Leslie Cooksy - Univ. of Delaware
Full Description: 

Computer access has opened an exciting new dimension for STEM education; however, if computers in the classroom are to realize their full potential as a tool for advancing STEM education, methods must be developed to allow them to serve as a bridge across the STEM disciplines. The goal of this 60-month multi-method, multi-disciplinary ICAC project is to develop and test a program to increase the number of students in the STEM pipeline by providing teachers and students with curricular training and skills to enhance STEM education in elementary schools. ICAC will be implemented in an urban and predominantly African American school system, since these schools traditionally lag behind in filling the STEM pipeline. Specifically, ICAC will increase computer proficiency (e.g., general usage and programming), science, and mathematics skills of teachers and 4th and 5th grade students, and inform parents about the opportunities available in STEM-centered careers for their children.

The Specific Aims of ICAC are to:

SA1. Conduct a formative assessment with teachers to determine the optimal intervention to ensure productive school, principal, teacher, and student participation.

SA2. Implement a structured intervention aimed at (1) teachers, (2) students, and (3) families that will enhance the students’ understanding of STEM fundamentals by incorporating laptops into an inquiry-based educational process.

SA3. Assess the effects of ICAC on:

a. Student STEM  engagement and performance.

b. Teacher and student computing specific confidence and utilization.

c. Student interest in technology and STEM careers.

d. Parents’ attitudes toward STEM careers and use of computers.

To enable us to complete the specific aims noted above, we have conducted a variety of project activities in Years 1-3. These include:

  1. Classroom observations at the two Year 1 pilot schools
  2. Project scaling to 6 schools in Year 2 and 10 schools in Year 3
  3. Semi-structured school administrator interviews in schools
  4. Professional development sessions for teachers
  5. Drafting of curriculum modules to be used in summer teacher institutes and for dissemination
  6. In-class demonstration of curriculum modules
  7. Scratch festivals each May
  8. Summer teacher institutes
  9. Student summer camps
  10. Surveying of teachers in summer institutes
  11. Surveying of teachers and students at the beginning and end of the school year
  12. Showcase event at end of student workshops

The specific ICAC activities for Years 2-5 include:

  • Professional development sessions (twice monthly for teachers), to integrate the ‘best practices’ from the program.
  • Working groups led by a grade-specific lead teacher. The lead teacher for each grade in each school will identify areas where assistance is needed and will gather the grade-specific cohort of teachers at their school once every two weeks for a meeting to discuss the progress made in addition to challenges to or successes in curricula development.  
  • ICAC staff and prior trained teachers will visit each class monthly during the year to assist the teachers and to evaluate specific challenges and opportunities for the use of XOs in that classroom.  
  • In class sessions at least once per month (most likely more often given feedback from Teacher Summer Institutes) to demonstrate lesson plans and assist teachers as they implement lesson plans.
  • ICAC staff will also hold a joint meeting of administrators of all target schools each year to assess program progress and challenges. 
  • Teacher Summer Institutes – scaled-up to teachers from the new schools each summer to provide training in how to incorporate computing into their curriculum.
  • Administrator sessions during the Teacher Summer Institutes; designed to provide insight into how the laptops can facilitate the education and comprehension of their students in all areas of the curriculum, discuss flexible models for physical classroom organization to facilitate student learning, and discussions related to how to optimize the use of computing to enhance STEM curricula in their schools.  Student Summer Computing Camps – designed to teach students computing concepts, make computing fun, and enhance their interest in STEM careers.  
  • ICAC will sponsor a yearly showcase event in Years 2-5 that provides opportunities for parents to learn more about technology skills their children are learning (e.g., career options in STEM areas, overview of ICAC, and summary of student projects). At this event, a yearly citywide competition among students also will be held that is an expanded version of the weeklong showcase event during the student summer camps.
  • Surveying of students twice a year in intervention schools.
  • Surveying of teachers at Summer Institutes and then at the end of the academic year.
  • Coding and entry of survey data; coding of interview and observational data.
  • Data analysis to examine the specific aims (SA) noted above:
    • The impact of ICAC on teacher computing confidence and utilization (SA 3.b).
    • Assess the effects of (1) teacher XO training on student computing confidence and utilization (SA 3.b), (2) training on changes in interest in STEM careers (SA 3.c), and (3) XO training on student engagement (SA 3.a).
    • A quasi-experimental comparison of intervention and non-intervention schools to assess intervention effects on student achievement (SA 3.a).
    • Survey of parents attending the yearly ICAC showcase to assess effects on parental attitudes toward STEM careers and computing (SA 3.d).

The proposed research has the potential for broad impact by leveraging technology in BCS to influence over 8,000 students in the Birmingham area. By targeting 4th and 5th grade students, we expect to impact STEM engagement and preparedness of students before they move into a critical educational and career decision-making process. Further, by bolstering student computer and STEM knowledge, ICAC will impart highly marketable skills that prepare them for the 81% of new jobs that are projected to be in computing and engineering in coming years (as predicted by the US Bureau of Labor Statistics).3 Through its formative and summative assessment, ICAC will offer intellectual merit by providing teachers throughout the US with insights into how computers can be used to integrate the elementary STEM curriculum. ICAC will develop a model for using computers to enhance STEM education across the curriculum while instilling a culture among BCS schools where computing is viewed as a tool for learning.

(Previously listed under Award # 0918216)

Building an Understanding of Science

Understanding Science provides an accurate portrayal of the nature of science and tools for teaching associated concepts. This project has at its heart a public re-engagement with science that begins with teacher preparation. To this end, its immediate goals are (1) improve teacher understanding of the nature of the scientific enterprise and (2) provide resources and strategies that encourage and enable K-16 teachers to incorporate and reinforce the nature of science throughout their science teaching.

Award Number: 
Funding Period: 
Mon, 03/12/2007 to Wed, 05/11/2011
Project Evaluator: 

The Role of Educative Curriculum Materials in Supporting Science Teaching Practices with English Language Learners

This project aims to determine whether curricula designed to support teacher and student learning have positive impacts on teacher knowledge, attitudes, and instructional practices; to what degree educative curricula help teachers with more and less experience teaching ELLs and how level of teaching experience relates to teacher knowledge, attitudes, and instructional practices; and the effects of the educative curricula in high implementation settings on ELLs knowledge and attitudes in science, and developing English proficiency.

Project Email: 
Award Number: 
Funding Period: 
Mon, 09/01/2008 to Fri, 08/31/2012
Project Evaluator: 
Jonna Kulikowich

Design and Use of Illustrations in Test Items as a Form of Accommodation for English Language Learners in Science and Mathematics Assessment

This project investigates how vignette illustrations minimize the impact of limited English proficiency on student performance in science tests. Different analyses will determine whether and how ELL and non-ELL students differ significantly on the ways they use vignettes to make sense of items; whether the use of vignettes reduces test-score differences due to language factors between ELL and non-ELL students; and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
Funding Period: 
Mon, 09/01/2008 to Sun, 08/12/2012
Full Description: 

This exploratory project within the Contextual Strand (Challenge a) addresses validity in the assessment of science and mathematics for English language learners (ELLs), and the urgent need for effective testing accommodations for ELLs. Motivation for this investigation originated from a previous, NSF-funded project on the testing of ELLs. We observed that items which were accompanied by illustrations tended to be responded correctly by a higher percentage of students than items without illustrations. We will investigate the factors that are relevant to designing and using a new form of accommodation in the assessment of science and mathematics for ELLs—vignette illustrations.

This three-year project will be guided by four research questions: What principles underlie the effective design of science and mathematics test items with illustrations in ways that minimize limited English proficiency as a factor that prevents ELLs from understanding the items? Is the presence of an illustration a moderator in students’ understanding test items? If so, Is the effect due to the simple presence of a graphical component or due to characteristics of the illustrations that are created based on principled design? Does the presence of an illustration have a different effect on the performance of ELLs and the performance of non-ELL students?

We expect to be able to: 1) identify the role of illustrations in the cognitive activities elicited by vignette-illustrated items; 2) determine whether any differences between performance on vignette-illustrated items and other kinds of items are due to the this form of accommodation’s capacity to address language as a construct-irrelevant factor; 3) identify the set of practical and methodological issues that are critical to properly developing and using vignette-illustrated items; and 4) propose a set of documents and procedures for the systematic and cost-effective design and development of vignette-illustrated items. 

We will test ELL and non-ELL students with items of three types (vignette-illustrated items whose illustrations are designed systematically, vignette-illustrated items whose illustrations are created arbitrarily, and items without illustrations) at two levels of distance to the enacted curriculum (close and distal). Diverse forms of analysis will allow us to determine whether and how ELL and non-ELL students differ on the ways in which they use vignettes to make sense of items, whether the use of vignettes reduces test score differences due to language factors between ELL and non-ELL students, and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.

Intellectual merit. This project will provide information that will help to advance our understanding in two assessment arenas: effective accommodations for ELLs, and item development practices. While illustrations are frequently used in test items, there is not guidance in the assessment development literature on how to approach illustrations. Furthermore, the value of illustrations as a resource for ensuring that ELL students understand what a given item is about and what the item asks them to do has not been systematically investigated. Semiotics, cognitive psychology, and linguistics and socio-cultural theory are brought together to develop systematic procedures for developing illustrations as visual supports in tests. Understanding the role that images play in test taking is relevant to devising more effective ways of testing students. While this project aims to improve testing accommodations practices for ELLs, knowledge gained from it will inform test development practices relevant to all student populations.

Broader impact. We expect outcomes of this project to contribute to enhanced practice in both classroom and large-scale assessment. The push for including ELLs in large-scale testing programs with accountability purposes is not corresponded by effective testing accommodation practices. Many testing accommodations used by national and state assessment programs are not defensible, are not effective, or are improperly implemented. Vignette illustrations have the potential to become a low-cost, easy-to-implement form of testing accommodation for ELLs. Results form this investigation will allow us to identify a set of principles for the proper design and use of vignette illustrations as a form of testing accommodation for ELLs. The project is important not only because it explores the potential of an innovative form of accommodation but because it uses a systematic procedure for designing that form of accommodation.

Applying Research on Science Materials Implementation: Bringing Measurement of Fidelity of Implementation (FOI) to Scale

This project is creating a suite of instruments for measuring fidelity of implementation of several science and mathematics instructional materials programs and a User's Guide for customizing those instruments to other programs. The instruments are grounded in a shared conceptual framework that organizes "critical components" that the programs share. The suite was piloted and field tested in over 50 schools in Chicago.
Lead Organization(s): 
Partner Organization(s): 
Award Number: 
Funding Period: 
Mon, 01/01/2007 to Wed, 06/30/2010


Subscribe to Implementation