This collaborative project between Tufts University and the Massachusetts Institute of Technology is researching and developing a new version of the Scratch programming language to be called ScratchJr, designed specifically for early childhood education (K-2). The current version of Scratch, which is widely implemented, is intended for ages 8-16 and is not developmentally appropriate for young children. This work will provide research-based evidence regarding young children's abilities to use an object-oriented programming language and to study the impact this has on the children's learning of scientific concepts and procedures. The team will develop ScratchJr in an iterative cycle, testing it in both in the Devtech lab at Tufts and the Eliot Pearson lab school and with a wider network of early childhood partners. At the end of the three-year project, ScratchJr will have been tested with approximately 350 students in K-2, 40 parents, and 58 early childhood educators.

ScratchJr will have three components: 1) a developmentally appropriate interface, with both touch screen and keyboard/mouse options; 2) an embedded library of curricular modules with STEM content to meet federal and state mandates in early childhood education; and 3) an on-line resource and community for early childhood educators and parents. The research questions focus on whether ScratchJr can help these young children learn foundational knowledge structures such as sequencing, causality, classification, composition, symbols, patterns, estimation, and prediction; specific content knowledge; and problem solving skills.

This interdisciplinary proposal makes contributions to the fields of learning technologies, early childhood education and human computer interaction. ScratchJr has the potential for broad implementation in both formal and informal settings.

This is a two-year exploratory study to identify critical aspects of effective science formative assessment (FA) practices for English Language Learners (ELLs), and the contextual factors influencing such practices. Three institutions join efforts for this purpose: University of Colorado at Boulder, University of Colorado at Denver, and University of Washington. FA, in the context of the study, is viewed as a process contributing to the science learning of ELLs, as opposed to the administration of discrete sets of instruments to collect data from students. The study targets Spanish-speaking, elementary and middle school students. Findings from this study contribute to advance knowledge and understanding of FA as an inherent component of the science learning process in linguistically diverse classrooms, and to define a research agenda aimed at enhancing science teachers' ability to enact equitable and effective assessment practices for this student subpopulation.

Three research questions guide the work: (1) What FA practices are occurring in science classrooms that serve predominantly mainstream students and in those serving predominantly ELLs?; (2) How are teachers' FA practices for mainstream students different from or similar to those used with ELLs?; and (3) How do contextual factors and teachers' cultural and linguistic competencies influence FA practices? To address these questions, two conceptual frameworks are used--one for characterizing FA events; the other for examining FA events as a communication process. The study employs a mixed-methods research approach with emphasis on case studies. The sample size consists of three school districts in Colorado and Washington, 16 classrooms (8 elementary, 8 middle school), 16 teachers, and 96 ELLs. Classrooms are selected to represent a particular combination of four factors: (a) teacher ethnicity, (b) teacher formal academic preparation in teaching ELLs, (c) type of linguistic student background, and (d) grade level. Students are selected through a stratified random sample, identified by achievement level (i.e., low, medium, high), and linguistic background (i.e., mainstream, ELL). Data collection strategies to document the implementation of FA at the beginning, during, and at the end of a science unit include: (a) classroom observation protocols, (b) classroom video-recording, (c) video/artifact simulated recall, (d) assessment artifacts, (e) student interviews, (f) teacher questionnaires, (g) teacher interviews, (h) school principal interviews, and (i) school observations. Reliability and validity of most of the data-gathering instruments is determined through pilot studies. Data interpretation strategies include: (a) coding based on the two conceptual frameworks, (b) scoring rubrics to identify levels of effectiveness, and (c) narratives and profiles to describe FA patterns. Publications and the development of a website constitute the main dissemination strategies. A technical advisory board is responsible for formative and summative evaluation. Key evaluation questions are: (1) To what extent does the project enhance research on ELL FA practices through case studies?, and (2) How effectively do the project dissemination activities facilitate understanding of FA practices?

Major project outcomes include: (1) a description of the patterns of formal and informal FA practices for ELLs; (2) a comparison of the FA practices observed in classrooms that vary on the dimensions of teacher characteristics and linguistic diversity; and (3) an empirically and theoretically informed set of findings and strategies for supporting teachers to enact and enhance FA practices sensitive to cultural and linguistic diversity. Three main products are developed: (1) a monograph describing the FA practices observed across the different classrooms with concrete examples; (2) a description of possible professional development strategies to improve in-service FA practices for linguistically diverse students; and (3) a research-informed approach for analyzing FA practices. Besides filling the existing research gap on FA with ELLs, outcomes and products serve as a foundation for a future research agenda and a comprehensive project aimed at ensuring equitable science learning for all students, including ELLs.

The goal of this project is to develop two tests (instruments) to assess conceptual understanding of statistics.

The instruments are based on the levels A/B and on level C of statistical understanding development as described in the American Statistical Association Guidelines for Assessment and Instruction of Statistics Education (GAISE) framework. These instruments will be used to assess knowledge of statistics by grades 6-12 students. The instruments will have multiple-choice and constructed response (CR) items. The CR items will have scoring rubrics. The assessments will be pilot tested in school districts in six states. The instruments will be used by teachers to analyze students' growth in understanding of statistics and will be useable for both formative and summative purposes. An assessment blueprint will be developed based on the GAISE framework for selecting and constructing both fixed-choice and open-ended items. An evidenced-based designed process will be used to develop the assessments. The blueprint will be used by the test development committee to develop items. These items will be reviewed by the advisory board considering the main statistics topics to be included on the assessments. Through a layering process, the assessments will be piloted, revised, and field tested with students in grades 6-12 in six states. A three-parameter IRT model will be used in analyzing the items. The work will be done by researchers at the University of Florida with the support of those at the University of Minnesota, the Educational Testing Service, and Kenyon College. Researchers from TERC will conduct a process evaluation with several feedback and redesign cycles.

The assessments will be aligned with the Common Core State Standards for mathematics (CCSSM) and made available as open-source to teachers through a website. The research team will interact with the state consortia developing assessments to measure students? attainment of the CCSSM. As such, the assessments have the potential of being used by a large proportion of students in the country. The more conceptually-based items will provide teachers with concrete examples of what statistics students in grades 6-12 should know.

 

The Consortium for Mathematics and Its Applications is engaged in an effort to bring together members of the community of mathematics curriculum developers and experts in assessment, including representatives and stakeholders from the two assessment consortia recently funded by the US Department of Education. The primary goal of the meeting is to discuss issues and recommend actions regarding assessment design and production that arise in an environment of Common Core State Standards (CCSS). The format will allow information exchange and resource sharing in the service of states and districts as they work to effectively implement the CCSS. Participants will include representatives from the governing states as well as from the leadership teams of the assessment consortia along with experts in curriculum development, standards implementation and assessment.

This work is being carried out with the support and cooperation of the National Council of Teachers of Mathematics, the National Council of Supervisors of Mathematics, the Association of Mathematics Teacher Educators, and the Association of State Supervisors of Mathematics.

This workshop developed a new, comprehensive, research-based framework for assessing environmental literacy. By bringing together, for the first time, experts in research, assessment, and evaluation from the fields of science education, environmental education, and related social science fields, this project accessed and built its work on the literature and the insights of many disciplines. The North American Association for Environmental Education (NAAEE) worked with the leaders of the only two large-scale assessments of environmental literacy used in the U.S. to date (Programme for International Student Assessment [PISA] and the National Environmental Literacy Assessment [NELA]) to conduct the workshop. The project leaders analyzed PISA and NELA and used a multi-disciplinary search and review of the literature to prepare a draft framework. At the workshop and thereafter, a diverse array of invited experts critiqued that draft and provided suggestions for revision. Then, the leaders/organizers produced a final Environmental Literacy Framework and disseminated it both electronically and at a nationally advertised event to a wide audience of assessment specialists, funding and policy-making agencies, and organizations working to develop assessments and achieve environmental literacy. Many institutions and agencies have noted the need to create an environmentally literate population, and government and private entities are investing hundreds of millions of dollars in projects aimed at enhancing environmental literacy. Given the scope and scale of these investments and the interest in this arena on the part of federal agencies, professional organizations, and corporations, assessments for gauging our progress in transforming our preK-12 education system to achieve that end are needed. The new Framework for assessing environmental literacy provides a foundation for measuring the extent to which we are enabling all learners to acquire the knowledge, skills, dispositions, and behaviors vital for competently making decisions about local, regional, national and global issues.

Having a geographically literate population will be critical to the economic stability, physical security, and environmental sustainability of the United States in the 21st century. Yet the U.S. still lags far behind the other developed nations in education in the geographical sciences. Recognizing the risk that geographic illiteracy poses for our country, the National Geographic Society (NGS), in collaboration with the Association of American Geographers, American Geographical Society, and National Council for Geographic Education, proposes to engage in a set of research synthesis and dissemination activities that will provide road maps for the design of assessment, professional development, instructional materials, public information, and educational research for the next decade. The work will be done by a broad range of experts from K-12 institutions as well as the geographical science and educational research communities

Building on a 25 year collaboration, NGS and its partners propose to engage in a community-wide effort to synthesize the literature from a broad range of fields and to use the findings to create frameworks that will guide the planning, implementation, and scale-up of efforts to improve geographic education over the next decade. The result of this effort will be a set of publicly reviewed, consensus reports that will guide the collaborative efforts of the project partners and the larger geographic education community, as well as broaden awareness of the increasingly significant and acute need for geographic literacy and education in the geographical sciences in our country.

This project will create three in-depth "roadmap" reports targeted at practitioners, takeholders, and policymakers. Developed by expert committees, these three reports will be on:

- Assessment frameworks for systematic monitoring and continuous improvement of geographic education programs.

- Professional development for teachers and instructional materials to support large-scale educational improvement across diverse contexts.

- Educational research agenda to set priorities and identify appropriate methodologies for research that will improve geographic education into the future.

These three reports will be summarized in an executive summary written for a broad audience of educators, policymakers, and concerned citizens.

In addition to these consensus reports, the project will also conduct research on public understanding of the nature and importance of geographic literacy, with particular attention to the key audiences of educators, policymakers, and citizens. In addition to shaping the project's reports, this research will inform the broader communications and dissemination efforts of this project and its partners.

This project--led by science educators at Michigan State University, the National Geographic Society, the Natural Resource Ecology Laboratory (NREL) at Colorado State University, the Berkeley Evaluation and Assessment Research (BEAR) Center, and AAAS Project 2061, and including schools in California, Colorado, Maryland, Michigan, and Washington--builds on prior efforts with learning progressions, and is focused on key carbon-transforming processes in socio-ecological systems at multiple scales, including cellular and organismal metabolism, ecosystem energetics and carbon cycling, carbon sequestration, and combustion of fossil fuels.

The project uses an iterative design research process to develop and refine a suite of tools for reasoning and test efficacy of those tools in geographically and culturally diverse schools. The project team is:

1. Refining and validating a detailed learning progression framework covering the middle and high school years; ultimately, the framework will describe the development of students' capacity to use fundamental principles such as conservation of matter and energy to reason about carbon-transforming processes at multiple scales.

2. Refining 'Tools for Reasoning' that make hidden scientific principles - matter, energy, and scale - visible to students; the power of these tools lies in their flexible use for different processes, systems, scales, and curricular contexts.

3. Developing and refining flexible teaching strategies that engage students in cognitive apprenticeship in the practices of environmental science literacy: a) inquiry and argumentation, b) explanations and predictions, and c) decision-making about environmental issues.

4. Using and refining existing summative assessments, and developing and testing formative assessment tools; these assessment tools will provide teachers and researchers with immediate information about their students' intellectual resources and will be linked to the learning progression framework.

5. Developing, field testing, and assessing the effectiveness of six middle school and six high school units that use project tools and enact project principles; the units introduce students to fundamental principles, engage them in reasoning about carbon-transforming processes at organismal scale, and at landscape and global scales. Each unit includes a) an online formative assessment and b) activity sequences that use tools for reasoning and teaching strategies.

6. Developing, field testing, and assessing professional development materials in both face-to-face and facilitated online forms; the materials introduce teachers to learning progressions in environmental science literacy, assessment tools, tools for reasoning, teaching strategies, and teaching materials and activities, and also address difficulties that teachers encounter in using learning progressions and enacting teaching strategies.

The primary project outcomes will be coordinated instructional tools that are useful to professionals at all levels in the science education system--classroom teachers, professional developers, and developers of curricula, standards and assessments.

This is an efficacy study through which the Denver Museum of Nature and Science, the Denver Zoo, the Denver Botanic Gardens, and three of Denver's urban school districts join efforts to determine if partnerships among formal and informal organizations demonstrate an appropriate infrastructure for improving science literacy among urban middle school science students. The Metropolitan Denver Urban Advantage (UA Denver) program is used for this purpose. This program consists of three design elements: (a) student-driven investigations, (b) STEM-related content, and (c) alignment of schools and informal science education institutions; and six major components: (a) professional development for teachers, (b) classroom materials and resources, (c) access to science-rich organizations, (d) outreach to families, (e) capacity building and sustainability, and (e) program assessment and student learning. Three research questions guide the study: (1) How does the participation in the program affect students' science knowledge, skills, and attitudes toward science relative to comparison groups of students? (2) How does the participation in the program affect teachers' science knowledge, skills, and abilities relative to comparison groups of teachers? and (3) How do families' participation in the program affect their engagement in and support for their children's science learning and aspirations relative to comparison families?

The study's guiding hypothesis is that the UA Denver program should improve science literacy in urban middle school students measured by (a) students' increased understanding of science, as reflected in their science investigations or "exit projects"; (b) teachers' increased understanding of science and their ability to support students in their exit projects, as documented by classroom observations, observations of professional development activities, and surveys; and (c) school groups' and families' increased visits to participating science-based institutions, through surveys. The study employs an experimental research design. Schools are randomly assigned to either intervention or comparison groups and classrooms will be the units of analysis. Power analysis recommended a sample of 18 intervention and 18 comparison middle schools, with approximately 72 seventh grade science teachers, over 5,000 students, and 12,000 individual parents in order to detect differences among intervention and comparison groups. To answer the three research questions, data gathering strategies include: (a) students' standardized test scores from the Colorado Student Assessment Program, (b) students' pre-post science learning assessment using the Northwest Evaluation Association's Measures for Academic Progress (science), (c) students' pre-post science aspirations and goals using the Modified Attitude Toward Science Inventory, (d) teachers' fidelity of implementation using the Teaching Science as Inquiry instrument, and (e) classroom interactions using the Science Teacher Inquiry Rubric, and the Reformed Teaching Observation protocol. To interpret the main three levels of data (students, nested in teachers, nested within schools), hierarchical linear modeling (HLM), including HLM6 application, are utilized. An advisory board, including experts in research methodologies, science, informal science education, assessment, and measurement oversees the progress of the study and provides guidance to the research team. An external evaluator assesses both formative and summative aspects of the evaluation component of the scope of work.

The key outcome of the study is a research-informed and field-tested intervention implemented under specific conditions for enhancing middle school science learning and teaching, and supported by partnerships between formal and informal organizations.

Research on student learning has developed separate progressions for scientific argumentation, explanation and scientific modeling. Engaging Learners in Scientific Practices develops a learning progression that characterizes how learners integrate and interrelate scientific argumentation, explanation and scientific modeling, building ever more sophisticated versions of practice over time using the three common elements of sense-making, persuading peers and developing consensus. The learning progression is constructed through improvements in students' performance and understanding of scientific practice as measured by their attention to generality of explanation, attention to clarity of communication and audience understanding, attention to evidentiary support, and attention to mechanistic versus descriptive accounts. The project is led by researchers at Northwestern University, the University of Texas, Wright State University, Michigan State University, and the BEAR assessment group. Two cohorts of 180 students each are followed for two years from 4th to 5th grade in Illinois and two cohorts of 180 students each are followed for two years from 5th to 6th grade in Michigan The elementary school students will work with FOSS curriculum units modified to embed supports for scientific practices. Two cohorts of 500 middle school students are followed for three years from 6th to 8th grade as they work with coordinated IQWST units over three years. The outcome measures include analyses of classroom discourse, pre- and pos-test assessments of student learning, and reflective interviews grounded in students' own experiences with practices in the classroom to assess their growth across the dimensions. The BEAR team is responsible for validation and calibration of the frameworks and instruments, and design of the scheme for analysis of the data. Horizon Research performs the formative and summative evaluation. The project will produce an empirically-tested learning progression for scientific practices for grades 4-8 along with tested curriculum materials and validated assessment items that support and measure students' ability in the scientific practices of explanation, argumentation and modeling. In the process of development, an understanding is gained about how to design and test this learning progression. The framework is articulated on a website for use by other researchers and developers. The project also builds capacity by educating several graduate students.

Led by STEM educators at Texas A&M University, this project is analyzing and sharing baseline data on the achievement of African American and Hispanic girls on national and state assessments. The objectives of the project are: (1) To conduct a critical analysis of National Assessment of Educational Progress (NAEP) and Texas Assessment of Knowledge and Skills (TAKS) achievement data for African American and Hispanic female students in grades 3-6 with a focus on sub-test objectives for science, mathematics, and reading over the years 2000-2010; (2) To organize a one-day conference for 100 teachers, administrators and parents from urban, rural and suburban school districts featuring presentation of the data analysis and a national speaker who will share information and lead discussion on why African American and Hispanic girls at the elementary level should begin to think about seeking STEM careers and the required expected academic preparations; (3) To provide conference participants with STEM career information and materials; and (4) To share results of the achievement data analysis at international/national conferences (National Council for Teachers of Mathematics, National Science Teachers Association, American Educational Research Association) and submit papers for publication in scholarly journals.

Quantitative and qualitative methodology will be used to respond to three research questions: (1) What are the differences in the academic achievement of African American and Hispanic girls in grades 3-6 on the National Assessment of Educational Progress (NAEP) and Texas Assessment of Knowledge and Skills (TAKS) during the years 2000-2010? (2) What are the voices of African American and Hispanic 6th grade girls about their TAKS test from third grade to sixth grade? (3) What is the impact of a one-day conference on raising the awareness level of educators and parents about academic achievement among African American and Hispanic girls on national and state assessment in grades 3-6 in reading, mathematics, and science? To address question number one, the study will determine if statistically significant differences exist among the variables of race, class, and gender by grades and subject on student performance on the NAEP and TAKS tests and sub-tests in the areas of reading, mathematics and science. To address question number two, a qualitative analysis will be conducted. Students will be interviewed and data will be transcribed, sorted, and categorized into themes. Member checks and triangulation of data will be used to establish validity and reliability of the findings. To address question number three, descriptive statistics will be used to analyze a Likert-type survey instrument that will be developed by the project PI and CoPIs to assess conference objectives. In addition, a purposive sample of participants (teachers and parents) will be interviewed about their participation in the conference and their responses analyzed using qualitative analysis.

With a focus on African American and Hispanic girls' academic achievement, the project will provide educators, parents and students through a conference venue and other outlets with valuable information to understand their competency in subjects that can impact their decisions to seek STEM careers.