Scale-up

The Challenges of Scaling Up

Day: 
Thu

Dynabook and SmartGraphs answer the questions: What are the major challenges in scaling up, and how do you plan to meet them?

Date/Time: 
1:45 pm - 3:45 pm
Session Type: 
PI-organized Discussion
Presenters: 

The goal of the session is to help attendees understand challenges of scaling up and to inform their design of scaling-up strategies for their own projects. The session has four parts: an introduction in which key challenges are identified, two parts in which Dynabook and SmartGraphs describe strategies for meeting challenges, and a part for audience to interactaction. To successfully scale up, organizations need to concisely identify the “value proposition” that appeals to users, which requires an understanding of need, approach, benefits, and competition. Discussions will focus on the value proposition, especially on user needs addressed by the project, benefits of the innovation compared with competing approaches, and compatibility with the environment in which it will be used. Other challenges include generating revenue (if a continuing revenue stream is needed), finding channels to promote the innovation, identifying partners with whom to work, and making sure the project fits comfortably in its own organization (including the bureaucracy).

Dynabook is an interactive Web-based tool for teaching mathematics content and pedagogy to middle school teachers. Dynabook’s value proposition for Special Education preservice and inservice providers is this: When utilized in a well-designed university classroom activity system, Dynabook advances content knowledge, technology integration, and creative ideas for pedagogy, and provides participants an opportunity to take part in a non-intimidating learning environment that encourages social and cognitive engagement in issues surrounding middle school mathematics instruction for diverse learners.

SmartGraphs (www.concord.org/smartgraphs) offers free Web-based software and activities for STEM classrooms. The SmartGraphs value proposition for teachers is, “Graphs are central to STEM learning in algebra, physical science, and other STEM courses yet many students have difficulty understanding graphs and the concepts they represent. Free SmartGraphs activities help teachers teach students to understand graphs better than other software. SmartGraphs works with standard curricula, in typical schools—and there’s nothing to install.”

Cluster Randomized Trial of the Efficacy of Early Childhood Science Education for Low-income Children

The research goal of this project is to evaluate whether an early childhood science education program, implemented in low-income preschool settings produces measurable impacts for children, teachers, and parents. The study is determining the efficacy of the program on Science curriculum in two models, one in which teachers participate in professional development activities (the intervention), and another in which teachers receive the curriculum and teachers' guide but no professional development (the control).

Project Email: 
vanegere@msu.edu
Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1119327
Funding Period: 
Mon, 08/15/2011 - Wed, 07/31/2013
Project Evaluator: 
Brian Dates, Southwest Counseling Services
Full Description: 

The research goal of this project is to evaluate whether an early childhood science education program, Head Start on Science, implemented in low-income preschool settings (Head Start) produces measurable impacts for children, teachers, and parents. The study is being conducted in eight Head Start programs in Michigan, involving 72 classrooms, 144 teachers, and 576 students and their parents. Partners include Michigan State University, Grand Valley State University, and the 8 Head Start programs. Southwest Counseling Solutions is the external evaluator.

The study is determining the efficacy of the Head Start on Science curriculum in two models, one in which 72 teachers participate in professional development activities (the intervention), and another in which 72 teachers receive the curriculum and teachers' guide but no professional development (the control). The teacher study is a multi-site cluster randomized trial (MSCRT) with the classroom being the unit of randomization. Four time points over two years permit analysis through multilevel latent growth curve models. For teachers, measurement instruments include Attitudes Toward Science (ATS survey), the Head Start on Science Observation Protocol, the Preschool Classroom Science Materials/Equipment Checklist, the Preschool Science Classroom Activities Checklist, and the Classroom Assessment Scoring System (CLASS). For students, measures include the "mouse house problem," Knowledge of Biological Properties, the physics of falling objects, the Peabody Picture Vocabulary Test-Fourth Edition, the Expressive Vocabulary Test-2, the Test of Early Mathematics Ability-3, Social Skills Improvement System-Rating Scales, and the Emotion Regulation Checklist. Measures for parents include the Attitudes Toward Science survey, and the Community and Home Activities Related to Science and Technology for Preschool Children (CHARTS/PS). There are Spanish versions of many of these instruments which can be used as needed. The external evaluation is monitoring the project progress toward its objectives and the processes of the research study.

This project meets a critical need for early childhood science education. Research has shown that very young children can achieve significant learning in science. The curriculum Head Start on Science has been carefully designed for 3-5 year old children and is one of only a few science programs for this audience with a national reach. This study intends to provide a sound basis for early childhood science education by demonstrating the efficacy of this important curriculum in the context of a professional development model for teachers.

Cluster Randomized Trial of the Efficacy of Early Childhood Science Education for Low-income Children

An Innovative Approach to Earth Science Teacher Preparation: Uniting Science, Informal Science Education, and Schools to Raise Student Achievement

The American Museum of Natural History in New York City, in partnership with New York University, and in collaboration with five high-needs schools, is developing, implementing, and researching a five-year pilot Master of Arts in Teaching (MAT) program in Earth Science. The program is delivered by the Museum's scientific and education teams and its evaluation covers aspects of the program from recruitment to first year of teaching.

Project Email: 
mat@amnh.org
Lead Organization(s): 
Award Number: 
1119444
Funding Period: 
Thu, 09/01/2011 - Wed, 08/31/2016
Project Evaluator: 
David Silvernail, Center for Education and Policy, University of Southern Maine
Full Description: 

The American Museum of Natural History (AMNH), in collaboration with New York University's Institute for Education and Social Policy and the University of Southern Maine Center for Evaluation and Policy, will develop and evaluate a new teacher education program model to prepare science teachers through a partnership between a world class science museum and high need schools in metropolitan New York City (NYC). This innovative pilot residency model was approved by the New York State (NYS) Board of Regents as part of the state’s Race To The Top award. The program will prepare a total of 50 candidates in two cohorts (2012 and 2013) to earn a Board of Regents-awarded Masters of Arts in Teaching (MAT) degree with a specialization in Earth Science for grades 7-12. The program focuses on Earth Science both because it is one of the greatest areas of science teacher shortages in urban areas and because AMNH has the ability to leverage the required scientific and educational resources in Earth Science and allied disciplines, including paleontology and astrophysics.

The proposed 15-month, 36-credit residency program is followed by two additional years of mentoring for new teachers. In addition to a full academic year of residency in high-needs public schools, teacher candidates will undertake two AMNH-based clinical summer residencies; a Museum Teaching Residency prior to entering their host schools, and a Museum Science Residency prior to entering the teaching profession. All courses will be taught by teams of doctoral-level educators and scientists.

The project’s research and evaluation components will examine the factors and outcomes of a program offered through a science museum working with the formal teacher preparation system in high need schools. Formative and summative evaluations will document all aspects of the program. In light of the NYS requirement that the pilot program be implemented in high-need, low-performing schools, this project has the potential to engage, motivate and improve the Earth Science achievement and interest in STEM careers of thousands of students from traditionally  underrepresented populations including English language learners, special education students, and racial minority groups. In addition, this project will gather meaningful data on the role science museums can play in preparing well-qualified Earth Science teachers. The research component will examine the impact of this new teacher preparation model on student achievement in metropolitan NYC schools. More specifically, this project asks, "How do Earth Science students taught by first year AMNH MAT Earth Science teachers perform academically in comparison with students taught by first year Earth Science teachers not prepared in the AMNH program?.”

An Innovative Approach to Earth Science Teacher Preparation: Uniting Science, Informal Science Education, and Schools to Raise Student Achievement

Promoting Science among English Language Learners (P-SELL) Scale-Up

This effectiveness study focuses on the scale-up of a model of curricular and teacher professional development intervention aimed at improving science achievement of all students, especially English language learners (ELLs). The model consists of three basic components: (a) inquiry-oriented science curriculum, (b) teacher professional development for science instruction with these students, and (c) school resources for science instruction.

Lead Organization(s): 
Award Number: 
1209309
Funding Period: 
Mon, 08/15/2011 - Fri, 07/31/2015
Project Evaluator: 
Lauren Scher
Full Description: 

This four-year effectiveness study focuses on the scale-up of a model of curricular and teacher professional development intervention aimed at improving science achievement of all students, especially English language learners (ELLs). The model consists of three basic components: (a) inquiry-oriented science curriculum, (b) teacher professional development for science instruction with these students, and (c) school resources for science instruction. The project's main goals are: (1) to evaluate the effect of the intervention on student achievement, (2) to determine the effect of the intervention on teacher knowledge, practices, and school resources, and (3) to assess how teacher knowledge, practices, and resources mediate student achievement. The project is conducted in the context of the Florida current science education policies and accountability system (e.g., adoption of the Next Generation Sunshine State Standards in Science, assessment of science at the fifth grade, a Race to the Top award state). The study draws on findings from research on a previous NSF-funded efficacy study (035331) in which the model to be scaled-up was tested in a single school district. The effectiveness study includes three (of 67) school districts as key partners, representative of racially, ethnically, linguistically, and socioeconomically diverse student populations; 64 elementary schools, 320 science teachers, and 24,000 fifth-grade students over a three-year period. Science learning is the primary subject matter, inclusive of life, physical, and earth/space sciences.

Six research questions corresponding to three research areas guide the proposed scope of work. For the research area of Student Science Achievement, questions are: (1) What is the effect of the intervention on fifth-grade students' science achievement, compared to "business as usual"?, and (2) To what extent are the effects of the intervention moderated by students' English as a Second Language (ESOL) level, SES status, and racial/ethnic backgrounds? For Teacher Knowledge and Practices as a research area, questions are: (3) What is the effect of the intervention on teachers' science knowledge and teaching practices?, and (4) To what extent is students' science achievement predicted by school resources for science instruction? For School Resources for Science, questions are: (5) What is the effect of the intervention on school resources for science instruction?, and (6) To what extent is student achievement predicted by school resources for science instruction? To assess the effect of the intervention on students' and teachers' outcomes, a cluster-randomized-control trial is used, resulting in a total of 64 randomly selected schools (after stratifying them by school-level percent of ESOL and Free Reduced Lunch students). All science teachers and students from the 64 schools participate in the project: 32 in the treatment group (project curriculum for fifth grade, teacher professional development, and instructional resources), and 32 in the control group (district-adopted fifth-grade curriculum, no teacher professional development, and no instructional resources). To address the research area of Student Science Achievement, formative assessment items are used at the end of each curriculum unit, along with two equated forms of a project-developed science test (to be used as pre-and posttests) with both treatment and control groups, in addition to the Florida's Comprehensive Assessment Tests-Science. Data interpretation for this research area employs a set of three-level HLMs (students, nested in classrooms, nested in schools). To address the research area of Teacher Knowledge and Practices and School Resources for Science, the project uses three measures: (a) two equated forms of a 35-items test of teacher science knowledge, (b) a classroom observation instrument measuring third-party ratings of teacher knowledge and teaching practices, and (c) a questionnaire measuring teachers' self-reports of science knowledge and teaching practices. All measures are administered to both treatment and control groups. Data interpretation strategies include a series of HLMs with emphasis on the relevant teacher outcomes as a function of time, and of school-level mediating variables. External project evaluation is conducted by Concentric Research and Evaluation using quantitative and qualitative methods and addressing both formative and summative components.

Project research findings contribute to the refinement of a model reflective of the new science standards in the State and the emerging national science standards. The value added of this effort consists of its potential to inform effective implementation of science curricula and teacher professional development in other learning settings, including ELLs and traditionally marginalized student populations at the elementary school level. It constitutes practically the only research study focused on the issue of scale-up and sustainability of effective science education practices with this student subpopulation, which has become prominent due to the dramatic growth of a racially, ethnically, and linguistically diverse school-aged population, low levels of U.S. student science achievement, and the role of science and mathematics in current accountability systems nationwide.

Promoting Science among English Language Learners (P-SELL) Scale-Up

Investigating and Supporting the Development of Ambitious and Equitable Mathematics Instruction at Scale

This project is supporting and investigating the implementation of reformed mathematics instruction at the middle school level in two large school districts. The primary goal of the project is to develop an empirically grounded theory of action for implementing reform at school and district levels. The researchers are investigating reform within a coherent system that focuses on leadership and school-based professional development.

 

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1119122
Funding Period: 
Mon, 08/15/2011 - Tue, 07/31/2012
Full Description: 

The Development of Ambitious and Equitable Mathematics Instruction project is supporting and investigating the implementation of reformed mathematics instruction at the middle school level in two large school districts. Project researchers are asking: What does it take to support mathematics teachers' development of ambitious and equitable instructional practices on a large scale? The project has built on what was learned in a previous, successful project studying the implementation of a middle school mathematics curriculum. The primary goal of the new project is to develop an empirically grounded theory of action for implementing reform at school and district levels. The researchers are investigating reform within a coherent system that focuses on leadership and school-based professional development. In addition, they are facilitating a longitudinal study of the curriculum implementation by continuing the data collection from the original study.

In order to build a theory of action, the project team is synthesizing data from a variety of domains including instructional systems (e.g., curriculum, materials, professional development, support for struggling students, and learning communities), mathematics coaching, networks of teachers, school leadership, and district leadership. Investigators are using a variety of analytic techniques to successfully integrate both quantitative and qualitative data as they seek to understand how school district strategies are playing out in schools and classrooms and how those strategies can be revised in order to improve student learning of mathematics.

An empirically grounded theory of action for implementing reform will help the mathematics education community to implement and to understand the process of reforming mathematics instruction at the middle school level. Many advances in mathematics instruction have been documented within a limited context, but researchers and practitioners need to understand the full range of action necessary to achieve similar successes at a district-wide level. The model developed from this project, in conjunction with longitudinal data, has the potential to guide future reform efforts that seek to provide ambitious and equitable mathematics instruction.

Investigating and Supporting the Development of Ambitious and Equitable Mathematics Instruction at Scale

Sustainability Roundable

Day: 
Thu

Join this roundtable to discuss what we understand about sustainability within dynamic systems, adoption versus adaptation, scaling practices up or down, and other challenges to keeping our work alive.

Date/Time: 
4:30 pm - 5:30 pm
Session Type: 
Project Management & Implementation Roundtable
Facilitators: 

Please bring your questions and ideas on sustainability to a roundtable to discuss with colleagues. 

 

Some initial thoughts to consider:

  • What is it that you want to sustain and what are the concerns you have about how to do so? 
  • What have you learned about sustaining best practices in STEM education within dynamic systems?
  • How does scaling practices up or down influence sustainability?  How does adoption versus adaptation influence sustainability?
  • How do we 'sustain' our research on STEM education reforms long enough and deeply enough to understand sustainability?

The Scalability and Sustainability of Professional Development: Challenges in Preparing Mathematics PD Facilitators

Day: 
Fri

Three projects investigating different approaches to preparing mathematics professional development facilitators will compare their approaches and discuss strengths and limitations with the participants.

Date/Time: 
10:00 am - 12:00 pm
Session Type: 
Panel

What are the central challenges that face research and development teams in their efforts to prepare facilitators to do the work of leading mathematics professional development? 

A growing body of empirical research and general consensus within the research community address the structure, content, and outcomes of effective professional development (Borko, Jacobs & Koellner, 2010 ; Desimone, 2009). The field lacks a corresponding body of research, however, on what professional developers actually do to cultivate the kind of teaching practice that will result in student learning of complex subject matter, and how those individuals are prepared to design and lead such PD (Borko, 2004 ; Even, 2008). Understanding the actual practice of professional developers is arguably a key component of the growing effort to investigate the contribution that PD makes to teacher knowledge, classroom practice, and student learning. The PD context is different from the classroom context, and what PD facilitators need to do with teachers is not simply a replication of what teachers should be doing with students. The knowledge and skills required to conduct effective teacher PD have not clearly been delineated. 

This session focuses on the professional development of leaders of mathematics PD. Our overall goal is to engage DR K–12 grantees in considering what we are learning about the knowledge and skills required by PD facilitators and the challenges of facilitator preparation. 

Three projects will participate:

  • Toward a Scalable Model of Mathematics PD is field testing the Problem-Solving Cycle model of PD and accompanying facilitation materials and support for middle school mathematics instructional leaders (ILs) in a large urban school district. Researchers are documenting and analyzing the preparation and support provided to the ILs, range and quality of their PD implementation, nature of adaptations, and impact of the PD on ILs, teachers, and students.
  • Learning and Teaching Geometry is creating five video case modules focused on classroom instruction. These materials are designed to support middle school mathematics teachers’ development of the mathematics knowledge for teaching related to geometric similarity. The project also attends explicitly to facilitation: materials are designed to support facilitators’ adherence to developers’ goals and intentions when enacting sessions. Field testing of the materials includes investigation of the challenges facilitators face and the adaptations they make when using the materials. 
  • Researching Mathematics Leader Learning is studying how leaders learn to cultivate mathematically rich professional development environments. Researchers explore what leaders learn in a series of leader seminars and how they facilitate mathematical tasks with teachers in PD focused on mathematical knowledge for teaching. The project is following leaders’ facilitation of PD, uncovering how they identify learning goals for teachers, attend to teachers’ mathematical reasoning, and learn how the mathematical demands of working with teachers in PD are different than the demands of working in classrooms. 

Panelists will describe their projects and the key challenges they have faced related to the preparation of mathematics PD facilitators, and then discuss similarities/differences in the projects’ approaches. During the final section, presenters will discuss with the audience their growing understanding of the facilitation demands of teacher PD and implications for facilitator preparation. 

Establishing a Roadmap for Large-scale Improvement of K-12 Education in the Geographical Sciences

This project will engage in a community-wide effort to synthesize the literature from a broad range of fields and to use the findings to create frameworks that will guide the planning, implementation, and scale-up of efforts to improve geographic education over the next decade. This will result in a set of publicly reviewed, consensus reports that will guide collaborative efforts and broaden awareness of the acute need for geographic literacy and geographic science education.

Lead Organization(s): 
Award Number: 
1049437
Funding Period: 
Wed, 09/15/2010 - Fri, 08/31/2012
Project Evaluator: 
Education and Training Institute
Full Description: 

Having a geographically literate population will be critical to the economic stability, physical security, and environmental sustainability of the United States in the 21st century. Yet the U.S. still lags far behind the other developed nations in education in the geographical sciences. Recognizing the risk that geographic illiteracy poses for our country, the National Geographic Society (NGS), in collaboration with the Association of American Geographers, American Geographical Society, and National Council for Geographic Education, proposes to engage in a set of research synthesis and dissemination activities that will provide road maps for the design of assessment, professional development, instructional materials, public information, and educational research for the next decade. The work will be done by a broad range of experts from K-12 institutions as well as the geographical science and educational research communities

Building on a 25 year collaboration, NGS and its partners propose to engage in a community-wide effort to synthesize the literature from a broad range of fields and to use the findings to create frameworks that will guide the planning, implementation, and scale-up of efforts to improve geographic education over the next decade. The result of this effort will be a set of publicly reviewed, consensus reports that will guide the collaborative efforts of the project partners and the larger geographic education community, as well as broaden awareness of the increasingly significant and acute need for geographic literacy and education in the geographical sciences in our country.

This project will create three in-depth "roadmap" reports targeted at practitioners, takeholders, and policymakers. Developed by expert committees, these three reports will be on:

- Assessment frameworks for systematic monitoring and continuous improvement of geographic education programs.

- Professional development for teachers and instructional materials to support large-scale educational improvement across diverse contexts.

- Educational research agenda to set priorities and identify appropriate methodologies for research that will improve geographic education into the future.

These three reports will be summarized in an executive summary written for a broad audience of educators, policymakers, and concerned citizens.

In addition to these consensus reports, the project will also conduct research on public understanding of the nature and importance of geographic literacy, with particular attention to the key audiences of educators, policymakers, and citizens. In addition to shaping the project's reports, this research will inform the broader communications and dissemination efforts of this project and its partners.

Establishing a Roadmap for Large-scale Improvement of K-12 Education in the Geographical Sciences

Confronting the Challenges of Climate Literacy (Collaborative Research: Ledley)

This project is developing inquiry-based, lab-focused, online Climate Change EarthLabs modules as a context for ongoing research into how high school students grasp change over time in the Earth System on multiple time scales. This project examines the challenges to high-school students' understanding of Earth's complex systems, operating over various temporal and spatial scales, and by developing research-based insights into effective educational tools and approaches that support learning about climate change and Earth Systems Science.

Project Email: 
Tamara_Ledley@terc.edu
Lead Organization(s): 
Award Number: 
1019721
Funding Period: 
Wed, 09/15/2010 - Fri, 08/31/2012
Project Evaluator: 
Susan Buhr
Full Description: 

This project is developing three inquiry-based, lab-focused, online Climate Change EarthLabs modules (focus is on the Cryosphere, Climate and Weather, and the Carbon Cycle) as a context for ongoing research into how high school students grasp change over time in the Earth System on multiple time scales. Climate literacy has emerged as an important domain of education. Yet it presents real challenges in cognition, perception, and pedagogy, especially in understanding Earth as a dynamic system operating at local to global spatial scales over multiple time scales. This research project confronts these issues by examining the challenges to high-school students' understanding of Earth's complex systems, operating over various temporal and spatial scales, and by developing research-based insights into effective educational tools and approaches that support learning about climate change and Earth Systems Science. The project is a collaborative effort among science educators at TERC, Mississippi State University, and The University of Texas at Austin.

The project uses a backward-design methodology to identify an integrated set of science learning goals and research questions to inform module development. Development and review of draft materials will be followed by a pilot implementation and then two rounds of teacher professional development, classroom implementation, and research in Texas and Mississippi. Research findings from the multiple rounds of implementation will allow an iterative process for refining the modules, the professional development materials, and the research program.

This project focuses on the design, development, and testing of innovative climate change curriculum materials and teacher professional development for Earth Systems science instruction. The materials will be tested in states with teachers in need of Earth Systems Science training and with significant numbers of low income and minority students who are likely to be hard hit by impending climate change. The research will shed light on the challenges of education for climate literacy.

Confronting the Challenges of Climate Literacy (Collaborative Research: Ledley)

Modeling Engineered Levers for the 21st Century Teaching of STEM (Collaborative Research: Schunn)

This project will develop three replacement units for biology and refine them through classroom testing. The units will be models of STEM integration by using the important concepts of proportional reasoning and algebraic thinking and engineering re-design to address big ideas in science while also promoting the learning of 21st century skills. The materials will be educative for teachers, and the teacher materials and professional development methods will work at scale and distance.

Project Email: 
birdy@pitt.edu
Lead Organization(s): 
Award Number: 
1027629
Funding Period: 
Wed, 09/01/2010 - Sun, 08/31/2014
Project Evaluator: 
Bill Bickel
Full Description: 

Research in biology has become increasingly mathematical, but high school courses in biology use little mathematics. To address this concern, this project will develop three replacement units for biology and refine them through classroom testing. The units will be models of STEM integration by using the important concepts of proportional reasoning and algebraic thinking and engineering re-design to address big ideas in science while also promoting the learning of 21st century skills. The materials build on existing work on the use of model eliciting activities and focus science and technology instruction on high-stakes weaknesses in mathematics and science. They address the scaling issue as part of the core design work by developing small units of curriculum that can be applied by early adopters in each context. The materials will undergo many rounds of testing and revision in the early design process with at least ten teachers each time. The materials will be educative for teachers, and the teacher materials and professional development methods will work at scale and distance.

Learning of science content will be measured through the use of existing instruments in wide use. Existing scales of task values, achievement goals and interest are used to measure student motivation. The work performed is guided by a content team; a scaling materials team; a scaling research team; the PI team of a cognitive scientist, a robotics educator, and a mathematics educator specializing in educational reform at scale; and the summative evaluation team lead by an external evaluator.

There is great interest in understanding whether integrated STEM education can interest more students in STEM disciplines. The focus on mathematics integrated with engineering in the context of a science topic is interesting and novel and could contribute to our understanding of integrating mathematics, engineering and science. The development team includes a cognitive scientist, a mathematics educator, teachers and scientists. The issues and challenges of interdisciplinary instruction will be investigated.

Modeling Engineered Levers for the 21st Century Teaching of STEM (Collaborative Research: Schunn)
Syndicate content