Teacher Content Knowledge

In this Spotlight, Julie Luft reflects on the various ways that teachers' subject-matter knowledge can be strained as teachers navigate the STEM education landscape and their careers as well as key questions that can guide innovation in content knowledge supports for teacher educators and teachers. In addition, nine DRK-12 projects discuss the approaches they're taking to developing and measuring different types of teacher content knowledge for improved STEM teaching and learning.

Subject Matter Knowledge is Important!

Julie Luft, Athletic Associaton Professor, University of Georgia

Aankit Patel

Any educator knows that content knowledge is important in teaching. In our work, we have been thinking about content knowledge as subject matter knowledge. For us, we see subject matter knowledge as knowledge in the discipline taught by a teacher. It often includes an understanding about how the discipline advances, along with what is known within the discipline. These areas are certainly intertwined because as inquiries are made, there is an advancement of knowledge.
 
Our recent work has been focused on understanding what happens when a teacher doesn’t have enough subject matter knowledge. There are certainly different ways in which teachers are constrained in their subject matter knowledge. For instance, science educators have long known that elementary science teachers often don’t have an adequate science background. As generalists, they have typically taken a limited number of subject matter courses.
 
Another variation can be found among new science teachers. A new science teacher may have taken the required number of subject matter courses to teach a topic, and may even be certified in the assigned area of instruction. Yet when the teacher steps into the classroom, the knowledge the teacher needs to draw upon may not be adequate in representing the topic of instruction. It’s not that the teacher doesn’t have the knowledge; it’s that their knowledge may not be coherent or organized in a way to support an understanding of the course content. For instance, a biology teacher may be a specialist in cellular biology, but still required to teach ecology...
 
Read the blog

Featured Projects

BioGraph 2.0 - Online Professional Development for High School Biology Teachers for Teaching and Learning About Complex Systems (NSF #1721003)

PI: Susan Yoon | Co-PIs: Eric Klopfer, Rebecca Maynard

Target Audience: In-service high school science teachers, specifically biology teachers

Target Teacher Content Knowledge (TCK): The project is developing and measuring teacher content knowledge around complex systems, modeling, and argumentation, through common biology topics.

Description: This project designed and developed an open-access, online system of professional development (PD) for high school biology teachers in order to build pedagogical competencies for teaching about complex systems and to support the application of those competencies in high school biology classrooms. The online PD was designed through a social capital framework that focused on developing teacher collaborative communities. Through emphasizing tie-quality, depth of interaction, trust, and access to expertise. The goals of the research are to study the effects of collaborative learning techniques in an online space and to compare teacher and student learning with results from a previous iteration of the PD that occurred in a face-to-face setting.

Measures of TCK: To measure teacher content knowledge, we are using a project-developed survey as well as interviews and classroom observations with a subset of teacher participants. Existing tools for measuring teacher content knowledge in the field of complex systems are limited due the variability of theoretical frameworks that education researchers have used (see Yoon, 2018b). Thus, for the last decade or so, we have constructed and used consistently several assessment protocols related to TCK that have appeared in various publications (e.g., Yoon et al., 2017).

Theoretical Framework: Our research is grounded in several educational theories about teaching and learning. These include theories and studies about social capital (e.g., Coburn & Russell, Coleman, 1988; Yoon et al., 2017, Yoon, 2018a)) teacher learning and characteristics of high quality teacher PD (e.g., Cochran-Smith & Lytle, 1999; Darling-Hammond et al., 2017; Desimone, 2009), complex systems teaching and learning (Yoon et al., 2018; Yoon, 2018b), online learning and teacher PD (e.g., Dede et al., 2009; Fishman et al., 2013), and transactivity (e.g., Vogel et al., 2016).

Initial Findings Related to TCK: Teachers entering our PD expressed concerns about a lack of knowledge and experience with complex systems modelling and a desire to learn more about strategies for cultivating complex systems mindsets among their students. Teachers also expressed concerns about their own pre-existing abilities to guide and support students through computer-based modelling simulations. Our PD aimed to not only build more developed understandings of complex systems among the teachers, but also to equip the teachers with the skills and knowledge to guide students through computer-based complex systems modelling. Importantly, we wanted to develop mechanisms for teachers to develop their TCK collaboratively vis-a-vis interactions with other teachers and expert peers.

Key Challenge: Most large online course platforms such as edX are not designed in a way that technologically supports participant collaboration and community building. In designing for social capital development, there are a number of challenges based on the limitations of the edX platform such as the structure and accessibility of the Discussion Boards and the lack of robust profile and tracking pages. The asynchronous nature adds to these challenges by adding a time component that can leave participants feeling isolated and makes it more difficult to build community among the participants. We have been working on various scaffolds such as discussion prompts to address this challenge and have demonstrated effects in increased collaboration.

Product: BioGraph: Teaching Biology Through Systems, Models, and Argumentation is a free online edX course for science educators. This course is co-taught by a teaching team from MIT and the University of Pennsylvania. The course is a six-week, interactive, collaborative, professional development that you can access anywhere, anytime. Learn how to use online simulations to develop core biology content and practices, aligned with the NGSS. Join an ongoing community of interested expert and novice teachers. The course provides a cutting edge curriculum that is ready to implement and has been shown to increase student learning and engagement in biology. The online simulations include optional entry-level coding and the course provides support for learning to teach this with students.

The course will run three sessions this summer: May 6th - June 27th, June 3rd - July 25th and July 8th - August 29th. Register (opening soon!) and receive updates as the course gets started. 

CAREER: Investigation of Beginning Teachers' Expertise to Teach Mathematics via Reasoning and Proof (NSF #1941720)

PI: Orly Buchbinder

Target Audience: The project targets pre-service secondary mathematics teachers and follows them through a yearlong internship and the first two years of autonomous practice. In addition, the project targets in-service teachers, who interact with the project participants in the internship sites and in the schools of practice. Together, the beginning teachers and in-service teachers will participate in the online professional learning community focused on improving teachers’ knowledge and practices of integrating proof and proving into their mathematics classrooms.

Target TCK: The targeted teacher knowledge is Mathematical Knowledge for Teaching Proof (MKT-P) which comprises knowledge of logical aspects of proof and proof-specific pedagogical content knowledge.

Description: Reasoning and proof are essential for teaching mathematics meaningfully, but are challenging to teach and learn. While much is known about those challenges, less in known about what knowledge, dispositions and practices do teachers need to engage students in proving. Even less is known about how beginning teachers develop this knowledge and how it translates to classrooms. This project follows beginning mathematics teachers from the last year of their program, through the yearlong internship and two years of individual practice, to study how teachers’ knowledge, dispositions and proof-related classroom practices evolve over time. The project also examines how this  professional knowledge is affected by schools’ sociocultural contexts and professional supports, specifically, a professional learning community which brings together the beginning teachers, cooperating teachers, mentors, the novice teachers and any colleagues in their respective schools who seek to improve their teaching of mathematics by integrating reasoning and proof.

Measures of TCK: The project employs multiple measures to capture the beginning teachers’ MKT-P, dispositions and proof-related practices over time. These include MKT-P questionnaire, Dispositions towards Proof Survey, and Lesson Enactment Rubric, all of which have been developed in a closely related NSF-funded project #1711163 (PI’s Orly Buchbinder & Sharon McCrone). In addition, the project will develop interview protocols to gauge the beginning teachers’ perspectives on integration of proof in their classrooms, and student engagement with proof and proving.

Theoretical Framework: The project adopts a situative perspective on teaching and learning (Borko et al., 2000; Peressini et al., 2004). This perspective conceptualizes knowledge as situated within physical and social contexts in which it develops through active participation in social practices; while learning is conceptualized as increased sophistication of participation in these social practices. The situative perspective attends to both the individual teachers and to the physical and social systems in which they participate.

To research teachers’ proof specific knowledge and classroom practices, as well as teachers’ dispositions towards proof and its teaching at the secondary level, the project uses the MKT-P framework (Buchbinder & McCrone, 2019, in press). In addition, the project examines the influences of sociocultural context of the school and professional support on teachers’ knowledge, dispositions and proof-related practices.

Methodology: This is a longitudinal multi-case study (Yin, 2017). The project develops six in-depth case studies of beginning teachers learning to integrate reasoning and proof in their own practice. The data collection spans three and a half years and uses with multiple instruments such as MKT-P questionnaire, Dispositions towards proof survey, classroom observations, interviews, student data and teacher generated artifacts . The development of the case studies will be followed by a cross-case synthesis to develop analytic (rather than statistical) generalizations, which are carefully composed theoretical statements and conjectured relationships extending beyond the original case studies.

Initial Findings Related to TCK: The project will begin in June 2020, and will include three phases: (1) exploring teacher learning in the university setting, specifically in the capstone course Mathematical Reasoning and Proof for Secondary Teachers, (2) a yearlong internship, and (3) first two years of individual practice. The capstone course in the first phase of the study was developed under NSF grant #1711163 (PI’s Orly Buchbinder and Sharon McCrone). The results of that study showed that prospective secondary teachers who participated in the course improved their MKT-P and dispositions towards proof. In addition, they have developed practical skills in designing and enacting proof-oriented activities that integrate logical aspects of proof with mathematical content of traditional US secondary curriculum. I am excited to follow these beginning teachers, and those who will take this course in the Fall of 2020 beyond their teacher education program to see how their knowledge, dispositions and proof related practices develop.  

Products: The current project, which will begin in June 2020, is grounded in my ongoing work on developing educational materials for assessing and enhancing secondary teachers MKR-P. Those efforts have been supported by NSF grants #1316241 (PI’s Daniel Chazan and Patricio Herbst) and #1711163 (PI’s Orly Buchbinder and Sharon McCrone). Two instructional modules: “Who is right?” and “What can you infer from this example?” can be found here along with a list of related publications.

Additional publications related to supporting teachers in integrating reasoning and proof in secondary classrooms can be found on orlybuchbinder.com, and publications related to the instructional materials in the current project include:

Buchbinder, O. & McCrone, S. (In press). Preservice Teachers Learning to Teach Proof through Classroom Implementation: Successes and Challenges. Journal of Mathematical Behavior. 

Buchbinder, O. (2018). “Who is right?” What students’ and prospective teachers’ responses to scripted dialog reveal about their conceptions of proof. In R. Zazkis & P. Herbst (Eds.), Scripting approaches in mathematics education: Mathematical dialogues in research and practice (pp. 89-113), New York, NY: Springer doi.org/10.1007/978-3-319-62692-5_5

Buchbinder, O., & Cook, A. (2018). Examining the mathematical knowledge for teaching of proving in scenarios written by pre-service teachers. In O. Buchbinder & S. Kuntze (Eds.). Mathematics Teachers Engaging with Representations of Practice (pp. 131-154). Springer, Cham. doi.org/10.1007/978-3-319-70594-1_8

Buchbinder, O., Ron, G., Zodik, I. & Cook, A. (2017). What can you infer from this example? Applications of on-line, rich-media task for enhancing pre-service teachers’ knowledge of the roles of examples in proving. In A. Leung and J. Bolite-Frant (Eds.), Digital Technologies in Designing Mathematics Education Tasks – Potential and Pitfalls. (pp. 215-235). Springer, Cham. doi.org/10.1007/978-3-319-43423-0_11

Collaborative Math: Creating Sustainable Excellence in Mathematics for Early Childhood Programs (NSF #1503486)

PI: Jennifer McCray | Co-PIs: Erika Gaylor, Ximena Dominguez, Erin Reid

Target Audience: Head Start and other programs that serve children between the ages of 3 and 5

Target TCK: Big Ideas of Early Mathematics (2014): Teachers’ knowledge of the central mathematical concepts preschool children need opportunities to construct

Description: Collaborative Math mobilizes entire early childhood sites—all teachers, assistant teachers, and instructional leaders—to promote the development of mathematical thinking among children. Participants rediscover and clarify for themselves the important abstractions that are the basis of early mathematics, and are coached to foster their development through routines, games, book-reading, and other developmentally appropriate activities. Instructional Leaders attend Learning Labs with their staff and follow these with an additional Leadership Academy meeting, to deepen content knowledge and gather support and ideas for on-site implementation. Between Labs, coaches at each site provide consultation to instructional leaders and group coaching on the implementation of new activities to each set of classroom teachers. Instructional leaders shadow coaches as they visit rooms, eventually taking greater responsibility for providing group coaching themselves. By emphasizing leadership and the entire site in these ways, Collaborative Math is meant to create greater possibilities for sustainability of the program.

Measures of TCK: To measure shifts in teachers’ content knowledge, Collaborative Math utilizes the Preschool Math – Pedagogical Content Knowledge survey. Based on an interview with established reliability and construct validity (McCray & Chen, 2012), this online survey asks teachers to read through two preschool play scenarios and identify mathematical thinking and learning opportunities they present.

We developed this measure because of our conviction that early math teaching opportunities are quite distinct from their later elementary-level counterparts. Children first begin to be able to document mathematical thinking in kindergarten, and many important elementary-level teaching tasks exploit the split between mathematical procedures and concepts that can result. There are already some useful tools for assessing this type of teacher thinking.

However, just because preschool children are not yet good at documenting their mathematical thinking, that does not mean it does not exist. In preschool, mathematical thinking is represented through objects, actions, and words. Because of this, teaching preschool math requires on-the-spot responses, and the awareness of key ideas that allows teachers to take full advantage of situations as they arise. While these qualities are hallmarks of all good teaching at every developmental level, they are particularly key before children are able to write and record.

We are still developing the coding system for our survey, but the interview version of this tool correlated with both teaching practices and child outcomes.

Products: The Idea Library is a set of videos and articles we have developed to help teachers and teacher educators think about early math. In particular, we have two video series that relate to teacher content knowledge. The Focus on the Child series presents clinical interviews with young children that help teachers see and hear their thinking; these help make clear the kinds of interests children bring as well as the confusions that can occur in their thinking as they develop. We also offer a Focus on Collaboration video series which shows processes for embedded professional development among teachers via co-planning, peer observation, and looking at student work together. While not topically oriented to content knowledge specifically, such processes are a powerful mechanism for deepening teacher thinking about mathematics. 

Developing and Validating Assessments to Measure and Build Elementary Teachers' Content Knowledge for Teaching about Matter and Its Interactions within Teacher Education Settings (NSF #s 1813254, 1814275)

PIs: Deborah Hanuscin, Jamie Mikeska | Co-PIs: Emily Borda, Katherine Castellano, Daniel Hanley

Target Audience: Our project focuses on developing tools to support elementary teacher educators and the preservice elementary teachers that they work with in teacher preparation programs, either in science methods or science content courses.

Target TCK: Our project is creating tools to support the measurement and development of content knowledge for teaching (CKT) -- which is the specialized knowledge teachers use to engage in the work of teaching elementary science.

Description: Teachers need specialized knowledge of the content that allows them to engage in the work of teaching science. That specialized knowledge can be developed in teacher education; however, it requires assessment tools and resources that can enable teacher educators to measure and build preservice teachers’ content knowledge for teaching (CKT). Our project is responding to this need by developing tools that are designed to target the content knowledge that elementary science teachers use to engage in the work of teaching about the structure and properties of matter, such as eliciting student ideas and selecting instructional activities that meet specific instructional goals. The first tool is an assessment that can be used to measure CKT proficiency over time. The second tool is a set of educative curriculum materials for teacher educators to use to build preservice teachers’ CKT.

Measures of TCK: A main component of our project is developing a summative assessment to measure preservice elementary teachers’ content knowledge for teaching (CKT) about matter and its interactions. We aimed to develop a summative assessment instrument that could be used on a large-scale to examine the nature and development of science teachers’ CKT across multiple sites and over time. Items were aligned to one of five sub-content area strands (e.g., properties of matter; conservation of matter; etc.) and one of seven Work of Teaching Science instructional tools (e.g., scientific models and representations; scientific investigations; etc.). All items were automatically scorable but included technology-enhanced items, such as drag-and-drop, inline choice, and grid items, as well as traditional multiple-choice single-select items. As part of the instrument development process, we conducted a small-scale pilot and a large-scale field test.

The pilot phase involved administering three batches of roughly 30 to 60 items to the same group of about 200 elementary preservice teachers, recruited from a recent pool of Praxis test-takers. We conducted a classical psychometric item analysis to assess the functioning of the items, inform item revisions, and to build our field test form. The average item difficulty and item-total correlations were .48 and .30, respectively. Very difficult (low proportion corrects) or low discriminating (near zero or negative item-total correlations) were either dropped from consideration for the field test form or revised if the reason for the poor item statistics was readily apparent.

The field test phase involved administering a 60-item form aligned to our test blueprint—specifications of numbers of items by each sub-content strand and each Work of Teaching Science instructional tool—to about 800 pre-service teachers, which were also recruited from a recent pool of Praxis test-takers but distinct from the pilot sample. All items were previously piloted but due to item revisions some could perform differently in the field test. Accordingly, a classical psychometric item analysis was first conducted to remove any poor functioning items and decide on the final form that will be used in the second phase of this project in teacher educator classrooms. We removed eight items for statistical and substantive reasons to create a 52-item final form still aligned to our test blueprint (the proportion correct ranged from .24 to .92 with a  mean of .59 and item-total correlation ranged from .2 to .56 with a mean of .37 for the final 52 items selected). We used these 52 items to conduct exploratory factor analysis (EFA) and dimensionality analyses to determine how best to scale the test and what, if any, (distinct and reliable) subscores by sub-content area and instructional tool it supported for reporting. The EFA and multidimensional Item Response Theory (MIRT) models all provided strong evidence of unidimensionality. The overall test reliability was high at .91.

Products: Our website, cktscience.org, showcases many of the products our project has developed to date. In the “About” tab of the website, we have links posted for three posters and one presentation given at national conferences. In the “Tools” tab of the website, we share examples of CKT matter items and CKT packets. Each CKT matter item situates the preservice teacher in a specific instructional situation and has them apply their content knowledge as they engage in a specific science teaching practice. Each CKT packet serves as an instructional resource that includes suggested lesson plans, readings, and resources to support teacher educators’ efforts to help elementary teachers develop their CKT in this science area.

DiALoG: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum (NSF #s 1621441, 1621496)

PI: Eric Greenwald, J. Bryan Henderson | Co-PIs: Audrey Beardsley, Megan Goss, P. David Pearson

Target Audience: In-service middle school science teachers and their students

Target TCK: Our hope is to allow teachers to more readily notice important aspects of critical speaking and listening skills as they transpire in their science classrooms.

Description: Speaking and listening are super important skills for collaboration and conveying important messages about who we are and what we know. Couple this with the fact that a new generation of standards for the teaching and learning of science champion the importance of evidence-based argumentation, and our project is keen on helping teachers support their students when engaging in oral argumentation in their classrooms. To do so, we have created a digital assessment system named DiALoG (Diagnosing the Argumentation Levels of Groups). DiALoG helps teachers recognize and assess 8 important aspects of oral argumentation as it happens in their classrooms and then allows them to take action on DiALoG scores with RMLs (Responsive Mini-Lessons) aligned with different levels of student proficiency for each of the 8 speaking/listing constructs assessed by DiALoG. RMLs are follow-up activities designed to further move the needle for these important facets of oral argumentation.

Measures of TCK: All participating teachers are completing a pre-/post- assessment of Teacher PCK for Argumentation (McNeill, González-Howard, Katsh-Singer, & Loper, 2016). This Pedagogical Content Knowledge (PCK) assessment consists of twenty multiple-choice items and two constructed-response items spanning across four different vignettes (i.e., teaching scenarios). This instrument allows us to look for differences in argumentation pedagogical content knowledge between participating teachers that do and do not use the DiALoG system in their classrooms.

Another form of teacher knowledge we seek to measure in this study surrounds literature on professional vision (Goodwin, 1994; Sherin, Jacobs, & Philipp, 2011). Professional vision concerns what teachers, based on their background knowledge and experience, notice during classroom practice. With differing levels of experience, teachers may notice very different things in their classrooms. In our study, all participating teachers will be shown the same video of classroom argumentation where we provide software to pause and annotate the video at time points that are interesting/conspicuous to them. At the end of the study, all teachers will be asked to watch and annotate the same video again. The difference in between annotation sessions is that only half of participating teachers will be provided access to the DiALoG system. This permits us to look for differences in annotation patterns between teachers that do and do not use the DiALoG system. Our hypothesis is that through repeated classroom use of DiALoG and the 8 components of oral argumentation it encourages teachers to look for when using the system, teachers provided DiALoG will have enhanced professional vision regarding important facets of speaking and listening when observing students engaged in oral argumentation. Comparing annotation patterns at post-test allows us to operationalize this idea and thereby put our hypothesis to the test.

Theoretical Framework: Oral arguments consist of both the content voiced by students and the dialogical process of generating and working with that content—what Erduran, Simon, and Osborne (2004) distinguish as argument and argumentation, respectively. This theoretical distinction allows for two bundles of assessment items—one group of items which assess the substantive content of what students say when engaging in activities designed to promote scientific arguments (intrapersonal component), and the other group of items to gauge the quality of social interaction between interlocutors during these same activities (interpersonal component). The figure below depicts a schematic of the framework guiding the development of DiALoG. Arrows denote the reflexive relationship between different intrapersonal and interpersonal aspects of oral arguments. Item development was guided theoretically by the work of Michaels, O’Connor, and Resnick (2008) on Accountable Talk. The Michaels et al. (2008) notions of Accountability to the Standards of Reasoning and Accountability to Knowledge guided the assessment of argument products (left side of the figure). That is, to evaluate the substantive content of oral arguments, we created items measuring the degree to which students were accountable for both the logical requirements of a valid argument, in addition to the scientific accuracy and relevance of their utterances. As for the assessment of argumentation processes (the right side of the figure), items were guided by the Michaels et al. notion of Accountability to the Learning Community, which emphasizes respect for, and critical attention to, the contributions of others so that ideas can be built upon one another.

Methodology: The project addresses two research questions:

  1. How can DiALoG be refined to provide a formative assessment tool for oral argumentation that is reliable, practical and useful in middle school classrooms?
  2. How does the use of DiALoG affect teacher formative assessment practices around evidence-based argumentation, when implementing science units designed to support oral argumentation? 

In order to answer these questions, the project is engaged in a mixed-methods study that includes the following activities:

  • A randomized controlled trial with 100 teachers, all of whom use the same argumentation-rich middle school science curriculum: 50 will teach with DiALoG, 50 will teach the same curriculum without DiALoG.
  • A close focal study with a separate group of 12 teachers are the subject of additional data collection and analyses. While not included in the RCT analyses, focal teachers will complete all RCT data collection activities, in addition to:
    • Video-recorded classroom observations with follow-up ‘debrief’ interviews. For this, we utilize a non-participant observation method to observe the N=12 focal teachers as they facilitate oral argumentation activities for each of the three Amplify Science units. In conjunction with an observation protocol derived from the Reformed Teaching Observation Protocol (RTOP) (Sawada, et al., 2002), data were collected as field notes, cross-checked across observers, and then expanded to classroom data reports to capture the co-constructed and agreed-upon dynamics of teacher implementation of oral argumentation. Furthermore, scores from the RTOP can be used as a convergent validity check with DiALoG, namely that there should be a positive association between scores generated by RTOP and scores generated by DiALoG.
    • Retrospective interviews with each focal teacher after completion of intervention. 
    • Analysis: Interview transcripts and video files are currently being analyzed using emergent coding techniques (Altheide, Coyle, DeVriese, & Schneider, 2008; Stemler, 2015), with the ultimate goal of identifying and demonstrating strong inter-rater reliability for a set of codes. These codes will then be used to identify salient themes and patterns in teacher’s use of DiALoG and the impacts of that use, including the nature and extent of any changes associated with the use of the DiALoG System.
  • Measures
    • PCK measure (discussed above)
    • Epistemology pre/post Teacher survey
    • FA practices survey
    • instructional log to capture instructional context and implementation variation across RCT teachers
    • Professional Vision pre/post (discussed above)
    • Student views of argumentation pre/post survey

Initial Findings Related to TCK: Placing a novel digital tool in the hands of teachers can mitigate possible inclinations to maintain control of classroom talk. We found on multiple occasions that our digital scoring tool prompted teachers to restrain from interjecting into student conversations as readily as they were accustomed to, and when interjections were made, they were more based on feedback from the assessment.

While the DiALoG system is intended to be a formative assessment that provides a digital scoring tool that is to be used to guide the selection of follow-up RMLs, multiple pilot teachers used the assessment in a summative fashion. When instructors find themselves preoccupied with working from a perceived need to assign individual grades in a higher-stakes summative context, this can divert instructor attention away from more important global trends in classroom talk.

Pilot teachers noticed gaps in their own understanding of more nuanced aspects of student discussion and felt the DiALoG system had sharpened their focus on specific discourse skills (or lack thereof). For example, teachers felt that the system called their attention to critique and co-construction – skills that they would not have looked for on their own. This suggests that regular use of digital tools encouraging teachers to attend to specific aspects of classroom practice may enhance teacher professional vision, i.e., what they are actually able to notice.

Key Challenge: The greatest challenge has been recruiting a sufficient number of teachers for the RCT. In fact, we needed to delay the launch of the RCT 6 months in order to meet our recruitment needs. Conversations with teachers and districts suggest that instructional changes in response to their locale’s adoption of NGSS curricular materials has meant reluctance to add other new things to the mix. This challenge is partially mitigated by the fact that the DiALoG system is, and is perceived by teachers to be, well-aligned with the NGSS’s attention to argumentation as a core practice of science and engineering.

Products: 

Focus on Energy: Preparing Elementary Teachers to Meet the NGSS Challenge (NSF #s 1418052, 1418211)

PIs: Sara Lacy, Lane Seeley | Co-PIs: Nathaniel Brown, Kara Gray, Amy Robertson, Roger Tobin

Target Audience: Inservice upper elementary teachers

Target TCK: Focus on Energy is researching how elementary teachers gain skills in identifying, tracking, representing and reasoning about forms and flows of energy.

Description: Current standards call for students to start learning foundational energy ideas in elementary school. To prepare teachers to meet this challenge, Focus on Energy developed classroom activities, a workshop and ongoing professional learning, teacher support materials, and assessments. These resources address significant challenges in energy learning:

  • Researchers have identified four key energy themes (forms and transformation, transfer, dissipation and degradation, and conservation) that cannot be learned sequentially or in isolation. Focus on Energy instruction weaves together these conceptual themes and advances them in tandem.
  • Energy is inherently abstract – it cannot be directly observed or measured – which means that tracking energy flow requires both reasoning from observable evidence and arguing by inference from a model of energy. Through increasingly complex investigations, learners use a common language, set of energy questions, and representational tools to collectively build a model of energy and learn to use it to construct explanations of energy flow.

Measures of TCK: We have found that teachers and students share many of the same initial ideas about energy and we have productively used some of the same assessments for both groups. Instruments for formative assessment of TCK are included in the Focus on Energy curriculum and can be found in the Focus on Energy resources.

Some of the additional instruments we found useful for teacher pd can be found in the Focus on Energy professional learning resources. Examples are:

  • Steel ball and ping-pong ball for motion energy
  • Banana Bread for thermal energy

Summative Assessments: The Sparklz assessment is an open-ended probe of energy tracking in a toy.

Theoretical Framework: Our approach to teaching and learning about energy in the early grades draws on research relating to three key conceptual strands: 1) Science as practice; 2) Learning Progressions; and 3) Modeling-based teaching and learning.

Methodology: We did not design the project as an efficacy study, but rather as a design-based research development project and, we hoped, a proof-of-principle demonstration. As the project proceeded, we observed what we considered remarkable growth in both teachers’ and students’ abilities to track energy. We carried out a mixed methods investigation of student learning, but our investigation of the evidence for teacher learning is ongoing.

Initial Findings Related to TCK: The pre- and post- teacher workshop scores on an open-response knowledge test (see figure) show significant (p < 0.005, N=9) gains in all categories.

Key Challenge: Since reasoning about energy demands both knowledge about an abstract concept and the ability to integrate and apply that knowledge meaningfully, a useful practical assessment tool must go beyond multiple choice items to see not just what learners know, but what they can do with that knowledge (Lee & Liu 2010). We developed an open-ended assessment of energy knowledge and reasoning that is fun and engaging, can be scored quickly and reliably, and probes both  basic knowledge about energy and the ability to use that knowledge to track energy flow in a real system.

Products: The project website, focusonenergy.tercu.edu, includes curriculum, resources, and facilitator’s guide for a 1-day professional learning workshop. Learn more about Focus on Energy through videos and publications.

Getting Unstuck: Designing and Evaluating Teacher Resources to Support Conceptual and Creative Fluency with Programming (NSF #1908110)

PI: Karen Brennan

Target Audience: 4th–6th grade in-service teachers

Target TCK: Getting Unstuck supports teachers in developing computational and creative fluency using the Scratch programming language.

Description: Getting Unstuck involves the design and evaluation of (1) an online learning experience for teachers to develop conceptual and creative fluency through short programming prompts (featuring the Scratch programming environment), and (2) educative curriculum materials for the classroom to support students with problem-solving strategies for computer programming.

K–12 introductory programming experiences are often highly scaffolded, and it can be challenging for students to transition from constrained exercises to open-ended programming activities encountered later in and out of school. Teachers can provide critical support to help students solve problems and develop the cognitive, social, and emotional capacities required for conceptually and creatively complex programming challenges. Teachers—particularly elementary and middle school teachers—often lack the programming content knowledge, skills, and practices needed to support meaningful programming experiences for students. Professional development opportunities can cultivate teacher expertise, especially when supported by curricular materials that bridge teachers' professional learning and students' classroom learning.

Measures of TCK: This project will evaluate teacher learning in the Getting Unstuck online experience using mixed-methods analyses of pre/post-survey data of teachers' perceived expertise and quantitative analyses of teachers' programs and evolving conceptual knowledge. Our team is addressing TCK through the framework of computational thinking concepts, practices, and perspectives (Brennan & Resnick, 2012). We are exploring teacher-created Scratch projects and developing visualizations to help teachers, as learners, understand more about what they have made, as well as to support teachers to learn from others by pointing to examples from specific near-peer projects.

Initial Findings Related to TCK: Through the online pilot of Getting Unstuck, teachers explored a number of Scratch programming prompts to develop their creative and computational fluency. We have since learned that teachers have adapted the programming prompts in a variety of ways for their own classroom contexts. Some teachers have designed opportunities for all students to engage in the same project prompt, while others have shared prompts with students as opportunities for extra credit. While teachers are excited to use these prompts to teach problem-solving strategies, they have asked for additional support in designing assessment methods, developing conceptual knowledge, and providing examples of student work. Our team at HGSE is working with a group of four elementary school teachers to design educative curriculum materials (ECMs) based on the Getting Unstuck summer 2018 experience.

Products: The Getting Unstuck website includes the original collection of Scratch programming prompts (which serves as the foundation for our research project) and a collection of debugging strategies. The following publications are also availabe:

  • Haduong, P., & Brennan, K. (2019). Helping K–12 teachers get unstuck with Scratch: The design of an online professional learning experience. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE ’19). Association for Computing Machinery, New York, NY, USA, 1095–1101. doi:10.1145/3287324.3287479
  • Brennan, K., & Resnick, M. (2012, April). New frameworks for studying and assessing the development of computational thinking. Paper presented at the annual meeting of the American Educational Research Association, Vancouver, Canada. Retrieved from https://web.media.mit.edu/~kbrennan/files/Brennan_Resnick_AERA2012_CT.pdf

Knowledge Assets to Support the Science Instruction of Elementary Teachers (ASSET) (NSF #1417838)

PI: Sean Smith | Co-PIs: Robert Esch, Courtney Plumley

Target Audience: Elementary teachers, particularly upper elementary

Target TCK: ASSET explored methods for generating canonical pedagogical content knowledge (C-PCK) in two areas:  (1) small particle model of matter and (2) interdependent relationships in ecosystems.

Description: Pedagogical content knowledge (PCK) has the potential to shape instruction, teacher professional learning, and instructional materials.  When the field speaks about PCK in terms of these benefits, a particular kind of PCK is envisioned.  We refer to this kind of PCK as “canonical” to convey that it is widely accepted by the field and transcends context.  However, examples of canonical PCK are lacking. In ASSET, we explored the possibility that the PCK held by teachers—“personal PCK”—could be compiled to grow the body of canonical PCK. We tested a model of personal-canonical PCK synergy, drawing on literature reviews and data collected directly from teachers.  We found that personal PCK does not accumulate to fill gaps in the canon, at least not within the topics we studied. Rather, personal PCK appears largely as variations on PCK themes already apparent in the literature.

Measures of TCK: In ASSET, we collected PCK from teachers, but we did not evaluate or measure it.  However, our efforts shared a common obstacle with those who have attempted to measure PCK.  As Shulman wrote in one of his earliest papers conceptualizing PCK: “Practitioners simply know a great deal that they have never even tried to articulate” (Shulman, 1987, p. 12).  Teachers seldom need to articulate their PCK for themselves, and they are rarely, if ever, asked to articulate it for others.  Consequently, their PCK tends to be tacit (Cohen & Yarden, 2009; Henze & Van Driel, 2015; Loughran et al., 2004, 2008).  Despite this formidable obstacle, testing our synergy hypothesis required eliciting and characterizing teachers’ personal PCK.  Like other PCK researchers, we found affordances and limitations in a survey approach.  Survey questions were based on the CoRe methodology (Loughran et al., 2004) used widely in studies of teacher knowledge (Alvarado et al., 2015; Williams & Lockley, 2012).

We administered a web-based survey to teachers from several states about their topic-specific PCK.  Some questions asked about teachers’ knowledge of student thinking; for example, “Please describe the ideas or misconceptions your students have that make it difficult for them to learn about the particle model of matter.”  Others asked about their instructional strategies; for example, “Please describe a question or activity you use to find out what ideas students already have about the interdependent relationships in ecosystems before you begin teaching about it.”  The survey allowed respondents to upload documents they used in their teaching, including laboratory activities and worksheets.  Respondents were also encouraged to share other resources, for example, online simulations and videos.  

The survey forced teachers to compartmentalize their knowledge (e.g., they responded to separate survey questions about student thinking patterns and instructional activities).  From an analysis standpoint, this feature was an affordance.  However, survey responses did not represent how different types of personal PCK related to each other.  As illustrated by the examples above, the survey asked respondents to describe misconceptions and instructional activities separately, rather than explain which activities they use to address their students’ misconceptions.  In addition, responses tended to be vague, lacking detail needed to characterize a teacher’s PCK adequately.   

We ultimately found a combined survey-and-interview approach most effective.  Teachers first completed the web-based survey.  We then conducted follow-up telephone interviews with survey respondents, during which we probed on each of their survey responses.  Before the interview, each interviewee received his or her survey responses by email and was encouraged to have them on hand during the interview.  The interview followed essentially the same structure as the survey, but researchers probed for elaboration of survey responses that were unclear and for connections among compartmentalized responses.  For example, a survey respondent may have written “I ask questions” when describing a particular activity.  During the interview, a researcher prompted the respondent to name the specific questions and asked for typical student responses.  Similarly, a survey respondent who provided only a sentence or two about an activity was asked to expand upon their description during the interview.  Interviewers also asked respondents how they used the resources that they had uploaded in their survey responses (e.g., lesson plans, student handouts). 

Theoretical Framework: We based our work on Gess-Newsome’s model of teacher professional knowledge described in this chapter: Gess-Newsome, J. (2015). A model of teacher professional knowledge and skill including PCK: Results of the thinking from the PCK Summit. In A. Berry, J. Loughran, & P. J. Friedrichsen (Eds.), Re-examining Pedagogical Content Knowledge in Science Education. Routledge.

Initial Findings Related to TCK: In framing our initial findings, we first acknowledge that canonical PCK is not universally accepted.  We attribute the resistance in part to semantics; the term “canonical” evokes strong negative reactions in some.  The idea that PCK exists outside of an individual is unacceptable to others.  But we suspect that most would agree our field has identified prominent, enduring patterns in students’ topic-specific thinking.  Young students tend to think that moving objects always stop, that dissolving solids cease to exist, that decaying matter just “goes away” without any biochemical action.  Similarly, the field has identified effective instructional strategies for helping students reconcile these ideas with accepted scientific concepts.  When students slide a block across progressively smoother surfaces, the experience (if well facilitated by a knowledgeable teacher) can challenge their idea that moving objects always stop.  Weighing the mass of a solid and liquid before and after dissolving suggests that the solid does not cease to exist.  And careful observations of composting can open students to the possibility of invisible processes they had never considered.  These are instances of what we call canonical PCK, but the term is not as important as what it represents—widely accepted knowledge about how students think and learn about a topic. 

In exploring the synergy hypothesis, we focused on the potential for accumulated personal PCK to fill gaps in canonical PCK.  We found that personal PCK does not accumulate to fill gaps in the canon, at least not within the topics we studied.  Rather, personal PCK appears largely as variations on PCK themes already apparent in the literature.  We gave little attention to the other aspect of synergy—that canonical PCK can become personal as one takes it up and uses it in teaching.  However, we see little evidence of this aspect of synergy either.  Reports from teachers suggest minimal exposure to empirical literature on student thinking, which is not surprising.  Attempts to synthesize the literature are infrequent, and attempts to make the knowledge accessible to teachers even less frequent. 

 We are unable to support either relationship in our synergy hypothesis, but a caveat is in order.  We synthesized canonical PCK by collecting, reviewing, and summarizing empirical and practitioner literature.  These tasks were straightforward, not substantially different from any other literature review.  Eliciting personal PCK from teachers was far more challenging.  We found the combined survey-and-interview approach promising, but we are uncertain that it adequately addresses the tacit nature of teachers’ PCK.  Other means of eliciting personal PCK may yield different findings, perhaps even evidence for the synergy hypothesis.

We did find complementarity between empirical and practitioner literature, at least for the interdependence topic.  Empirical literature focused heavily on student thinking, practitioner literature almost exclusively on instructional activities.  Together, they form a richer knowledge base for teaching the topic than either does alone.  What is lacking are efforts to (1) synthesize within and across these types of literature and (2) make the product accessible to teachers in a form they are likely to use.  Also lacking in either type of literature is knowledge about the affordances and limitations of an activity in terms of student thinking.

For more information, see the book chapter listed below.

Products:

Mathematics Immersion for Secondary Teachers (NSF #s 1719554, 1719555)

PI: Al Cuoco, Daniel Heck | Co-PIs: Matthew McLeod

Target Audience: Our audience is in-service 6th–12th grade teachers who are teaching a mathematics course. We also welcome district leaders, instructional coaches, school administrators, teachers in other grades, anyone who is interested in being immersed in doing mathematics for professional learning and for fun.

Target TCK: Our focus for TCK is the Mathematical Habits of Mind (or Mathematical Practices) within the context of Probability through Games and Connections between Algebra and Geometry.

Description: We engage mathematics educators who are interested in doing mathematics for learning and fun. A group of colleagues gather to work together and are connected via videoconference to a facilitator and 2 – 3 similarly gathered groups for synchronous collaboration. Analogous to face-to-face PD, each group works at their table as the facilitator “floats” to check-in and support their learning. Occasionally, the facilitator pulls the groups together so they can share ideas, questions, and understandings. The mathematics is presented through carefully crafted problem sets that lead to guided discovery of concepts. There are 9 sessions per course which are designed to offer a low-floor-high-ceiling concept so participants can engage readily and delve in as far as they are interested.  The sessions build the concepts from one to the next and contain content that we anticipate is novel to the participants, thus providing authentic mathematics learning experiences and building their content knowledge.

Measures of TCK: We measure TCK at three time points using the Assessing Secondary Teachers Algebraic Habits of Mind tool (see http://mhomresearch.edc.org/).

Theoretical Framework: Two theoretical frameworks ground our research. We view the professional opportunities teachers experience to develop TCK through Cultural Historical Activity Theory (CHAT), specifically the ideas of learning by doing, learning in community, and engaging in learning that pertains in specific ways to the professional work to which it is meant to apply. We view the TCK that is addressed through the lens of mathematical habits of mind for curriculum and teaching first described in Cuoco, Goldenberg, & Mark (1996).

Methodology: We use a two-group, delayed treatment, semi-randomized block design. Each local group was assigned to complete the MIST course during the first year (Cohort A) or second year (Cohort B) of the study. Some Cohort A participants also chose to complete a second course in the second year. All participants in both cohorts are completing measures at the beginning of the first year, at the end of the first year, and at the end of the second year. This design supports multiple comparisons within and across cohorts to examine initial impacts with either comparison group or prior year contrasts, and sustained impacts with and without additional participation beyond a single year.

Key Challenge: An important challenge for us has been appropriately incentivizing and supporting both consistent participation in the professional learning opportunity and fulfilment of expectations for data collection. Attrition from the study related to both participation and data collection has been an ongoing concern.

Products: Our website, mist.edc.org, includes videos that illustrate the work we are doing and two courses worth of problem sets that we have used in the project.

Additional Resources

References

Altheide, D., Coyle, M., DeVriese, K., & Schneider, C. (2008). Emergent qualitative document analysis. Handbook of emergent methods, 127-151.

Brennan, K., & Resnick, M. (2012, April). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American educational research association, Vancouver, Canada (Vol. 1, p. 25).

Borko, H., Peressini, D., Romagnano, L., Knuth, E., Willis-Yorker, C., Wooley, C., ... & Masarik, K. (2000). Teacher education does matter: A situative view of learning to teach secondary mathematics. Educational Psychologist, 35(3), 193-206. DOI: 10.1207/S15326985EP3503_5

Buchbinder, O. & McCrone, S. (In press). Preservice Teachers Learning to Teach Proof through Classroom Implementation: Successes and Challenges. Journal of Mathematical Behavior.

Buchbinder, O. & McCrone, S. (2018) Mathematical Reasoning and Proving for Prospective Secondary Teachers. Proceedings of the 21st Annual Conference of the Research in Undergraduate Mathematics Education, Special Interest Group of the Mathematical Association of America: San Diego, CA. (pp. 115-128). http://sigmaa.maa.org/rume/RUME21.pdf

Coburn, C. E., & Russell, J. L. (2008). District policy and teachers’ social networks. Educational Evaluation and Policy Analysis, 30(3), 203–235.                                                            

Coleman, J. S. (1988). Sociology and economic approaches to the analysis of social structure. The American Journal of Sociology, 94, S95–S120.             

Cuoco, A., Goldenberg, E. P., & Mark, J. (1996). Habits of mind: An organizing principle for mathematics curricula. The Journal of Mathematical Behavior, 15(4), 375-402.

Darling-Hammond, L., Hyler, M. E., & Gardner, M. (2017). Effective teacher professional development. Learning Policy Institute.

Dede, C., Ketelhut, D. J., Whitehouse, P., Breit, L., & McCloskey, E. M. (2009). A research agenda for online teacher professional development. Journal of Teacher Education, 60(1), 8–19.

Desimone, L. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181–199. 

Erduran, S., Simon, S., & Osborne, J. (2004). TAPping into argumentation: Developments in the application of Toulmin’s Argument Pattern for studying science discourse. Science Education, 88(6), 915–933.

Fishman, B., Konstantopoulos, S., Kubitskey, B. W., Vath, R., Park, G., Johnson, H., & Edelson, D. (2013). Comparing the impact of online and face-to-face professional development in the context of curriculum implementation. Journal of Teacher Education, 64(5), 426–438.

Goodwin, C. (1994). Professional vision. American Anthropologist, 96(3), 606–633.

Lee, H. S., & Liu, O. L. (2010). Assessing learning progression of energy concepts across middle school grades: The knowledge integration perspective. Science Education, 94(4), 665-688.

McNeill, K. L., González‐Howard, M., Katsh‐Singer, R., & Loper, S. (2016). Pedagogical content knowledge of argumentation: Using classroom contexts to assess high‐quality PCK rather than pseudoargumentation. Journal of Research in Science Teaching, 53(2), 261-290.

Michaels, S., O’Connor, C., & Resnick, L. B. (2008). Deliberative Discourse Idealized and Realized: Accountable Talk in the Classroom and in Civic Life. Studies in Philosophy and Education, 27(4), 283–297.

Peressini, D., Borko, H., Romagnano, L., Knuth, E., & Willis, C. (2004). A conceptual framework for learning to teach secondary mathematics: A situative perspective. Educational Studies in Mathematics, 56(1), 67-96. DOI: 10.1023/B:EDUC.0000028398.80108.87

Sawada, D., Piburn, M. D., Judson, E., Turley, J., Falconer, K., Benford, R., & Bloom, I. (2002). Measuring reform practices in science and mathematics classrooms: The reformed teaching observation protocol. School science and mathematics, 102(6), 245-253.

Sherin, M., Jacobs, V., & Philipp, R. (Eds.). (2011). Mathematics teacher noticing: Seeing through teachers' eyes. Routledge.

Stemler, S. E. (2015). Content analysis. Emerging trends in the social and behavioral sciences: An Interdisciplinary, Searchable, and Linkable Resource, 1-14.

Vogel, F., Kollar, I., Ufer, S., Reichersdorfer, E., Reiss, K., & Fischer, F. (2016). Developing argumentation skills in mathematics through computer-supported collaborative learning: The role of transactivity. Instructional Science, 44(5), 477–500.

Yin, R. K. (2017). Case Study Research and Applications: Design and Methods 6th Edition. Thousand Oaks, CA: Sage publications.

Yoon, S., Goh, S., Park, M. (2018). Teaching and learning about complex systems in K–12 science education: A review of empirical studies 1995–2015. Review of Educational Research, 88(2), 285–325.

Yoon, S. (2018a). Mechanisms that couple intentional network rewiring and teacher learning to develop teachers’ social capital for implementing computer-supported complex systems curricula. In S. Yoon and K. Baker-Doyle. Networked by design: Interventions for teachers to develop social capital. Routledge Press.

Yoon, S. (2018b). Complex systems and the Learning Sciences: Implications for learning, theory, and methodologies. In F. Fischer, C. Hmelo-Silver, S. Goldman and P. Reimann (Eds.) The International Handbook of the Learning Sciences (pp. 157–166). New York, NY: Routledge Press.

Yoon, S., Koehler-Yom, J., & Yang, Z. (2017). The effects of teachers’ social and human capital on urban science reform initiatives: Considerations for professional development. Teachers College Record, 119(4), 1–32.

Yoon, S., Anderson, E., Koehler-Yom, Evans, C., Park, M., J., Sheldon, J., Schoenfeld, I., Wendel, D., Scheintaub, H., & Klopfer, E. (2017). Teaching about complex systems is no simple matter: Building effective professional development for computer-supported complex systems instruction. Instructional Science, 45(1), 99–121.

Year