Paper

Highlights of the Year Two LSC Cross-Site Report

Summary

In its second year of implementation, the Local Systemic Change Initiative supported 26 projects involving a total of 121 districts throughout the United States. LSC projects are characterized by their focus on adoption and use of exemplary science and mathematics instructional materials. These projects are also distinguished from teacher enhancement projects of the past by their systemic focus, including targeting all teachers in participating schools rather than volunteer teachers, and reviewing district and school policies and practices to determine the extent of alignment with the reform vision. The LSC Initiative is proving to be an effective mechanism for providing high quality professional development to large numbers of teachers. Moreover, by operating in a systemic context, the LSC projects have the potential for greater and more sustained impact than has typically been the case in teacher enhancement projects involving individual teachers from a large number of schools and districts.

Data on the quality and impact of the LSC projects come from teacher and principal questionnaires, teacher interviews, and observation of both professional development activities and science and mathematics classrooms.

Project evaluators observed a number of professional development sessions in each project using a standardized protocol developed by Horizon Research, Inc. These sessions generally received high marks for the quality of their design and implementation, for the appropriateness of the disciplinary and pedagogical content, and their inclusive, collegial culture. Overall, the LSC programs appear to be well-aligned with national science and mathematics standards, and in particular those for professional development; projects generally attend to important science and mathematics content and pedagogy in a spirit of inquiry, reflection, and continuous improvement.

Evaluators noted a number of ways in which the LSC professional development reflects current standards for best practice in professional development. Among these are the relevancy of professional development activities to teachers' work in classrooms, the way in which professional development typically modeled effective pedagogical strategies; the opportunities for teachers to reflect individually and with their colleagues; and the attention to follow-up support as teachers worked on implementing standards-based curriculum materials in their classrooms.

At the same time, evaluators in some projects have noted a need for greater attention to the balance between content and pedagogy when helping teachers learn to implement exemplary instructional materials; "going through the activities" is valuable, but teachers also need opportunities to explore the conceptual underpinnings of these activities in more depth. Also, evaluators note that projects using inexperienced professional development providers need to pay greater attention to ensuring consistently high quality service delivery; this issue was particularly important in projects where university scientists and/or lead teachers were responsible for conducting professional development sessions.

Teachers who had participated in LSC professional development were generally quite positive about their experiences, indicating that they had become more confident in teaching science as a result. Moreover, teachers who have participated extensively in science and mathematics professional development report higher levels of preparedness and more frequent use of standards-based instructional strategies than do teachers with less professional development involvement.

It is important to note that most teachers in LSC districts reported very positive attitudes towards science and mathematics reform, indicating that they recognize the value of many standards-aligned instructional strategies. However, there is a large disparity between the perceived importance and the preparedness teachers feel for implementing reform strategies in their classrooms. Similarly, sizable proportions of teachers report feeling less than well prepared to teach a number of science and mathematics topics they are expected to teach.

There was a marked consistency in the picture of classroom instruction that emerged from teacher questionnaires and classroom observations. Teacher presentations are quite common in the LSC districts, with most observed lessons including a formal presentation by the teacher. While most observed lessons also included some kind of hands-on/investigative activity, these typically involved students following detailed instructions to answer predetermined questions, rather than allowing the students to pose questions and design their own investigations to address these questions.

In keeping with NSF's focus on system-wide reform, the core evaluation questionnaires and interviews asked teachers about the support they receive from parents, principals, and other teachers in their schools. Teachers in the LSC districts generally feel supported by other teachers in their schools to try out innovative ideas in science and mathematics teaching, but they rarely have time during the regular school week to work with one another. In fact, lack of time to work with other teachers, inadequate funds for purchasing equipment and supplies, lack of access to computers, and the need for more planning time headed the list of problems for science and mathematics instruction reported by both teachers and principals.

Evaluators were also asked to describe the extent to which district policies and resources were aligned in support of the LSC reforms and the likelihood that the reforms would be sustained after the NSF funding period had ended. Most projects were rated at Level 3 (out of a possible 5) in terms of district support, indicating that district policies were in transition toward a more supportive context. Many spoke of commitments the districts had made to the LSC reforms, including purchasing kit-based instructional programs and establishing centers for maintaining and refurbishing the kits.

Finally, while many LSC projects have begun the process of developing lead teachers and otherwise increasing internal capacity to provide high quality professional development, many of the LSC districts do not yet have the mechanisms or resources to sustain these systems.

The core evaluation of the Local Systemic Change Initiative requires the energy, efforts, and insights of a very large number of people.

Conrad Katzenmeyer in the National Science Foundation's (NSF) Division of Research, Evaluation, and Communication; Joyce Evans, Susan Snyder, and Diane Spresser in NSF's Division of Elementary, Secondary, and Informal Education (ESIE); and Joy Frechtling of Westat, Inc. were instrumental in the design and implementation of the core evaluation system. Numerous other NSF/ESIE program officers provided valuable assistance throughout the process, as well.

Within HRI, in addition to the authors of this report, Joan Pasley, Ruth Pershing, Ben Kurgat, Eugene Soar, Scott Hanrath, Jarrod Jenzano, Alison Schaff, Carol McDonald, Susan Hudson, Claudia Templeton, and John Supovitz were responsible for various tasks essential to the production of this report, from instrument design, to data processing and analysis, to report production.

Most notably, this report would not have been possible without the efforts of the LSC project lead evaluators, who persevered as we attempted to fine-tune the core evaluation system. And special thanks are due to the thousands of teachers throughout the 121 participating districts who took time from their busy schedules to provide information about their science and mathematics teaching.