Paper

In addition to overall continuum ratings, evaluators provided ratings for each of seven component areas for professional development, including: the design of activities, their implementation, the disciplinary and pedagogical content, the professional development culture, the likelihood that the professional development would build participants' capacity to provide exemplary science/mathematics instruction, and the likelihood it would build their capacity to be leaders in science/mathematics instructional reform. The largest difference between the cohorts was in ratings for capacity-building for instruction, with Cohort 1 projects more likely to be highly rated in this area. (See Figure 11.) No evaluators rated a component area for their project at Level 1, and very few components were rated at Level 2.

Figure 11

Strengths of the LSC Projects

Evaluators were asked to describe their rationale for their ratings, particularly highlighting strengths of the professional development activities and areas in need of improvement. Aspects of the LSC professional development that were most often described by evaluators as strengths were:

• strategies for professional development;
• the skills, competence, and expertise of facilitators;
• the systemic focus on the broader science/mathematics education system;
• a culture conducive to and supportive of learning;
• relevant, high-quality content;
• models for effective development of teacher leaders; and
• reflection of project staff for program improvement.

Two features that distinguish this group of professional development projects from many past teacher enhancement projects are the systemic focus and the consistency of attention directed toward the use of exemplary curriculum materials. Evaluators commented frequently on these aspects of the LSC projects.

Evaluators of the LSC projects described a number of ways in which projects targeted the broader science/mathematics education system. For example, a requisite commitment for LSC funding is to serve all eligible teachers for a substantial number of hours. Many programs are choosing to begin by "exposing" large proportions of teachers to one or two kits and achieve at least a "mechanical" knowledge of the materials early on. One evaluator pointed out the benefit of this strategy as a step toward building support among stakeholders:

"Through the [teacher workshops], the Partnership is reaching larger numbers of teachers and providing exposure to inquiry-centered instructional approaches that can readily be used in elementary classrooms. These workshops, by increasing the number of teachers involved in the Partnership, are also expanding the cadre of stakeholders involved in the reform of science and mathematics."

Most projects are implementing curricula that the district has chosen or agreed to adopt, reinforcing the alignment of district curriculum and goals. One evaluator made this observation:

"[The district] has recently adopted FOSS (Full Options Science System) as its district science curriculum. Many of the pedagogy sessions and some of the content sessions encourage exploration of specific FOSS kits. By utilizing the district curriculum as a starting point for good hands-on science, [the project] is seen by participants as supporting the district-identified process to improve science teaching."

The LSC projects are also distinguished, as a group, by being highly reflective of current standards for best practice in professional development. Features described most often by evaluators of the LSC projects are described below:

• Relevancy of professional development activities to teachers' work in classrooms

  Evaluators found, for example, that activities in which teachers were engaged were closely linked to their needs as classroom teachers. Said one,

  "This professional development model establishes a different dynamic between practitioners and their professional growth. There is a relationship between what occurs inside and outside of their classroom. The school adopts math and science curricula considered exemplary and in the best interest of students, and complements the adoption with extended staff development to assist teachers in the implementation process."

  A strategy mentioned as particularly successful by another evaluator was allowing teachers to focus on grade-specific activities. This evaluator reported these teacher comments:

  "What I got a lot from in [the workshop] was when I sat with other fourth grade teachers and talked about how we implemented the kit"

  "The people who did kindergarten [sessions] were actual kindergarten classroom teachers so when they gave information it had been used and tested and worked. Those were very useful, very helpful."

• Modeling effective strategies

  Evaluators found that strategies advocated by the projects were modeled in professional development activities. Activities were characterized, for example, by active involvement of teachers and participatory hands-on activities; in many cases teachers worked on activities from modules that they would later use in their classes.

  Experiencing instructional strategies firsthand was particularly important for teachers being introduced to inquiry-based strategies, as described by this evaluator:

  "In order to teach in an inquiry-based environment, teachers must experience inquiry learning for themselves. The introductory summer institute provides the foundation upon which teachers must stand if they are to transform their teaching approaches to be truly inquiry-based. Two weeks enables participants to thoroughly explore a particular scientific phenomenon, and to come to conclusions of their own about why "x" works, drawing on their own experiential learning. In addition, teachers learn how a real classroom might function as an inquiry-based learning environment. This is often difficult to even imagine, given the relatively fixed structured nature of most elementary schools. In discussions and debates, the institute directors encourage participants to consider how to translate their experience back to their classrooms."

• Provides opportunities for teachers to reflect and talk with peers

  Time for reflection was incorporated into many of the projectsi professional development activities sometimes within individual sessions; sometimes as the major focus of entire sessions as in school-based peer group meetings to discuss the implementation of a kit. This comment is reflective of evaluators' (and participants') praise for projects' providing opportunities for reflection:

  "Overall, lead teachers, in interviews and in the post-institute evaluation surveys highly praised the academic year sessions-particularly noting the opportunities to share classroom practices, to practice inquiry, and to discuss national standards with peers. Through the [institute] lead teachers have opportunities for reflection, dialogue, and peer support."

• Strong components of support and follow-up

  Support throughout the academic year is a consistent feature of the LSC projects, and one that evaluators recognized as a strength. One evaluator, for example, highlighted the importance of providing opportunities for teachers to meet throughout the year for discussions with peers after actually trying out modules in classrooms.

  Projects have explored different strategies for providing on-going support to teachers. A number of projects that focus on kit-based curriculum have organized regular meetings in which teachers may gather to discuss the implementation of the kits. Some projects have worked to build the expertise of in-school teacher leaders who are available for individual support to their peers. Another follow-up strategy has been for teachers to develop personal action plans with support provided throughout the year by mentor teachers or project staff.

• Active involvement of teachers

  The LSC projects are clearly attentive to actively involving the teachers as key players in their own professional development. Evaluators described a number of mechanisms designed to engage teachers as active participants and leaders of professional development in their districts.

  A number of projects incorporate teacher-leader models designed to build capacity of teachers within the schools and districts, and to reach larger numbers of teachers. Other strategies for encouraging teachers' involvement include options for teachers to select from multiple professional development activities; facilitating active sharing, reflecting and collaboration in small groups; fostering school-based implementation built on "school-site learning communities"; and assembling action research teams that offer teachers the opportunity to network and share with each other.

Challenges Faced by LSC Projects

Interestingly, some of the same features highlighted as strengths by some evaluators were identified by other evaluators as areas of challenge for their projects. Aspects of the professional development most often described as needing increased attention for improvement were:

• Time for participant reflection;
• Attention to content;
• Attention to classroom application;
• Consistency in the skills and expertise of facilitators;
• Attention toward equity; and
• Teachers' use of available resources.

The overlap in features most likely to be mentioned as strengths when they are present and as areas in need of improvement when they are not include: time for reflection, attention to classroom application, high-quality content, and skill and expertise among facilitators. The following comments of evaluators are representative of the concerns expressed by some.

Regarding the need for increased time for reflection:

"Although there was active learning going on in this and other activities during the session, there were missed opportunities for reflection, individually and as a group on what was being taught and learned. That is, although teachers were asked good questions, for example comparing actual [data] with predictions and providing what they thought accounts for differences, the discussions tended to be very limited in time and scope."

"Complex ideas were demonstrated in an interesting way, but some of the teachers did not internalize the concepts. The minds-on portion of the workshop was missing. As a result, the teachers may have some difficulty developing assessment tools and questioning skills when introducing the concepts in their classroom. The workshop seemed more of a show than an integration of content, process, reflection and application. The teachers needed to reflect on the demonstrated concepts and perhaps discuss age-appropriate applications of the concept."

Regarding content-particularly finding an appropriate balance in the content and process of inquiry:

"To elevate the quality of the kit workshops still higher the focus needs to expand to include an emphasis not only on the procedures and organization of the kits, but on the science content and the conceptual development embedded within them. Sole focus on the mechanics of the kits does not allow teachers time to reflect on their teaching practice or the rationale for hands-on, inquiry-based science education. This may lead participants to view kit-based science teaching as an end rather than a means to exemplary inquiry-based science instruction."

Regarding attention to classroom application:

"The teachers needed more time to consider classroom applications of resources, strategies, and techniques introduced in the sessions. For example, in one of the sessions observed, the participants viewed a video called "A Private Universe" which illustrates how students often develop misconceptions about science concepts and suggests methods to identify and resolve these misconceptions. The teachers in this session, however, were unclear as to how to use this information in their own classrooms. The method demonstrated in the video, individual student interviews, was not an option for most teachers, and they needed time to discuss other strategies."

Regarding the skill and expertise of facilitators:

In some cases, lapses in quality of the professional development were due to facilitators whose expertise in content areas was greater than their knowledge of appropriate professional development strategies. In other cases the challenge was to achieve consistency among teacher leaders working with peers. For example, one evaluator made this observation:

"The quality of those sessions was somewhat variable. The presentations made in the fall for colleagues were exceedingly well-executed and reflective of best practice, keeping in mind that many of the activities had been modeled for them by [project staff] in their first summer of professional development. When they were on less familiar ground [the] sessions were not always as well-executed."

Regarding increased attention toward equity:

In addition to the general need for equity issues to "gain visibility", evaluators commented on the challenge of establishing equity as a priority for participants:

"A major focus of the professional development sessions during the summer and year is how to successfully implement the science curricula in a manner that is accessible to all students. In addition, this summer's institute had a series of workshops on cooperative learning as a direct attempt to address the issue of access for new and beginning teachers. Although [the project] is accomplishing some things in the area of equity, the associate superintendent for curriculum and instruction believes that much more needs to be done in this area as test scores for the African-American and Latino populations have not showed expected gains."

"One issue that must be dealt with in the future is the perception of teachers that they know all about equity already and they do not understand why time is spent on this topic when there are so many other content, assessment, and procedural topics that they would like to hear about in professional development sessions."

Regarding use of available resources:

Teachers' failure to use existing resources was described as a challenge for a number of the projects. In most cases the issue was that teachers were not taking advantage of opportunities available to them, as illustrated by these evaluators' comments:

"While monthly meetings, content courses, and [teacher] assistance is available to classroom teachers, many do not make use of these resources..."

"Generally, [the project] is providing high-quality professional development activities by teachers are not feeling supported in the classroom implementation of what they have learned. One area where [the project] attempted to address this need was the creation of kit clubs. Unfortunately, not many teachers attended these even though they would have been paid $20 for every two-hour after-school session they attended."

1 One project in Cohort 2 did not collect professional development observation data in 1996 because of a late start-up date; this project's baseline questionnaire data are included in the cross-site analyses.

2 HRI re-rated all 1995-96 professional development observations based on the updated continuum developed for the 1997 observation protocol. Evaluators were asked to verify these ratings and given the opportunity to make changes as needed. Ratings reported here have either been provided by or verified by the evaluators.

3 A number of projects reported observing professional development sessions in addition to those rated for the core evaluation; these additional observations were considered by evaluators when making overall ratings of the project's professional development system.

4 A project placed at Level 3.5 was included with the Level 3 group in this report.