Communication Center  Conference  Projects Share  Reports from the Field Resources  Library  LSC Project Websites  NSF Program Notes
 How to Use this site    Contact us  LSC-Net: Local Systemic Change Network
Educational Reform & Policy

Professional Development

Teaching and Learning

LSC Papers and Reports

Cross Site Reports

LSC Case Study Reports

Papers / Presentations authored by LSC Members

About LSC Initiatives and their impact




LSC Year Two Cross-Site Report

author: Iris R. Weiss, Kathleen A. Rapp, Diana L. Montgomery
published in: Horizon Research
published: 02/04/1998
posted to site: 02/04/1998

Teacher Ratings of the Quality of LSC Professional Development

As part of the core evaluation, evaluators of Cohort 1 projects were asked to interview a random sample of 10 teachers who had participated in the project's professional development activities. Among the teachers interviewed, over two-thirds of the group had received more than 30 hours of LSC professional development; the others had participated in 10 to 30 hours of professional development. As seen in Figure 12 the majority of teachers had quite positive impressions of the professional development in which they had participated.

Comments of participants who were most positive about the professional development indicate that high-quality curriculum materials were a particularly important component of the professional development. Teachers described curriculum materials as "complete and comprehensive," providing "easy access to materials" and access to "meaningful investigations." Teachers were also very positive about their experiences with inquiry-based science, the support provided through on-going activities, and strategies that involved lead teachers or core teams at their schools.

Teacher Assessment of LSC Professional Development Activities

Figure 12

Even teachers who had overall very positive comments about the LSC professional development sometimes had suggestions for improvement. A number of comments focused on the need for more direct experience with the type of teaching they were striving to implement through the LSC project. A couple of participants commented, for example, that they would like to observe teachers using inquiry-based or constructivist strategies; another participant expressed the desire to have a visiting teacher come to his classroom to demonstrate these instructional strategies.

Only a small proportion of teachers who were interviewed were described by evaluators as either "somewhat" or "mostly" negative, and all of these teachers came from 3 of the 8 Cohort 1 projects. Those teachers who were negative about the professional development they had experienced complained of being unclear about expectations of the district and needing professional development that better met their particular needs with regard to content or pedagogical strategies.

Ratings of Individual Professional Development Sessions

Observations of selected professional development sessions at each LSC site are key to the core evaluation analysis of the quality of the professional development offered by LSC projects; although evaluators were asked to take other factors into account when making continuum and component ratings of their project, observations were the main standardized data source.

The following sections describe elements of professional development that observers were asked to rate for each professional development session observed, including:

  • Design of the session;
  • Its implementation;
  • The disciplinary and pedagogical content;
  • The character of the culture of the session;
  • The likely impact on participants' capacity for exemplary science and mathematics instruction; and
  • The likely impact on participants' leadership capacity.

Each section includes details about the pertinent indicators that contributed to evaluators' ratings of the individual professional development sessions observed for the core evaluation, as well as the percent of sessions receiving overall ratings of 4 or 5 on a 5-point scale.

Design of Professional Development

Evaluators were asked to assess the design of professional development sessions by rating a series of individual indicators based on current understandings of best practice. Several of these indicators received high ratings (4 or 5 on a five-point scale) in the majority of observed sessions. Those indicators that were most often highly rated included the following:

  • The appropriateness of strategies used in the session for accomplishing the purposes of the LSC professional development (78 percent);
  • How well the session encouraged a collaborative approach to learning and incorporated tasks and interactions consistent with a spirit of investigation (75 percent); and
  • How effectively the session built on participants' knowledge of content, teaching, learning, and/or the reform/change process (71 percent).

Somewhat fewer sessions were rated highly on:

  • How well the strategies reflected attention to the varied needs and learning styles of the participants (65 percent); and
  • Whether the design of the session provided adequate time and structure for reflection (53 percent).
Professional Development Session Ratings - Design

Figure 13

Evaluators were also asked to provide an overall rating for the design of the session. As can be seen in Figure 13, 74 percent of the professional development sessions received overall design ratings of 4 or 5.

Implementation of Professional Development

Evaluators rated several indicators regarding the implementation of the professional development session. Ratings on the implementation tended to be lower than those on design. The indicators most frequently rated 4 and 5 were:

  • Whether the facilitator's background and expertise enhanced the quality of the session (82 percent); and
  • The appropriateness of the pace of the session for adult learners (69 percent).

Fewer LSC professional development sessions were rated highly on other indicators, including:

  • How well the session modeled questioning strategies that are likely to enhance the development of conceptual understanding (60 percent); and
  • How well the session modeled effective assessment strategies (42 percent).
Professional Development Session Ratings - Implementation

Figure 14

As can be seen in Figure 14, about two-thirds of the sessions received overall ratings of 4 or 5 on the quality of implementation.

Content of Professional Development

Evaluators rated both disciplinary and pedagogical content of the professional development sessions. The following individual indicators were highly rated in most of the observed sessions:

  • The appropriateness of the disciplinary and pedagogical content for the participants' backgrounds (80 percent);
  • The extent to which appropriate connections were drawn to real-world contexts (70 percent); and
  • The soundness of the disciplinary/pedagogical content and how appropriately it was explored or presented at the session (68 percent).

As can be seen in Figures 15 and 16, 65 percent of projects were rated 4 or 5 on disciplinary content and 67 percent were rated a 4 or 5 on pedagogical content. Note also that while ratings on disciplinary content were similar for the two cohorts, Cohort 1 projects were more likely than those in Cohort 2 to receive the highest possible ratings for pedagogical content.

Professional Development Session Ratings - Disciplinary Content

Figure 15

Professional Development Session Ratings - Pedagogical Content

Figure 16

Professional Development Culture

Most individual indicators for professional development culture were rated highly by evaluators. Indicators that received ratings of 4 or 5 in most of the sessions included:

Whether interactions reflected collaborative working relationships among participants and also between facilitators and participants (77 percent);
  • How encouraged participants were to generate ideas, conjectures, and propositions and to what extent investigation and risk-taking were valued (74 and 70 percent, respectively);
  • Whether participants were intellectually engaged with important ideas relevant to the focus of the session (72 percent); and
  • How highly valued intellectual rigor, constructive criticism, and the challenging of ideas were during the session (65 percent).
Professional Development Session Ratings - Classroom Culture

Figure 17

Figure 17 shows that 73 percent of the sessions received ratings of 4 or 5, with fairly similar ratings for the two cohorts.

Impact on Participants' Capacity for Exemplary Science/Mathematics Instruction

LSC evaluators were asked to rate how likely they felt the session was to have an impact on teachers' capacity for exemplary science/mathematics instruction. Ratings for this area were quite low, with 60 percent or fewer projects receiving ratings of 4 or 5 on any of the indicators:

  • How likely the session was to enhance participants' ability to identify and understand important ideas of science/mathematics (60 percent);
  • Whether the session enhanced professional networking among participants (60 percent);
  • The likelihood that the session would enhance participants' ability to plan and implement exemplary classroom instruction (55 percent); and
  • Whether the session was likely to help participants better understand how students learn (48 percent).
Professional Development Session Ratings - Capacity for Exemplary Instruction

Figure 18

As seen in Figure 18, only 50 percent of the sessions received overall ratings of 4 or 5 in this category. Apparently, evaluators often saw individual elements of high quality, but felt that the experience as a whole was not likely to translate into a marked positive improvement in teaching.

Impact on Participants' Leadership Capacity

The 38 observed sessions in which leadership development was a focus were rated on their likelihood to have an impact on participants' capacity to be leaders in science/mathematics instructional reform. In these sessions, the indicators most frequently receiving ratings of 4 or 5 were:

  • The likelihood of the session to enhance participants' understanding of the reform/ change process (70 percent); and
  • Whether the session would help participants be more confident in their ability as leaders of science/mathematics reform efforts (68 percent).

Fewer of the observed sessions that focused on leadership-building received high ratings on other indicators such as:

  • Whether it would enable participants to plan/implement exemplary professional development (54 percent);
  • How well the session enhanced professional networking among participants with regard to leadership (50 percent);
  • Whether the session would help teachers identify and understand important strategies for reform (44 percent); and
  • Whether the session enhanced participants' understanding of adult learners (37 percent).
Professional Development Session Ratings - Leadership Capacity

Figure 19

As seen in Figure 19, 64 percent of the sessions received overall ratings of 4 or 5 in this area; however, there were striking differences in the ratings of Cohort 1 and Cohort 2 projects. Only 5 of the 13 Cohort 1 leadership sessions (38 percent) were rated a 4 or 5 on likelihood of building leadership capacity compared to 18 of the 23 (79 percent) Cohort 2 sessions.

Capsule Ratings of Observed Professional Development Sessions

Evaluators were asked to provide an overall "capsule rating" of each observed session, taking into account the relative impact of each of the component areas. As would be expected, given the high overall continuum ratings assigned by evaluators for the professional development systems, ratings for individual professional development sessions were similarly quite favorable. Very few LSC sessions were rated as ineffective professional development (Level 1), and fewer than 10 percent were rated at Level 2, having quite limited likelihood of helping participants implement exemplary science/mathematics instruction or be leaders in reform. (See Figure 20.) More than 60 percent of the observed professional development sessions received ratings of 4 or 5, with Cohort 1 sessions somewhat more likely than Cohort 2 sessions to be considered "exemplary professional development."

Professional Development Capsule Ratings of Observed Sessions

Figure 20

The vignettes below illustrate a variety of types of sessions observed and some of the strengths and areas of need noted by evaluators.

Professional Development Session #1:
"Learning Content and Pedagogy Through Kit-Based Activities"

Third and fourth grade teachers were engaged in professional development around a kit-based unit on water. One session focused on a lesson involving construction of a water thermometer. There were multiple goals for participants: (a) to experience learning by inquiry, (b) to try out the activity, and (c) to learn about instructional strategies. The facilitator began by questioning the participants to learn what they already knew about thermometers, modeling the kinds of questions and problems teachers might pose with their own students.

Pairs of participants constructed thermometers, and were engaged in making predictions, investigating how those predictions could be tested, and actually carrying out mini-investigations. The implementation was a real strength of the session. The facilitator knew just when to ask thought-provoking questions to keep participants thinking and investigating further.

Participants were encouraged to share their ideas about thermometers--e.g., how they work, what might go wrong, and how to deal with that. These discussions led to generating ideas about how their "hypotheses" might be tested. At appropriate points throughout the session, the facilitator wove in discussions about classroom application and classroom management issues. Overall, the facilitator used a kit-based unit to skillfully model effective content, process, and inquiry.

This session received ratings of 5 in the areas of design and implementation; ratings of 4 on content, professional development culture, and likelihood of enhancing teachers' capacity to implement exemplary science instruction. The session was placed at Level 5 for the overall capsule rating, indicating this was an exemplary professional development session.

Professional Development Session #2:
"Study Groups: Classroom Observation and Discussion"

A third-grade team of teachers met at the school on a regular school day for a four-hour professional development session focused on the district's recently-adopted mathematics curriculum and aimed at refining teachers' knowledge of the content and pedagogical strategies needed to successfully implement the curriculum. Self-study/discussion teams organized by grade-level, were meeting regularly around each of the modules as they were implemented throughout the school year.

On this day, early in the school year, the entire study team and a lead teacher/facilitator on leave to work with the LSC project would observe a class taught by two of the study-team members. The two teachers had combined their classes, and were team teaching a lesson from the current module on fractions. Prior to the lesson, the group met to discuss the lesson, reflecting on what they hoped students would get from the lesson; potential questions the teachers might ask to prompt student inquiry and reflection; some potential pitfalls that might be encountered; and how they might orchestrate some informal assessment of students within the lesson. The group then gathered in the classroom to observe the lesson.

Following the 45-minute lesson, the study team reconvened to discuss what they had observed. At first the members of the study team were very reserved. They congratulated two team teachers on their implementation, pointing out what they felt were key strengths of the lesson. They discussed specific incidents that were telling demonstrations of what students knew and what they did not know about fractions. There was some confusion and contradiction in different teachers' assessment of where the team should go next with their students.

Particularly critical for setting the tone of the discussion was the non-threatening way in which the teachers discussed the lesson and the willingness of the team teachers to reflect openly and honestly with the group. An especially fruitful part of the conversation occurred when one teacher pointed out to one of the team teachers that she had called only on boys during the whole group discussions. The teacher's initial expression of surprise, followed by acknowledgment of a pattern she had not been aware of, led to an in-depth discussion of that and other issues of equity. The facilitator's role throughout the session was to help keep the discussion focused issues raised by the group, and to prompt reflection about implications of what they had observed.

The session received ratings of 5 on design, implementation, and professional development culture. Content was rated 4. There were a couple of areas in which teachers were clearly struggling with content themselves. Those areas were not addressed sufficiently in this session, but the facilitator suggested they orient their next meeting around a more in-depth exploration of those concepts that were proving problematic for the students and teachers, alike. Overall, this session was described as "exemplary," receiving a capsule rating of 5.

Professional Development Session #3:
"Exploring Student Assessments"

Teachers of grades K-6 met with facilitators form the LSC project staff in the final half-day session of a week-long workshop on assessment. This session focused on a participant-identified need for assessment strategies for cooperative learning groups. The session had a dual purpose of enhancing the participants' expertise in designing/scoring student assessments, and building a network of teachers who could serve as "assessment experts" in the districts.

The day began with a whole group discussion of an article the participants were to have read the previous evening, on the role of cooperative learning in assessing and communicating progress in student leanings. This article was one of many resources available in a packet provided for each participant.

Following the discussion of the article, teachers worked in groups to develop criteria for selecting reporting options to effectively communicate student progress. Group discussions were characterized by active participation and a collaborative approach to addressing the issues. The facilitators circulated among the groups, encouraging participants to use each other as resources and modeling effective questioning techniques in the process.

A third component of the session was an in-depth discussion of equity issues in assessing students' work. Discussions focused on whole group versus individual grading, heterogeneous versus homogeneous ability grouping, single-gender versus mixed gender groups, and the modification of rubrics to accommodate students with special needs.

The week-long workshop was to be followed up with additional activities designed to further enhance teacher understanding of assessment and provide opportunities for continued sharing of experiences and building the network of assessment experts.

This session received ratings of 3 on design and content and ratings of 4 on implementation and professional development culture. The key weakness noted by the evaluator was the "generic" nature of the work around assessment strategies. There was only incidental application of the strategies to the mathematics modules teachers were to implement in their classrooms. The overall ratings for the session was 4, indicating that the session was very effective and likely to enhance the participants' classroom instruction.

Professional Development Session #4:
"An Attempt to Address Science Content"

A professional development session for first-grade teachers was designed to help the teachers learn the content targeted by a kit-based unit on liquids, and to become familiar with the activities included in the kit. Some participants were new to the system; others were veteran teachers who wanted to learn more about the curriculum that they would be implementing in their classes in the coming year. The session was led by two facilitators-a lead teacher in the district and a scientist from a local university.

On the previous day, the lead teacher had taken participants through several activities included in the kit. Most of the two-hour session on this day was focused on introducing the "big ideas" of the unit to the whole group, with the scientist presenting content about atoms, measuring, volume, density, and phase changes. While the presented information was accurate, there was little integration of the content with the activities of the previous day. The presentation ended with an activity in which the entire group role-played the movement of molecules at different temperatures.

In the last half-hour of the session, the teachers met in small groups to discuss the content covered in the morning presentation and to identify questions they may have about the content or the activities included in the unit. The scientist and lead teacher initially circulated among the groups; however, both of the facilitators spent the majority of the remaining time with one of the four small groups to address questions about one of the activities from the previous day. The session ended with an announcement of when the group would meet next for professional development on a second unit to be implemented in all first-grade classes.

This session received ratings of 2 on a five-point scale on its design and implementation; and a rating of 3 on content. Due to the lack of participant engagement and absence of collaborative or participant-centered interactions, the session was rated 2 in professional development culture. Overall, the session received a capsule rating of 2, indicating that there were elements of effective professional development, but that there were some major areas in need of improvement.


In summary, the LSC projects are providing high-quality professional development to their targeted teachers. All projects were rated at level 3 or above on a five-level continuum, indicating that the projects are either transitioning toward, or have already achieved a level of professional development that incorporates many features reflective of current standards-based approaches and that these activities will likely enhance participants' capacity to provide high-quality science and mathematics instruction to their students.

Each of the component areas of professional development-the design, the implementation, the disciplinary and pedagogical content, and the culture of the sessions-received very high ratings, as well. Mean ratings for each of those areas, as well as the likelihood that the project would enhance teachers' instructional capacity were consistently between 3.5 and 4.0 on the five-point scale. Projects that focused on developing the leadership capacity of teachers in the district were rated somewhat lower in that area, with a mean rating of approximately 3.3.

The LSC projects are distinguished from teacher enhancement projects of the past by their systemic focus, including professional development for all teachers of science/mathematics at the targeted grade levels, the implementation of exemplary instructional materials, and the alignment of district policies and practices with the reform vision.

Evaluators found that the professional development sessions generally incorporated state-of-the-art strategies, including activities that were relevant to the teachers' work in classrooms, modeling of effective pedagogical strategies, opportunities for teachers to reflect and talk with peers, strong components of follow-up and support, and active involvement of teachers as key players in their own professional development. The sessions were generally led by competent and skilled facilitators and characterized by very positive cultures that encouraged the engagement of teachers as professionals in learning communities.


5 The elements of individual professional development sessions discussed in the following sections parallel the components of the professional development system reported earlier (Figures I-6 and I-7). When rating the system components, evaluators had been asked to rate each component holistically, using all the data available to them. In contrast, ratings in the following sections are based solely on individual professional development sessions observed for the core evaluation.

 to previous page   next page