Communication Center  Conference  Projects Share  Reports from the Field Resources  Library  LSC Project Websites  NSF Program Notes
 How to Use this site    Contact us  LSC-Net: Local Systemic Change Network
Educational Reform & Policy

Professional Development

Teaching and Learning

LSC Papers and Reports

Cross Site Reports

LSC Case Study Reports

Papers / Presentations authored by LSC Members

About LSC Initiatives and their impact




LSC Year Two Cross-Site Report

author: Iris R. Weiss, Kathleen A. Rapp, Diana L. Montgomery
published in: Horizon Research
published: 02/04/1998
posted to site: 02/04/1998
Chapter Two


Core Evaluation Question I
What is the overall quality of the LSC professional development activities?

Core evaluation data to assess the quality of LSC professional development activities come from observations of the professional development activities, and to a lesser extent, from teacher questionnaires. Cohort 1 projects augmented these data with teacher interviews. Based on these data, evaluators were asked to rate the quality of the project's professional development in eight component areas and to provide an overall rating of the project's professional development system.

Cohort 1 projects reported on their second year of implementing LSC professional development activities. These projects all target elementary science teachers, and are analyzed in a single group, referred to as C1S. For the newly-funded Cohort 2 projects, these 1996 data represent the status of professional development during the first year of LSC funding1. Cohort 2 includes both projects that target elementary science (C2S) and projects that target elementary mathematics (C2M).

Teacher Ratings of District-Provided Professional Development

The LSC teacher questionnaire included several questions about the quality of the science/mathematics professional development available in their district. There was no attempt to focus on LSC professional development programs, but rather teachers were asked to think about district-provided professional development generally.

The composite for quality of district professional development combines teacher ratings of the extent to which they were:

  • Involved in planning their professional development;
  • Encouraged to develop an individualized professional development plan;
  • Given time to work with other teachers and to reflect on what they have learned; and
  • Received follow-up support.

Ratings on this composite are relatively low and show a lot of variation among projects; project means ranged from 43 to 65 percent of total possible points. (See Figure 6.)

Figure 6

Roughly one-fourth to one-third of teachers in each cohort reported that they were fairly heavily involved in planning their own science- or mathematics-related professional development and encouraged to develop individual professional development plans to address their needs and interests. Only about 20 percent of the teachers indicated that they are given ample opportunities to work with other teachers or reflect on how to apply what they have learned to the classroom as part of their professional development. Roughly 1 in 3 Cohort 1 teachers, but only 1 in 5 Cohort 2 teachers reported that they were given the follow-up support they needed to implement what they had learned in professional development experiences. (See Table 2.)

Table 2
Teacher Descriptions of District Professional Development

Percent of Teachers*
Cohort 1 Science Cohort 2 Science Cohort 2 Mathematics
Involved in planning my science (mathematics) related professional development 34 25 30
Encouraged to develop an individual professional development plan 32 26 33
Received follow-up support in implementation 32 18 22
Given time to work with other teachers as part of my professional development 23 17 21
Given time to reflect on what I've learned and how to apply to the classroom 21 19 25
* Includes teachers marking 4 or 5 on a five-point scale where 1 = "not at all" and 5 = "to a great extent."

Given current standards for best practice in professional development, there is a great deal of room for improvement in the quality of district-provided professional development in science and mathematics. In the following sections, data from evaluators' observations of LSC professional development activities are used to describe how LSC projects are addressing this challenge.

Description of LSC Professional Development Activities

Evaluators and PIs were asked to decide jointly which project activities would be observed for the core evaluation, selecting sessions to represent the diversity of the project's activities and reflect the extensiveness and importance of the various activities offered. LSC evaluators observed a total of 180 professional development sessions, an average of seven observations per project. The following description of LSC professional development activities is based on these sessions.2


The typical professional development session observed for the LSC core evaluation had between 21 and 50 participants, mostly teachers from a range of the elementary grades. As can be seen in Figure 7, Cohort 1 projects continued to focus on preparing lead teachers (almost three-quarters of Cohort 1 science sessions included lead teachers); far fewer Cohort 2 sessions included lead teacher participants (36 percent). Overall, 7 out of 10 observed professional development sessions included regular (non-lead) teachers and fewer than 2 in 10 included administrators as participants.

Participants in Observed LSC Professional Development Activities

Figure 7


Typically, LSC professional development sessions were conducted by a team, sometimes including lead teachers, but more frequently composed of other personnel both from the district and from outside the district. Almost 90 percent of the observed sessions had at least one female presenter or facilitator, and 17 percent included non-white women. A little over half of the sessions had male presenters, in a few cases including African American or Hispanic males.

Session Characteristics and Foci

Evaluators were asked to indicate the purposes of each professional development session they observed, choosing from twelve possibilities; for most sessions, evaluators checked from 2 to 4 purposes. Nearly half of the observed sessions focused on helping participants create a vision of learning by inquiry, try out activities or curricula for classroom use, or learn science/mathematics concepts. (See Table 3.) Almost 40 percent of sessions centered on working with teachers on pedagogical/classroom management strategies; about a fifth of observed sessions concentrated on building a professional network among educators or developing participants' leadership capacity.

Table 3
Major Intended Purposes of Observed
LSC Professional Development Sessions

Percent of Sessions
All Cohort 1 Science Cohort 2 Science Cohort 2 Mathematics
Creating a vision of learning by inquiry 47 41 54 31
Trying out specific (e.g., newly-adopted) activities/curricula for classroom use 46 46 46 46
Learning science/mathematics concepts 45 48 43 46
Learning pedagogical/classroom management strategies 39 35 41 38
Building a professional network among educators 21 37 12 23
Developing leadership capacity of participants 21 24 21 15
Orientation to the project 14 22 6 27
Developing instructional materials/activities for classroom use 14 13 15 12
Aligning district curricula with state/national standards 12 19 9 7
Planning future professional development 11 28 2 12
Designing/scoring student assessments 9 20 6 0
Developing capacity of participants to use the Internet/on-line telecommunications 2 6 1 0

Nearly half of the observed science sessions included physical science content, 37 percent focused on life science, and 33 percent had an earth-science content focus. (See Figure 8.) Among projects targeting mathematics, the content areas of focus were most often computation and estimation, numeration and number theory, patterns and relationships, and mathematical problem solving. While a few of the science sessions included emphasis on one or more mathematics topics (especially measurement), none of the mathematics sessions included science content as a major focus. (See Figure 9.)

Major Disciplinary Content Areas of Observed Science Professional Development Sessions*

Figure 8

Major Disciplinary Content Areas of Observed Mathematics Professional Development Sessions*

Figure 9

As can be seen in Figure 10, when asked to describe the "pedagogical content" of the observed session, evaluators in each cohort noted that over half of the observed sessions emphasized instructional strategies. Somewhat fewer sessions focused on providing visions of effective science/mathematics instruction and even fewer on how students learn. Professional development sessions in Cohort 1 were much more likely than those in Cohort 2 to include consideration of assessment strategies and issues related to access, equity, and diversity.

Pedagogical Content Areas of Observed LSC Professional Development Activities

Figure 10

Most LSC professional development sessions included a formal presentation by a facilitator or other presenter. Teachers were involved in field activities, research, or other hands-on activities during 70 percent of the observed sessions; discussions or seminars occurred at a similar number of sessions. Very few sessions (12 percent) involved participants actively exploring uses of technology. (See Table 4.)

Table 4
Major Activities at Observed LSC Professional Development Sessions

Percent of Sessions
All Cohort 1 Science Cohort 2 Science Cohort 2 Mathematics
Formal presentations by presenter/facilitator76787577
Hands-on/ manipulative/ research/ field activities70697265
Problem-solving activities36333738
Explored assessment strategies34443123
Formal presentations by participants16221412
Explored technology use1220519

Evaluator Ratings of the Quality of the LSC Professional Development

Evaluators' overall ratings of the LSC professional development indicate that LSC projects are generally providing high-quality staff development. Each project evaluator was asked to rate the project's professional development system holistically, considering all of the data available, and place their LSC project on a five-level continuum. (See Table 5.)3

Ratings for the two cohorts were quite similar, with mean ratings of 3.8 on the 5 point scale. Eleven out of 25 projects were rated at Level 3 ("Transitioning to Quality Professional Development") by their project evaluators.4 All of the remaining projects were rated even higher on the continuum, indicating that their professional development plan and activities incorporated many features reflective of current standards-based approaches, that the professional development activities were well-implemented, and that in the evaluator's judgment they will likely enhance participants' capacity to provide high quality science/mathematics instruction to their students.

Table 5
Continuum Ratings for Quality of LSC Professional Development

Percent of Projects
All Cohort 1 Cohort 2
Level 1: Predominance of Ineffective Professional Development000
Level 2: Exploring Quality Professional Development000
Level 3: Transitioning to Quality Professional Development445041
Level 4: Emerging Infrastructure of Well-Designed Professional Development362541
Level 5: Predominance of Well-Designed Professional Development202518
Mean Continuum Rating Level3.83.83.8

In addition to overall continuum ratings, evaluators provided ratings for each of seven component areas for professional development, including: the design of activities, their implementation, the disciplinary and pedagogical content, the professional development culture, the likelihood that the professional development would build participants' capacity to provide exemplary science/mathematics instruction, and the likelihood it would build their capacity to be leaders in science/mathematics instructional reform. The largest difference between the cohorts was in ratings for capacity-building for instruction, with Cohort 1 projects more likely to be highly rated in this area. (See Figure 11.) No evaluators rated a component area for their project at Level 1, and very few components were rated at Level 2.

Components of Professional Development System
Evaluator Ratings by Cohort

Figure 11

Strengths of the LSC Projects

Evaluators were asked to describe their rationale for their ratings, particularly highlighting strengths of the professional development activities and areas in need of improvement. Aspects of the LSC professional development that were most often described by evaluators as strengths were:

  • strategies for professional development;
  • the skills, competence, and expertise of facilitators;
  • the systemic focus on the broader science/mathematics education system;
  • a culture conducive to and supportive of learning;
  • relevant, high-quality content;
  • models for effective development of teacher leaders; and
  • reflection of project staff for program improvement.

Two features that distinguish this group of professional development projects from many past teacher enhancement projects are the systemic focus and the consistency of attention directed toward the use of exemplary curriculum materials. Evaluators commented frequently on these aspects of the LSC projects.

Systemic Focus

Evaluators of the LSC projects described a number of ways in which projects targeted the broader science/mathematics education system. For example, a requisite commitment for LSC funding is to serve all eligible teachers for a substantial number of hours. Many programs are choosing to begin by "exposing" large proportions of teachers to one or two kits and achieve at least a "mechanical" knowledge of the materials early on. One evaluator pointed out the benefit of this strategy as a step toward building support among stakeholders:

"Through the [teacher workshops], the Partnership is reaching larger numbers of teachers and providing exposure to inquiry-centered instructional approaches that can readily be used in elementary classrooms. These workshops, by increasing the number of teachers involved in the Partnership, are also expanding the cadre of stakeholders involved in the reform of science and mathematics."

Most projects are implementing curricula that the district has chosen or agreed to adopt, reinforcing the alignment of district curriculum and goals. One evaluator made this observation:

"[The district] has recently adopted FOSS (Full Options Science System) as its district science curriculum. Many of the pedagogy sessions and some of the content sessions encourage exploration of specific FOSS kits. By utilizing the district curriculum as a starting point for good hands-on science, [the project] is seen by participants as supporting the district-identified process to improve science teaching."

State-of-the-Art Strategies

The LSC projects are also distinguished, as a group, by being highly reflective of current standards for best practice in professional development. Features described most often by evaluators of the LSC projects are described below:

  • Relevancy of professional development activities to teachers' work in classrooms

    Evaluators found, for example, that activities in which teachers were engaged were closely linked to their needs as classroom teachers. Said one,

    "This professional development model establishes a different dynamic between practitioners and their professional growth. There is a relationship between what occurs inside and outside of their classroom. The school adopts math and science curricula considered exemplary and in the best interest of students, and complements the adoption with extended staff development to assist teachers in the implementation process."

    A strategy mentioned as particularly successful by another evaluator was allowing teachers to focus on grade-specific activities. This evaluator reported these teacher comments:

    "What I got a lot from in [the workshop] was when I sat with other fourth grade teachers and talked about how we implemented the kit"

    "The people who did kindergarten [sessions] were actual kindergarten classroom teachers so when they gave information it had been used and tested and worked. Those were very useful, very helpful."

  • Modeling effective strategies

    Evaluators found that strategies advocated by the projects were modeled in professional development activities. Activities were characterized, for example, by active involvement of teachers and participatory hands-on activities; in many cases teachers worked on activities from modules that they would later use in their classes.

    Experiencing instructional strategies firsthand was particularly important for teachers being introduced to inquiry-based strategies, as described by this evaluator:

    "In order to teach in an inquiry-based environment, teachers must experience inquiry learning for themselves. The introductory summer institute provides the foundation upon which teachers must stand if they are to transform their teaching approaches to be truly inquiry-based. Two weeks enables participants to thoroughly explore a particular scientific phenomenon, and to come to conclusions of their own about why "x" works, drawing on their own experiential learning. In addition, teachers learn how a real classroom might function as an inquiry-based learning environment. This is often difficult to even imagine, given the relatively fixed structured nature of most elementary schools. In discussions and debates, the institute directors encourage participants to consider how to translate their experience back to their classrooms."

  • Provides opportunities for teachers to reflect and talk with peers

    Time for reflection was incorporated into many of the projectsi professional development activities sometimes within individual sessions; sometimes as the major focus of entire sessions as in school-based peer group meetings to discuss the implementation of a kit. This comment is reflective of evaluators' (and participants') praise for projects' providing opportunities for reflection:

    "Overall, lead teachers, in interviews and in the post-institute evaluation surveys highly praised the academic year sessions-particularly noting the opportunities to share classroom practices, to practice inquiry, and to discuss national standards with peers. Through the [institute] lead teachers have opportunities for reflection, dialogue, and peer support."

  • Strong components of support and follow-up

    Support throughout the academic year is a consistent feature of the LSC projects, and one that evaluators recognized as a strength. One evaluator, for example, highlighted the importance of providing opportunities for teachers to meet throughout the year for discussions with peers after actually trying out modules in classrooms.

    Projects have explored different strategies for providing on-going support to teachers. A number of projects that focus on kit-based curriculum have organized regular meetings in which teachers may gather to discuss the implementation of the kits. Some projects have worked to build the expertise of in-school teacher leaders who are available for individual support to their peers. Another follow-up strategy has been for teachers to develop personal action plans with support provided throughout the year by mentor teachers or project staff.

  • Active involvement of teachers

    The LSC projects are clearly attentive to actively involving the teachers as key players in their own professional development. Evaluators described a number of mechanisms designed to engage teachers as active participants and leaders of professional development in their districts.

    A number of projects incorporate teacher-leader models designed to build capacity of teachers within the schools and districts, and to reach larger numbers of teachers. Other strategies for encouraging teachers' involvement include options for teachers to select from multiple professional development activities; facilitating active sharing, reflecting and collaboration in small groups; fostering school-based implementation built on "school-site learning communities"; and assembling action research teams that offer teachers the opportunity to network and share with each other.

Challenges Faced by LSC Projects

Interestingly, some of the same features highlighted as strengths by some evaluators were identified by other evaluators as areas of challenge for their projects. Aspects of the professional development most often described as needing increased attention for improvement were:

  • Time for participant reflection;
  • Attention to content;
  • Attention to classroom application;
  • Consistency in the skills and expertise of facilitators;
  • Attention toward equity; and
  • Teachers' use of available resources.

The overlap in features most likely to be mentioned as strengths when they are present and as areas in need of improvement when they are not include: time for reflection, attention to classroom application, high-quality content, and skill and expertise among facilitators. The following comments of evaluators are representative of the concerns expressed by some.

Regarding the need for increased time for reflection:

"Although there was active learning going on in this and other activities during the session, there were missed opportunities for reflection, individually and as a group on what was being taught and learned. That is, although teachers were asked good questions, for example comparing actual [data] with predictions and providing what they thought accounts for differences, the discussions tended to be very limited in time and scope."

"Complex ideas were demonstrated in an interesting way, but some of the teachers did not internalize the concepts. The minds-on portion of the workshop was missing. As a result, the teachers may have some difficulty developing assessment tools and questioning skills when introducing the concepts in their classroom. The workshop seemed more of a show than an integration of content, process, reflection and application. The teachers needed to reflect on the demonstrated concepts and perhaps discuss age-appropriate applications of the concept."

Regarding content-particularly finding an appropriate balance in the content and process of inquiry:

"To elevate the quality of the kit workshops still higher the focus needs to expand to include an emphasis not only on the procedures and organization of the kits, but on the science content and the conceptual development embedded within them. Sole focus on the mechanics of the kits does not allow teachers time to reflect on their teaching practice or the rationale for hands-on, inquiry-based science education. This may lead participants to view kit-based science teaching as an end rather than a means to exemplary inquiry-based science instruction."

Regarding attention to classroom application:

"The teachers needed more time to consider classroom applications of resources, strategies, and techniques introduced in the sessions. For example, in one of the sessions observed, the participants viewed a video called "A Private Universe" which illustrates how students often develop misconceptions about science concepts and suggests methods to identify and resolve these misconceptions. The teachers in this session, however, were unclear as to how to use this information in their own classrooms. The method demonstrated in the video, individual student interviews, was not an option for most teachers, and they needed time to discuss other strategies."

Regarding the skill and expertise of facilitators:

In some cases, lapses in quality of the professional development were due to facilitators whose expertise in content areas was greater than their knowledge of appropriate professional development strategies. In other cases the challenge was to achieve consistency among teacher leaders working with peers. For example, one evaluator made this observation:

"The quality of those sessions was somewhat variable. The presentations made in the fall for colleagues were exceedingly well-executed and reflective of best practice, keeping in mind that many of the activities had been modeled for them by [project staff] in their first summer of professional development. When they were on less familiar ground [the] sessions were not always as well-executed."

Regarding increased attention toward equity:

In addition to the general need for equity issues to "gain visibility", evaluators commented on the challenge of establishing equity as a priority for participants:

"A major focus of the professional development sessions during the summer and year is how to successfully implement the science curricula in a manner that is accessible to all students. In addition, this summer's institute had a series of workshops on cooperative learning as a direct attempt to address the issue of access for new and beginning teachers. Although [the project] is accomplishing some things in the area of equity, the associate superintendent for curriculum and instruction believes that much more needs to be done in this area as test scores for the African-American and Latino populations have not showed expected gains."

"One issue that must be dealt with in the future is the perception of teachers that they know all about equity already and they do not understand why time is spent on this topic when there are so many other content, assessment, and procedural topics that they would like to hear about in professional development sessions."

Regarding use of available resources:

Teachers' failure to use existing resources was described as a challenge for a number of the projects. In most cases the issue was that teachers were not taking advantage of opportunities available to them, as illustrated by these evaluators' comments:

"While monthly meetings, content courses, and [teacher] assistance is available to classroom teachers, many do not make use of these resources..."

"Generally, [the project] is providing high-quality professional development activities by teachers are not feeling supported in the classroom implementation of what they have learned. One area where [the project] attempted to address this need was the creation of kit clubs. Unfortunately, not many teachers attended these even though they would have been paid $20 for every two-hour after-school session they attended."


1 One project in Cohort 2 did not collect professional development observation data in 1996 because of a late start-up date; this project's baseline questionnaire data are included in the cross-site analyses.

2 HRI re-rated all 1995-96 professional development observations based on the updated continuum developed for the 1997 observation protocol. Evaluators were asked to verify these ratings and given the opportunity to make changes as needed. Ratings reported here have either been provided by or verified by the evaluators.

3 A number of projects reported observing professional development sessions in addition to those rated for the core evaluation; these additional observations were considered by evaluators when making overall ratings of the project's professional development system.

4 A project placed at Level 3.5 was included with the Level 3 group in this report.

 to previous page   next page