 |  |  |
|
|||
|
|||
|
|||
| |
||
| |||
 |
         |
|
|
Paper
Highlights of the Year Two LSC Cross-Site Report
Classroom Practice A number of core evaluation data sources contributed information about classroom practices. The teacher questionnaire data provide self-report information from a sample of all LSC-targeted teachers about the frequency with which they use various instructional strategies in their classes. The classroom observations provide an external look at the quality of classroom instruction, including information related to equity, classroom culture, and student engagement that are difficult to measure with questionnaire items. As is the case nationally, elementary teachers in LSC districts report that they teach science less frequently than they teach reading/language arts or mathematics, and about as often as they teach social studies. For example, only about 25 percent of teachers reported teaching science (or social studies) on all 5 of the last 5 days instruction, compared to more than 80 percent for mathematics and nearly 90 percent for reading/language arts. Figure 5 shows the purposes of the observed science and mathematics lessons. Note that teachers were most likely to cite developing conceptual understanding as a major purpose of science lessons and reviewing concepts as a major purpose of mathematics lessons.
 Figure 5
Observers noted that science lessons were much more likely to focus on life science than on either physical or earth/space science and that mathematics lessons most frequently focused on computation, followed by measurement, numeration and number theory, and mathematical patterns and relationships. (See Figures 6 and 7.)
 Figure 6
 Figure 7
Most observed lessons included both a formal presentation by the teacher and some kind of hands-on/investigative activity by the students, typically ones in which students followed a detailed set of instructions rather than designing and implementing their own investigations. Cohort 1 science classes taught by teachers who had already participated in LSC professional development were more likely than others to be involved in writing reflections in a notebook or journal and less likely to be working on answering textbook/worksheet questions. (See Table 4). Instructional Activities in Observed Classes
When asked how the LSC professional development had affected their classroom teaching, teachers pointed to less reliance on textbooks and more use of student-centered strategies, including cooperative learning groups. Observations confirmed the teacher reports; lessons taught by teachers who had participated in LSC professional development were more likely to receive high ratings from evaluators on lesson design and implementation, as well as on the quality of the content and the classroom culture. (See Figure 8.)
 Figure 8
Lessons taught by teachers who had participated in LSC professional development were rated particularly high on the extent to which:
As the final step in completing the observation protocol, evaluators were asked to indicate the "level" that best described each lesson they observed. Table 5 shows the percent of lessons in each Cohort considered effective instruction, beginning stages of effective instruction, and ineffective instruction.3 Note that only 32 percent of Cohort 1 science lessons were rated as effective instruction, quite close to the 27 percent of Cohort 2 science lessons and 29 percent of Cohort 2 mathematics lessons that were rated that highly. The major difference between Cohort 1 and Cohort 2 science lessons is at the lower end of the scale, with lessons of treated teachers more likely to be considered in the beginning stages of effective instruction and less likely to be considered ineffective instruction. Overall Ratings of Observed Classes
The shaded boxes contain examples of lessons that were assigned to each of these categories.
A complementary analysis based on questionnaire data looked at classroom practice of teachers who reported varying levels of involvement in science and mathematics professional development. Teachers who had participated in a substantial amount of professional development in the previous year were most likely to feel well prepared to teach science and mathematics, and to use the instructional strategies advocated by national standards. For example, as can be seen in Figure 9, only 37 percent of classes whose teachers had not participated in science professional development engaged in hands-on activities at least once a week, compared to 70 percent whose teachers had had in-depth professional development.
 Figure 9
Supportive Context for Reform The major emphasis of the LSC is on professional development to enable teachers to effectively implement high quality instructional materials. At the same time, NSF recognized the need for projects to look at the entire system in these districts, not only professional development and instructional materials, but also policies related to student assessment; teacher recruitment, orientation, and evaluation; and systems for purchasing/managing materials and supplies. Clearly, the support of key stakeholders for the reform effort is crucial, not only from district and school-level administrators, but also from parents and the larger community. Finally, NSF hypothesized that reforms would be greatly enhanced if they had the support of business and industry; colleges and universities; and other "science-rich" institutions. The core evaluation includes information about the supportiveness of the context for reform from a number of sources. In questionnaires and interviews, teachers were asked about the support they receive from parents, principals, and other teachers in their schools. Principals were also asked about their opinions in relation to science and mathematics reform, including how important they considered various reform approaches, and how well prepared they felt to support teachers in the implementation of national standards. In addition, teacher and principal questionnaires asked about the extent to which school and district policies affect science and mathematics instruction. Finally, evaluators were asked to describe the extent to which district policies and resources were aligned in support of the LSC reforms. Teachers in the LSC districts generally feel supported by other teachers in their schools to try out innovative ideas in science and mathematics teaching, with roughly 3 out of 4 teachers agreeing with statements to that effect. Somewhat fewer, about 2 out of 3, indicated that teachers in their school regularly share ideas and materials. Only about 1 out of 8 teachers indicated they had time during the regular school week to work with their peers on science and mathematics curriculum and instruction. In both science and mathematics, teachers are most likely to report that principals encourage innovative instructional practices and accept the noise that comes with an active, inquiry-based classroom (roughly 4 out of 5 in each group). Mathematics teachers were more likely than science teachers to report that their principals provided them with the necessary instructional materials and equipment; and encourages them to gear curriculum and instruction to address individual students' learning. Principals and teachers were also given a list of "factors" that might affect instruction and asked to indicate the extent, if any, of the problem each caused for science and mathematics instruction. Teachers were answering about problems in their own instruction, and principals were answering about the school as a whole. From both the teacher and principal perspective, there is considerable room for improvement in the extent to which the policy environment supports science and mathematics instruction. In both science and mathematics, lack of time to work with other teachers was the most frequently cited problem, with more than half of the Cohort 1 and Cohort 2 science teachers and nearly that many Cohort 2 mathematics teachers labeling it a major problem. Access to computers ranked second, again with more than half of the teachers in science and nearly half in mathematics indicating that a lack of computers was a major problem. Other frequently cited problems, especially for science instruction, included lack of funds for purchasing equipment and supplies, and lack of time to plan and prepare lessons. Principals generally echoed the concerns expressed by teachers, frequently citing such problems as inadequate funds for purchasing equipment and supplies, and the lack of teacher planning time and opportunities to work together. Evaluators were also asked to rate the extent to which district policies and practices tended to facilitate reform or act as barriers for reform. Based on their reports, the most frequent facilitators of reform in LSC districts are the professional development systems; the curriculum, scope and sequence; the fact that the various reform efforts are consistent with one another; and the quality of instructional materials. Cohort 1 districts received higher marks than Cohort 2 districts in a number of areas, including the quality of instructional materials and new teacher orientation. In contrast, Cohort 2 districts tended to be more highly rated in terms of consistency among reforms. In both cohorts, teacher evaluation policies and within-school policies such as time for teachers to plan and prepare lessons were rarely seen as facilitators of reforms. A number of evaluators spoke of the commitment the districts had made to aligning their policies in support of the LSC reforms. The following are topical comments: "Districts have given [LSC project] leaders and connected curriculum review groups the responsibility of revising the district science standards and aligning them with the emerging state [frameworks] in science." "The district's recent decision to adopt new standards-based science and mathematics materials demonstrates a willingness to work in conjunction with, if not in direct support of, the goals of the LSC." "The district has purchased kits which are already in use in some schools, and the district has made a major commitment to purchase more. Efforts are underway to create a system for purchasing and managing materials and supplies." In other cases, evaluators noted that there had been major improvements in the availability of materials for hands-on instruction, but the improvements appeared project-based rather than rooted in district commitments. "Prior to joining [the project], teachers bought their own science materials or, in some cases, districts provided materials. Currently, [the project] assembles, delivers, picks up, and refurbishes materials, greatly reducing the burden on teachers and districts." Finally, some evaluators in other projects reported that financial difficulties pose serious problems for reform in the districts. "[Our site] is implementing its LSC within the context of depressing and undermining budget cuts and staff readjustments. The position of teacher expert has disappeared form the system, vice principals have been cut, and much professional development has been eliminated." "A lack of materials has been prohibitive...since these schools started with no materials, they will need to make future investments for several years." "Currently, many teachers are forced to supplement their stock of materials with items purchased with their own money. They look forward to the time when all classrooms will be equipped with adequate materials." Evaluators were asked to summarize their findings by placing each project at the appropriate point on a continuum of "supportiveness of context". As can be seen in Table 6, most projects in each cohort were considered "transitioning toward a supportive context." Continuum Rating for Supportiveness of Context
Sustainability of the LSC Reforms One of the "systemic" aspects of the LSC initiative is the expectation that districts will be able to sustain the reforms after the NSF funding period has ended. Evaluators were asked to rate the extent to which the participating districts have the capacity to implement high-quality professional development, the resources available to support it, and the structures in place to sustain high-quality professional development systems. As several evaluators noted, these ratings are based on a mixture of data and "impressions," but they do provide a broad indication of the status of the districts' professional development systems. Evaluators were asked to indicate the "continuum level" which best describes the status of the professional development system within the targeted districts. As can be seen in Table 7, most Cohort 1 science projects were judged to be "transitioning toward a high-quality professional development system." Apparently, a number of the Cohort 2 districts had made progress towards institutionalizing high quality professional development systems prior to receiving the LSC awards, with a third of the projects rated as "emerging infrastructure" or even having already institutionalized such a system in the baseline year of their LSC. Continuum Rating for Sustainability
While data from Cohort 1 projects are technically not "baseline," it is still early in the life of these projects to be looking for signs of sustainability. For that reason, and also because the Cohort 2 projects appear to have been further along in institutionalizing high-quality professional development at the time they began their LSC grants, evaluator ratings of individual aspects of sustainability are presented together. Evaluators report that in roughly two-thirds of the projects, districts have invested in enhancing the expertise and capacity of lead teachers and that these teachers are actively engaged in facilitating professional development activities (based on the percentages responding 4 or 5 on a five-point scale from 1, "not at all" to 5, "to a great extent"). In addition to the capacity to provide high-quality professional development, districts will need resources to support these programs. Figure 11 shows evaluators' ratings of a number of aspects of resource availability. Note that while in more than half of the LSC projects the districts actively seek opportunities to provide on-going, high-quality professional development experiences for their teachers, the use of district funds for these purposes is less common.
 Figure 11
Finally, districts will need structures in place to sustain high-quality professional development and provide a supportive context for exemplary instruction in science and mathematics. Evaluators in less than half of the projects reported that the districts provide incentives for teachers to participate in on-going professional development. And while 4 out of 10 have structures in place for continuously assessing and improving their professional development systems, only 1 in 5 has mechanisms in place for monitoring policy alignment, and only 1 in 4 for maintaining support for reform among community stakeholders. Horizon Research, Inc. 01/06/98 15 1 An abstract of each of these projects is included in the "Local Systemic Change Project Directory" available from the National Science Foundation and can be accessed through the NSF home page at www.nsf.gov/cgi-bin/getpub?nsf 97145. 2 A more detailed treatment of the evaluation results can be found in the Local Systemic Change through Teacher Enhancement 1995-96 Cross-Site Technical Report, available from Horizon Research, Inc. 3 "Ineffective instruction" includes lessons the observers categorized as level 1a, passive learning; level 1b, activity-for-activity's sake; and level 2, elements of effective instruction. Lessons rated a low or solid 3 are considered "beginning stages of effective instruction," while those rated a high 3, 4, or 5 are considered "effective instruction."
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
