 |  |  |
|
|||
|
|||
|
|||
| |
||
| |||
 |
     |
|
|
Annual Report Overviews
Annual Progress Overview
EXECUTIVE SUMMARY
Local Systemic Change (LSC) Core Evaluation Report The purpose of this report is to provide a summary and evaluation of the activities of the Elementary Science Education Partners (ESEP) Program since the submission of its first Local Systemic Change (LSC) Core Evaluation annual report. The ESEP program is a partnership between the Atlanta Public Schools (APS) and a consortium of colleges including Emory University, Georgia State University, and the Atlanta University Center Complex campuses of Morehouse School of Medicine, Spelman College, Morris Brown College, and Clark-Atlanta University. This report covers the period from September 1, 1996 through August 31, 1997. Based on its NSF funding date, the ESEP program is considered to be a Cohort 2 project, and this is its second Local System Change (LSC) Core Evaluation Report. This report is structured in compliance with the guidelines set forth by Horizon Research, Inc. (HRI), the company with which NSF has contracted to facilitate its core evaluation of the many NSF-funded projects. In addition, the report is augmented by an extended sample of observations conducted by the project's lead evaluator and a consultant cultural anthropologist. This report has four major parts. The report begins with an update of the ESEP program, including a brief synopsis of the major activities the program has undertaken during this year. The second section focuses on the data collection methods. The third part addresses the findings that address the six evaluation questions that have been identified by NSF and HRI as keys to understanding the LSC projects. Finally, the fourth section includes a summary and recommendations as drawn by the ESEP evaluator. The appendices provide specific data that support the report's content. The reader of this report may find it helpful to peruse ESEP's first annual evaluation report. This report provided a detailed description of the ESEP program, its philosophy, and infrastructure. Such information provides a background for all that follows in this report. Major Highlights of ESEP's Second Year as a NSF-Funded Project The major accomplishments of the ESEP project during the 1996-97 school year fall into four categories: (1) refinement of staff development sessions, including the use of teacher-leaders as facilitators, (2) identification of other groups of APS personnel who need training, (3) improvement of recruitment of student partners, and (4) full implementation of the Science Materials Support Center. The following sections briefly describe the status of these components. Findings Regarding Core Evaluation Questions The various data collected support different components of the LSC Core Evaluation. NSF has identified six evaluation questions that need to be addressed in this annual report. These six questions are:
Extent of School and Teacher Involvement in LSC Activities: The principal questionnaire data indicate that the LSC reform effort is in all 70 schools. The number of "not at all" responses from principals to questions # 6a-e surprised this evaluator. The great majority of principals or their designated representative attended the principals' workshop in January. A number of them attended the luncheon at the SKIL Institute in summer 1996. Principals with SKIL teachers at their schools were invited to a banquet in May 1997. It is clear that the ESEP team must make the connection between ESEP and LSC more evident to the principals. While this evaluator thought that this was clear in the cover letter to the principals that accompanied the questionnaire, the principals did not make the connection. The good news is that the majority (50% +) of the principals rated the five questions as either 4 or 5, which indicates that they do feel the impact of the science reform effort. In addition to the principal questionnaire data, ESEP records show the program to be in all grades in 16 pilot schools and in two grades in the remaining 54 elementary schools. Teachers' responses to the questionnaire show that the respondents reflect fairly accurately the demographic characteristics of the school district's teaching population. Based on the teachers self-report on the number of hours spent on professional development, 29 percent indicated that they had no science-related staff development and another 30 percent indicated less than 10 hours. Another 38 percent indicated that they had participated in 10 or more hours of staff development. ESEP training records show that the ESEP program has touched all (approximately 300 teachers) teachers in 16 pilot schools and approximately 257 teachers fourth- and fifth-grade teachers in the remaining 54 schools. Please note that numbers are approximated here because of attempts to unduplicate teacher counts from the multiple sessions. Based on the targeted teacher file of 1601 teachers, the figures from ESEP training records indicate that approximately 35 percent had received 18-plus hours of staff development. Last year's annual evaluation reported that about 20 percent of the teachers had been impacted for at least 12-hours. To this evaluator, it appears that the LSC project is right on target with its schedule for reaching its goal of providing in-depth professional development to all targeted teachers. Beyond teachers and principals, the ESEP program has also involved 35 instructional liaison specialists (ILSs) this year and some 30 media specialists last year. The intent of ESEP is to cultivate an environment within each school and across the district that is conducive to changing the science instruction. Impact of the LSC Professional Development on Teacher Preparedness, Attitudes, and Beliefs. Of the various data that is available to answer the question of impact on teacher preparedness, attitudes, and beliefs, the most insightful information is gleaned from the classroom observations. Teachers' self-reporting of their knowledge and practices seems a little inflated. From classroom observations, it seems that teachers are equating hands-on kit-based activities with inquiry. While the teacher and principal questionnaires reveal that both teachers and principals are becoming familiar with the concepts of exemplary science teaching, the observations reveal that implementation in the classroom is much harder to accomplish. In order to make an impact on teachers' behavior in the classroom, it is apparent that more development is needed. Teachers do express the need for more information on performance-based assessments including portfolios. Teachers also desire more training on computers, which will be forthcoming to schools, as new computers are installed over the next year. However, the training that APS will offer will be on using word-processing software packages. What teachers really need help with is on using computers as an instructional tool. These are needs identified by the teachers' responses to the teacher questionnaire. Classroom observations reveal that teachers probably need on-site coaching and demonstration of inquiry-based instruction so that they can move beyond the mechanical stage of implementation. The formal teacher interviews and informal conversations that this evaluator has had with teachers at workshops clearly indicate that most teachers are aware of the program and its goals and are attempting to implement science instruction in their classrooms using their interpretation of the inquiry method. Most, however, are concerned with management issues of time to prepare materials, time to plan the lessons, and how to work science in with the curricular demands for reading and mathematics that are reinforced by the system and state assessments. These very real everyday management concerns prevent them from moving to the next stages of implementing the program by focusing on the students and their outcomes and reflecting how they could improve themselves and the overall science program. Impact of the LSC Professional Development on Classroom Practices in Science. Information from classroom observations, teacher interviews, teacher questionnaires, and principal questionnaires provided this evaluator with data for this aspect of the LSC reform effort. From the interviews and the questionnaires, the teachers report that science is being taught and probably on a more regular basis than before the NSF project was funded. While teachers self-report positively toward inclusion of the LSC reform principles, classroom observations somewhat temper this positive perception of impact. As stated previously under the discussion related to core evaluation question III, most teachers are probably at the mechanical implementation stage of using kits and inquiry science. This impression is reinforced by the informal observations conducted by certain members of ESEP's Executive Council. These informal observations were conducted by people who are not trained to use the HRI observation instrument and did not employ that tool during the observation. These observers were primarily focusing on the teacher's use of inquiry. The consensus of these informal observations is that the teachers did not exemplify the desired level of inquiry science. From the questionnaire results, teachers appear to need more assistance also in the following areas:
In conclusion, teachers self-report via the questionnaire that they know about and implement best practice in science at a higher rate than was evidenced via the classroom observations. The evaluator does not find this too inconsistent. Changing attitudes takes less time than to change habits. What people say they do and what they actually do are not always the same. ESEP has heightened elementary teachers' awareness of science reform and the underlying principles of inquiry pedagogy; putting these principles into practice is much more difficult. Extent of District and School Contexts Being Supportive. To place the ESEP program on a continuum to indicate the supportiveness of context is not an easy task. This evaluator feels that the ESEP program has made great strides and has brought many barriers and issues to the foreground. While solutions to all such problems have not yet evolved, they are being worked on. For this reason, this evaluator would rate the supportiveness of context as a level 3 on HRI's 5-point scale. Support for the reform efforts as espoused by the ESEP program is "increasingly apparent, but the follow-through of such support from the district itself is still 'patchy.'" Extent of Institutionalization of High-Quality Professional Development Systems in the LSC District. To summarize the status of the professional development system within the targeted district for this first year's efforts, the evaluator assigns the ESEP Program in the APS school district a level 3 rating on HRI's 5-point scale. This rating denotes a positive direction from last year's rating. While APS has policies in place governing staff development in general, the system has little direct involvement at this time in the organizing and conducting of the training. Training events are scheduled, planned, and executed by members of the ESEP Executive Council. A level 3 rating signifies that the school district has some components in place to support an on-going, high-quality professional development system and efforts are underway to modify other key components of the system and work toward institutionalization. Summary and Recommendations The second year of the ESEP program was filled once again with remarkable successes. The training that ESEP provides continues to be met with much enthusiasm by the classroom teachers. The ESEP staffs at the different universities planned and implemented quality professional staff development workshops for more than 300 teachers (an unduplicated count) this past year. The program now has expanded its cadre of SKIL teachers to 38 APS teachers. Another way to examine the impact of ESEP is in the total number of contact hours; ESEP conducted 1,176 days of workshop or 7,056 hours of training. In addition to the efforts of ESEP staff external to APS, the school district has also made quite a commitment both in time and money. APS continues to purchase kits for the added grade-levels of implementation so teachers will not rely solely on textbooks for science instruction. To ensure that kits are refurbished and distributed, APS has funded and staffed a science kit center. This past year has shown that the center has established itself as exemplary as evidenced by other school districts, embarking on kit usage, now visit the APS center as a prototype. APS has also allowed time for training and fiscal support for substitute pay and teacher incentives. A major strength of the ESEP program this past year has been its improvement in long-range planning. In order to ensure that the goals of the grant are reached by the time the funding ends, the evaluator makes the following suggestions in order to improve the overall ESEP Program:
|
|
||||||||||||||||||||
