Communication Center  Conference  Projects Share  Reports from the Field Resources  Library  LSC Project Websites  NSF Program Notes
 How to Use this site    Contact us  LSC-Net: Local Systemic Change Network
Newsclippings and Press Releases

LSC Reference Materials

LSC Case Study Reports

Annual Report Overviews

Summer Workshop Plans

Annual Report Overviews

  New!     

Annual Report Executive Summary

author: Wendy DenBesten, Dave Calhoun
submitter: Teacher Enhancement for Student Success (TESS)
published: 02/05/1998
posted to site: 02/05/1998

Fresno Unified School District

Carlos Arturo Garcia, Superintendent

 

 

 

 

Local Systemic Change

Evaluation Report

 

1996-97

 

 

 

 

Wendy L. DenBesten, B.A.

David O. Calhoun, M.A.

 

 

 

 

Approved Report from the

Office of Research, Evaluation and Assessment

 

 

 


David O. Calhoun, M.A. Robert P. Grobe, Ph.D.

Administrative Analyst Director

Fresno, California

October 1997

FRESNO UNIFIED SCHOOL DISTRICT
EXECUTIVE SUMMARY
1996-1997 LOCAL SYSTEMIC CHANGE EVALUATION REPORT

Fresno Unified School District's (FUSD) Teacher Enhancement for Student Success (TESS) project submitted in response to the National Science Foundation's (NSF) Local Systemic Change Initiative (LSC) provides for extensive professional development for teachers in mathematics and science instruction for a sustained period of time. Having just completed its second year of implementation, the goal of Fresno Unified's LSC project is to provide teacher enhancement for all K-8 teachers in instructional strategies consistent with national standards and state framework documents.

 

Background

Fresno Unified is the fourth largest district in the state of California, with a student population of approximately 79,000 students (92 schools) during the 1996-1997 school year. The sheer size, ethnic diversity, and socio-economic variability of the district has created significant needs for developing and implementing systemic reform in all instructional areas. The combined efforts of the Urban Systemic Initiative, TESS (LSC), and Design for Excellence, the 1992 district plan for systemic reform, are considered to be integral components of the Fresno Systemic Initiative (FSI). The LSC project (TESS) focuses specifically on improving science and mathematics instruction by providing intensive staff development to K-8 teachers and administrators.

 

In 1996-1997, all elementary school teachers and middle school mathematics and science teachers were involved in LSC related mathematics and/or science professional development. While the structure of the three-year implementation plan is slightly different for mathematics and science, both plans call for professional development activities that focus on implementation of national science and mathematics standards. Many project activities are tied to state reform initiatives. The district's adopted K-6 mathematics and science curricula are used as vehicles to educate teachers about effective instructional strategies and standards-based content.

 

Core Evaluation Questions

In order to effectively evaluate the implementation and progress of the LSC project, six Core Evaluation Questions were developed. Interpretive responses to questions related to these core evaluation questions form the basis for the 1996-1997 annual report. Each of these questions will be addressed in this report. The core evaluation questions are:

 

  1. What is the overall quality of the LSC professional development activities?
  2.  

  3. What is the extent of school and teacher involvement in LSC activities?
  4. What is the impact of the LSC professional development on teacher preparedness, attitudes, and beliefs about science and mathematics teaching and learning?
  5. What is the impact of the LSC professional development on classroom practices in science and mathematics?
  6.  

  7. To what extent are the district and school contexts becoming more supportive of the LSC vision for exemplary science and mathematics education?
  8.  

  9. What is the extent of institutionalization of high-quality professional development systems in the LSC districts?

 

Evaluation Procedures

To address the many components of the Core Evaluation Questions, several procedures were developed. These include:

 

    • Professional development observations
    • Classroom observations
    • Teacher interviews
    • Teacher questionnaires
    • Principal questionnaires
    • Interviews with the Principal Investigator and other LSC project staff
    • Professional development participant evaluations

Core Evaluation Question I:
What is the Overall Quality of the LSC Professional Development Activities?

Eleven professional development sessions were observed in the 1996-1997 school year. These observations along with teacher questionnaire and interview data and the professional development participant evaluations comprise the data sources used to answer this question.

 

Summary of professional development synthesis ratings. Synthesis ratings and the capsule descriptions of the professional development sessions provide an overall picture of the quality of science/mathematics professional development in FUSD. Table 1 displays mean ratings for mathematics and science professional development as well as the number of sessions rated at "4" or "5" on a 5-point scale with "5" representing "to a great extent."

 

Conclusions. Observations from this table and additional summary information from other sources which should be highlighted include:

 

  • The design of the professional development is high-quality and reflective of standards-based instruction (Overall mean ratings for design: Mathematics = 3.86, Science = 4.50).

 

  • The implementation of the professional development is high-quality and reflective of standards-based instruction (Overall mean ratings for implementation: Mathematics = 3.71, Science = 4.00).

 

  • Active participation of all and respect for differences is highly valued (Overall mean ratings for culture/equity: Mathematics = 3.71, Science = 4.00).

 

  • A vast majority of interviewed K-8 teachers have a mostly neutral to positive impression of the LSC related professional development (Mathematics: 83%, Science: 95%).

 

  • A majority of the professional development sessions (80%) received "generally average to high" to "consistently high" ratings from participants.

 

  • The professional development functioned well to prepare teachers to plan and implement exemplary mathematics and science instruction (Mean rating: Mathematics = 4.0, Science = 5.0)

 

  • The facilitators are knowledgeable and sensitive to participants' backgrounds and needs.

 

  • Science professional development sessions were highly investigative and collaborative in nature.

 

  • Many of the mathematics professional development sessions focused on developing teachers' knowledge of NCTM standards and the California Frameworks content strands.

 

  • Only one of the observed sessions focused explicitly on developing leadership capacity, and only a few included explicit attention to assessment strategies.

 

  • The science and mathematics departments have leadership components built into their professional development plan but a majority of teachers have not yet had the opportunity to participate in leadership activities.

 

  • Based on teacher questionnaire responses, teachers express the need for more time to collaborate with their peers to implement instructional activities, with only 20% (mathematics) and 19% (science) of LSC trained teachers indicating they have enough time. However, the LSC project seems to be providing teachers with significantly more time to work with other teachers to implement reform and reflect on what they have learned (differences between trained and not trained teachers of 22% and 14% in mathematics, and differences of 20% and 13% in science).

 

Overall, the impact of professional development on the participants' capacity to provide high-quality science/mathematics education is rated at a "4". Consistently strong ratings in all aspects of the design, implementation and culture of the professional development sessions along with the positive ratings given by participants indicate an organized and effective means of implementing long term changes in teachers' abilities to implement standards-based curriculum that reflects "best practices" in both mathematics and science education. At this point, only slight adjustments seem necessary in what is already a solid professional development system.

 

Building leadership capacity for all participants is an area that Fresno Unified is continuing to address. Both science and mathematics have leadership components in place as part of their professional development plan. Science in particular seems to have built an effective model through the use of staff developers and lead teachers working with their school sites. At this time the overall rating is a "3" indicating that while the professional development activities are highly likely to build leadership capacity for some participants (lead teachers), currently, not all teachers have this opportunity.

 

Core Evaluation Question II: What is the Extent of School and Teacher Involvement in LSC Activities?

Information related to the extent of involvement in LSC activities is primarily found in the teacher and principal questionnaires. Other sources of useful information were district demographic data and the LSC implementation plan.

 

In 1996-1997, all elementary teachers along with middle school mathematics and science teachers were scheduled for either mathematics or science professional development. Some teachers, as part of the implementation plan were involved in both mathematics and science professional development. Particular attention was given this year to inservicing new teachers. This need arose as a result of a statewide K-2 class size reduction initiative that provided districts with additional funding if K-2 classes were reduced to a maximum of 20 students. Attention was also focused on effectively coordinating professional development schedules between mathematics and science and providing inservice options for year-round schools.

 

Conclusions. A majority of LSC trained teachers reported they were involved at least 20 hours of professional development in 1996-1997 (mathematics = 85%, science = 63%). Science numbers may be low due to confusion of the LSC acronym with other reform effort acronyms (USI and CSIN) usually associated with professional development sessions. Additional details related to this question include:

 

  • An overwhelming majority of principals report their schools are in the initial to heavy stages of reform. (Mathematics = 99%, Science = 92%).

 

  • LSC trained teachers report more college coursework than not trained teachers in mathematics and science.

 

  • During 1996-1997, schools and teachers from all demographic sub-groups in the district were involved in LSC project activities because the project implementation plan calls for all schools to be involved in mathematics and/or science training as part of the three-year implementation cycle.

 

Core Evaluation Question III: What is the Impact of the LSC Professional Development on Teacher Preparedness, Attitudes, and Beliefs About Science and Mathematics Teaching and Learning?

Data related to the impact of professional development on teachers preparedness, attitudes, and beliefs comes primarily from responses to the teacher questionnaires for mathematics (N = 126) and science (N = 126), as well as the 118 classroom observations and 104 interviews conducted during the 1996-1997 school year.

 

Conclusions. FUSD teachers generally have positive opinions about their preparation, attitudes, and beliefs toward teaching mathematics and science. Disaggregation of the data into LSC trained and not trained teachers revealed many significant differences. These differences imply that LSC professional development is having a large impact on teachers. Specific examples include the following:

 

  • Significantly more LSC trained teachers report feeling prepared to teach science (LSC trained = 84%, not trained = 59%) and mathematics (LSC trained = 91%, not trained = 80%).

 

  • More LSC trained mathematics and science teachers agree they are well informed about national standards than not trained teachers. (Mathematics: LSC trained = 68%, not trained = 55%; Science: LSC trained = 67%, not trained = 33%)

 

  • LSC trained teachers report feeling significantly more prepared than not trained teachers in mathematics and science to implement standards-based, investigative instruction (e.g., providing concrete experiences before abstract, developing students' conceptual understanding, leading a class using investigative strategies).

 

LSC trained teachers indicate feeling significantly more prepared to engage students in hands-on, inquiry-oriented science than not trained teachers. LSC training also appears to be impacting teachers' feelings of preparedness to teach science in general and several specific content areas. This information is summarized in Table 2.

 

 

LSC training also appears to have affected teachers' confidence and enjoyment in teaching mathematics and science. Table 3 illustrates the differences between trained and not trained teachers.

 

Table 3

Teachers Enjoyment of and Confidence in Their Teaching a

 

Percent of teachers rating their enjoyment of teaching at a "5" on a 5-point scale.

Percent of teachers rated as confident ("4" or higher on a 5-point scale) in their teaching abilities.*

 

LSC Trained

Not Trained

LSC Trained

Not Trained

Mathematics

42%

31%

62%

33%

Science

38%

23%

82%

52%

a Enjoyment: N for each cell is at least 40. Confident: N for each cell is at least 18.

*Differences between LSC trained and not trained teachers were statistically significant at the p < .05 level.

 

Teachers in general are somewhat prepared to implement alternative assessment activities such as utilizing open-ended test questions and performance-based assessment. Based on differences between trained and not trained teachers, LSC training appears to have increased levels of preparation for science teachers related to use of alternative assessment strategies but has had little impact in mathematics.

 

Areas identified as needing further attention in the coming year include: (a) the generally lower levels of preparation and knowledge in science, (b) teacher attitudes regarding traditional types of instruction in mathematics, and (c) the need for additional preparation in the use of alternative assessment in mathematics and science.

 

 next page