Paper

A Summary of Project Efforts to Examine the Impact of the LSC on Student Achievement

Project 4 (K- 12 Mathematics)

This study examined the results of the school district's implementation of a standards- based educational system using student achievement scores. The district used two national assessments for this purpose. The first was the New Standards Mathematics Reference Examination for grade 4 that contains sub- scales for skills, concepts, and problem solving. The second assessment used was the Iowa Test of Basic Skills (ITBS), chosen to verify that students using the new reform- oriented curriculum did not suffer in basic skills.

The district first administered the New Standards exam in 1996. Comparing achievement scores from 1996 to 1998, the district showed a significant increase in the percent of students meeting or exceeding the standard in all three areas: skills, concepts, and problem solving. Further, data from the ITBS show a small but significant increase in student achievement, lending evidence to the claim that students experiencing a reform- oriented curriculum do not do worse in basic skills, and may in fact do better than students experiencing a traditional curriculum.

Taking the analysis a step further, the project then compared schools based upon whether they were rated as strong or weak implementers of the mathematics program. Ratings of individual teachers were made by teacher leaders at each site and were based upon 1 st through 4^th grade

teachers' use of the curriculum (Everyday Mathematics) as intended. The three sites classified as weak schools were those where all but one or two teachers in grades 1 through 4 were rated as weak implementers. In order to be classified as a strong implementing school, all 3rd and 4^th

grade teachers had to be strong implementers (8 schools met this criterion). To protect against initial differences between strong and weak schools, the strong schools were further split into two groups. One contained three strong schools with similar demographics to the weak schools (number of students, percent free/ reduced lunch, percent African- American, etc.) and the other contained the remaining strong schools.

Students in both groups of strong implementing schools outperformed those from weak implementing schools on all three sub- scales of the New Standards exam and on the ITBS. Further, while a sizable achievement gap existed between white and African- American students at all schools, both groups in strong implementing schools outperformed their counterparts at weak implementing schools.

Project 5 (6- 12 Mathematics)

This project looked at the percent of students passing the state's 8^th grade mathematics

assessment, comparing average passing rates for districts within the project to the state as a whole. As can be seen in Table 9, districts in the LSC averaged a 10% increase in their pass rates (from 36% to 46%) one year after the implementation of the LSC compared to a 7% increase state- wide (from 59% in 1998 to 66% in 1999). Unfortunately, the study does not build a case as to why or how the LSC is responsible for the larger gains in LSC district. As the LSC districts started with a pass rate well below the state average, it could be argued that the increase is to some extent due to regression to the mean.

Science Studies

HRI was able to obtain the results of studies from four science projects. While there is quite a bit of variability in the quality of the study designs, in general, the science studies did a better job of controlling for threats to internal validity. As with the mathematics studies, the results of the science studies are generally positive.

Project 6 (K- 8 Science)

With no state assessment in science, individual districts are given the prerogative to choose when and how to measure student achievement in science. Although they use a variety of instruments, most districts do administer a science assessment at the 4^th and 8^th grade level. These include the

ITBS, CTBS, and SAT/ OLSAT. Thus, the project was able to collect student achievement trend data from sixteen districts it serves. However, as nine of the districts are very small, rural districts, the project aggregated their data into one composite district, leaving eight districts (seven of the original sixteen and the one composite district) to be analyzed. Of these, four have baseline data (student test scores from the year prior to LSC implementation) giving them four data points. Three districts have three years of data, and one district has two years of data.

From project records, the average number of professional development hours per school as well as the level of kit- usage (low, medium, and high) for each district were computed. Then, districtwide scores were examined (visually) to see if any patterns emerged. The data, shown in Table 10, reveal no clear trends in the relationship between PD hours and changes in student achievement scores or between kit usage and increased achievement. However, given the inadequacy of the data available (district means of student scores and measures of participation in the LSC), it is highly unlikely that any changes, good or bad, could be detected, much less an argument be made that the LSC was responsible for those changes.

Project 7 (K- 8 Science/ Mathematics)

The study, one of the strongest submitted to HRI, made comparisons of students' test scores on the SAT- 9 open- ended assessment, grouping students by the number of years they had a LSC trained teacher. The analysis was done separately for two grade levels, 5^th and 7^th grades with over 1000 students participating at each grade level. Additionally, students' reading test scores (3rd and 4^th grade respectively for the two analyses) were used to control for initial differences in student abilities.

As can be seen in Table 11, students at the 5^th grade level who had a LSC trained teacher for one or two years outperformed students who had never had a LSC trained teacher by about 3.5 points (.17 standard deviations).