## PaperA Summary of Project Efforts to Examine the Impact of the LSC on Student Achievement
Serving schools in several states in its geographic region, the project finds it difficult to collect
common data from the schools it serves. Thus, the project has encouraged each school in their
LSC to undertake its own study. The project provided results from five of the thirty- three
participating schools to students in traditional mathematics classes. The schools used a variety of instruments including the mathematics portion of the Scholastic Achievement Test (SAT), a state mandated mathematics assessment, the New Standards Reference Exam (NSRE), and the Terra Nova. One school also used a self- developed problem- solving test. The first school categorized students into four groups, advanced IMP, regular IMP, advanced traditional, and regular traditional, though there was no information given as to how the advanced/ regular distinction was made. While this distinction may have been intended to control for initial differences in student ability levels, not enough information is provided to judge whether this goal was accomplished. The school found that, at both levels, students in IMP classes had higher mathematics SAT scores than students in non- IMP classes (see Table 3).
Table 3
Mathematics SAT Scores by Class Type
The study also shows that IMP students had higher verbal SAT scores than non- IMP students (see Table 4). While the school may have provided these data to show impact on verbal SAT (given the writing intensive nature of IMP), it raises the question about initial differences between the two groups of students, as there was only a marginal control for initial differences in student abilities. No significance testing was reported.
Table 4
Verbal SAT Scores by Class Type
The second school compared IMP and traditional students using the mathematics portion of the SAT and the state mathematics assessment. As can be seen in Table 5, students in IMP classes outperformed students in basic and standard level mathematics courses, but not those in honors courses (the IMP students were roughly 75% standard level students and 12.5 % each of basic and honors, though no information was provided as to how this categorization was made). They also showed that IMP juniors outperformed non- IMP juniors on a school- developed problem solving test (an average score of 3.4 out of 5 for IMP students compared to 2.5 for non- IMP students). No significance testing was reported.
Table 5
Student Performance by Track
The next school administered the Terra Nova test to all 9
Table 6
Terra Nova Results by Class Type
The fourth school compared mathematics SAT scores of IMP and non- IMP students, grouping the students in two levels - honors and regular. IMP students scored higher at both levels than did non- IMP students (see Table 7). No significance testing was reported and not enough information is provided to judge whether IMP and non- IMP students were initially equivalent.
Table 7
Mathematics SAT Scores by Class Type
The final school compared IMP students to non- IMP students using scores from the NSRE
administered at the 10 motivated students, the IMP course was offered only to students who had a raw SAT score of 40
or higher (though they could choose a traditional mathematics class). To compensate for initial
differences between IMP students (high achievers and self- selected) and non- IMP students, the
school constructed three matched- pair samples to use as comparison groups. They then used an
ANOVA to show that there were no significant differences on the 8 The results of the school's comparisons of IMP and non- IMP students can be seen in Table 8. On
the 10 control group on the 10 also found that the IMP students scored higher than the control group on the 10 Scale Score (an effect of .54 standard deviations). There were no differences among the groups on the Math Concepts scale, but IMP students scored higher on the Math Skills and Math Problem Solving scales (effect sizes of .75 and .56 respectively). |