 |  |  |
|
|||
|
|||
|
|||
| |
||
| |||
 |
          |
|
|
Conference Material
An emerging profile of the mathematical achievement of students in the Core-Plus mathematics project
The Core-Plus Mathematics Project (CPMP) is one of four comprehensive high school curriculum projects that were funded by NSF in 1992. CPMP has completed development and evaluation of student and teacher materials for an integrated three-year high school mathematical sciences curriculum for all students. The completed curriculum is published under the title Contemporary Mathematics in Context (Coxford, Fey, Hirsch, Schoen, Burrill, Hart, Watkins, Messenger, & Ritsema, 1998). Research and development are currently in progress on a flexible fourth-year course continuing the preparation of students for college mathematics.
In this paper, we provide a brief overview of the CPMP curriculum in terms of its design and theoretical framework and a profile of the mathematical achievement outcomes of students who participated in the national field test of the curriculum. The emphasis here is on large scale quantitative achievement test results that may be of particular interest to educational policy makers. More focused research studies conducted in CPMP classrooms are reported elsewhere (cf. Flowers, 1995; Wilson & Lloyd, 1995; Kett, 1997; Lloyd & Wilson, 1997; Truitt, 1998). These references and others are included in the bibliography of CPMP publications appended to this paper. The profile of achievement presented in this paper is an "emerging profile" in that the CPMP curriculum remains under development and its evaluation will continue for several more years. In Spring 1998, Course 3 is being revised and edited for publication on the basis of the 1996-97 field test. Selected data from the field test are still being processed and analyzed. Many of the Course 3 field-test students, now mostly seniors, have completed the ACT and/or SAT. That data is being collected and will eventually comprise an important part of the profile of CPMP student achievement. In addition, Course 4 is being pilot tested in 1997-98 and will be field tested the following year. The Course 4 field-test students will complete achievement measures that include college mathematics placement tests, which also will be an important part of the achievement profile. Thus, the achievement profile presented in this paper is based on much of the achievement data gathered during the first three years of the CPMP field test, but it does not include some important results that are still being compiled.
Development of the CPMP curriculum is based on several principles shaped by the practice of mathematics today, by research on teaching and learning, and by emerging technology.
In developing the CPMP three-year core curriculum, we employed a "zero-based" process in which the inclusion of a topic was based on its own merits. In particular, in designing a particular course, we always asked; "If this course is the last mathematics students will have the opportunity to learn, is the most important mathematics included?" The fourth-year course is being designed to provide a smooth transition to collegiate mathematics. The CPMP four-year curriculum builds upon the theme of mathematics as sense-making. Investigations of real life contexts lead to (re)invention of important mathematics that makes sense to students and that, in turn, enables them to make sense of new situations and problems. Throughout it acknowledges, values, and extends the informal knowledge of data, shape, change, and chance that students bring to situations and problems. Each course in the CPMP curriculum features interwoven strands of algebra and functions, geometry and trigonometry, statistics and probability, and discrete mathematics. Mathematical modeling and its related concepts of data collection, representation, interpretation, prediction, and simulation serve as an organizing principle for each of the strands. These four strands are connected within instructional units by common topics such as: symmetry, functions, matrices, and data analysis and curve-fitting. The strands also are connected across units by mathematical habits of mind such as: visual thinking, recursive thinking, searching for and describing patterns, making and checking conjectures, reasoning with multiple representations, inventing mathematics, and providing convincing arguments. The strands are unified further by the fundamental themes of data, representation, shape, and change. Each course in the CPMP core curriculum consists of seven units, each comprised of three to five multi-day lessons centered on big ideas, and a thematic capstone which enables students to pull together and apply the important mathematical concepts and methods developed in the course. Numerical, graphics, and programming/link capabilities of graphic calculators are assumed and capitalized on throughout each course. Use of this technology supports the emphasis of the curriculum and instruction on multiple representation (numeric, graphic, symbolic) and on goals in which mathematical thinking is central. The instructional materials are designed to promote a four-phase cycle of classroom activities, described below, designed to engage students in investigating and making sense of problem situations, in constructing important mathematical concepts and methods, and in communicating orally and in writing their thinking and the results of their efforts. Most classroom activities are designed to be completed by students working together collaboratively in heterogeneous groupings of two to four students. Each lesson is launched with a situation and related questions to think about which sets the context for the student work to follow. In the second or explore phase, students investigate more focused problems and questions related to the launch situation. This investigative work is followed by a class discussion in which students summarize mathematical ideas developed in their groups, providing an opportunity to construct a shared understanding of important concepts, methods, and approaches. Finally, students are given a task to complete on their own, assessing their initial understanding of the concepts and methods. Each lesson also includes tasks, intended primarily for out-of-class work, to engage students in modeling with, organizing, reflecting on, and extending their mathematical understanding. The CPMP curriculum and instructional model (Hirsch, Coxford, Fey & Schoen, 1995; Schoen, Bean & Ziebarth, 1996; Hirsch & Coxford, 1997) and professional development programs for teachers (Van Zoest & Ritsema, 1998) are described in more detail elsewhere.
Sample Each CPMP course was field tested in 36 high schools in Alaska, California, Colorado, Georgia, Idaho, Iowa, Kentucky, Michigan, Ohio, South Carolina, and Texas. A broad cross-section of students from urban, suburban, and rural communities with ethnic and cultural diversity is represented. Because of difficulties of interpretation of tests given at different times in the school year, data from three semester-block schools was analyzed separately. This report focuses on achievement results for the 33 field test schools who were on a regular two-semester schedule. The field test schools were encouraged to include students with a wide range of achievement and interest in mathematics, and, where possible, students were grouped heterogeneously. Limitations at local sites did not always make this possible. Course 1 teachers' descriptions of their entering CPMP students are summarized in Table 1.
Table 1.
About one-fifth of the teachers reported that their classes included the full range of ninth-grade students. The most common CPMP class (as reported by 43.0% of the teachers) was comprised of students with a wide range of prior mathematics achievement and interest. Often, however, honors students were not included because they completed the grade nine course in eighth grade and moved on to a tenth-grade mathematics course in grade nine. Thus, the CPMP field test sample, as reported by the teachers, included students with a wide range of prior achievement and interest in mathematics, but honors or accelerated students are probably underrepresented. Traditional Comparison Classes At the beginning of Course 1, eleven field test schools volunteered to pretest and posttest students in traditional, comparison classes. The comparison classes were comprised of 20 algebra 1, five pre-algebra, three general mathematics, and two honors geometry ninth-grade classes. The nature of the instruction in the comparison classes was not specified in advance, but at the end of the year comparison teachers described what transpired. For example, a variety of traditional textbooks were used. Small group work was reported to be used either not at all or less than once a week by about 80% of the comparison teachers. About 74% of the comparison teachers reported that their students used a calculator more than once per week, although there is no data about how it was used. Solving linear equations in one variable was the main instructional goal for an average of 23% of the class time for the year, with up to 50% of the time spent on this topic in some algebra I classes. The Course 2 comparison group consisted of all students who were in the Course 1 comparison group, completed a traditional sophomore mathematics class, and completed the Course 2 posttests. Only five of the 11 schools who had comparison groups in Course 1 were able to maintain them in Course 2. The main reason for this drop in number is that the Course 1 comparison students enrolled in a variety of mathematics classes in their sophomore year and were difficult to locate and posttest at the end of the year. By the end of Course 3, the number of comparison students from the original pretested group that were available for posttesting was so small that a Course 3 comparison group was not feasible. Instruments to Assess Achievement Standardized Tests One measure of mathematics achievement used in the CPMP field test is a standardized test called Ability to Do Quantitative Thinking (ATDQT), which is the mathematics subtest of the Iowa Tests of Educational Development (ITED) (Feldt, Forsyth, Ansley & Alnot, 1993). The ITED is a nationally standardized battery of high school tests developed by the Iowa Testing Programs, the same group that writes the widely used elementary school level Iowa Tests of Basic Skills (ITBS). The ATDQT is a 40-item multiple-choice test with the primary objective of measuring students' ability to employ appropriate mathematical reasoning in situations requiring the interpretation of numerical data and charts or graphs that represent information related to business, social and political issues, medicine, and science. The ATDQT correlates highly with other well-known measures of mathematical achievement. According to research conducted by the test's developers, correlation of the ATDQT, when given in grade nine, with the ITBS Mathematics total score in grade eight is .81; with students' final cumulative high school grade point average in mathematics courses is .59; with the ACT Mathematics test is .84; and with the SAT Mathematics test is .82. The ACT and SAT are usually completed in eleventh or twelfth grade. In addition to the ATDQT, a posttest comprised of released multiple-choice items from the 1990 or 1992 administration of the National Assessment of Educational Progress (NAEP) in twelfth-grade mathematics was administered at the end of Course 3. Items were chosen that measured outcomes that were of interest and provided a balance across the NAEP content (numbers & operations; measurement; geometry; data analysis, statistics & probability; algebra & functions) and process dimensions (concepts, problem solving, procedures). Performance Assessment Instruments The CPMP evaluation team developed open-ended achievement tests, called the Course 1 Posttest and Course 2 Posttest, each in two parts. Part 1 was designed to be a test of content that both the CPMP and the comparison students would have had an opportunity to learn that year, algebraic content for Course 1 and both algebraic and geometric content for Course 2. Part 2 of each CPMP Posttest also included subtests of Data Analysis, Discrete Mathematics, Probability, and (in Course 1) Geometry; that is, content that the comparison students did not have the opportunity to study. Thus, the comparison students completed only Part 1 of the CPMP Posttest at the end of each year, and CPMP students completed both parts. These tests required students to construct their responses and to show and often explain their work. Assessment Administration Schedule The paper-and-pencil, mathematics achievement portion of the CPMP field test used a pretest-posttest comparison group design for Courses 1 and 2 and pretest-posttest only for Course 3. The ATDQT test administered at the beginning of Course 1 served as the pretest for all courses, so the pretest-posttest analyses for Courses 1, 2, and 3 are for one, two, and three years of mathematics instruction, respectively. Students were allowed unrestricted use of a calculator (usually a TI-82 or TI-83) on all achievement tests. The administration schedule for the mathematics achievement tests is given in Table 2, and this is followed by a section in which the results are presented.
Table 2.
The results presented here are mainly quantitative summaries of achievement outcomes of Core Plus and comparison students across all field test schools and within educationally important subsets of students and schools. A broad profile of achievement levels for these subsets of students and schools across several important content areas and types of achievement measures is also provided. Specifically, the following results are presented:
The results given below were obtained by first converting each student's raw score to a standard score. The standard score is a number that describes the student's location on an achievement continuum, regardless of the ATDQT test form or the students' grade level. Means and other statistics were then computed using the standard scores. As needed for interpretation, these summary statistics were then converted to national student or school mean percentiles. Norms for the current edition of the ITED were compiled by the test developers in 1992 using a nationally representative sample of 13,935 high school students. Both student norms from the distribution of all students in the norm sample and school mean norms from the distribution of all school means in the norm sample are provided by the test publisher. Schoen and Ziebarth (1998) have reported results which are based on individual student scores as statistical unit. They found that in the 11 schools with comparison groups, the Course 1 posttest mean of CPMP students was significantly greater than that of the comparison students even though the comparison students' pretest mean was slightly higher. They also found a significant school by treatment interaction suggesting that achievement levels of CPMP students compared to those of comparison students differed significantly by school. To account for this interaction, results reported here use the school mean on the ATDQT as the statistical unit. Results are reported first for two cohort groups, Course 1 and Course 2. In each course, the cohort group consists of all CPMP students and all comparison students who completed both the ATDQT pretest (given at the beginning of Course 1) and the ATDQT posttest for that course (given in May near the end of each school year). Thus, the Course 1 cohort group results are indicative of CPMP's effect in Course 1, and the Course 2 cohort group results show the combined effect of Courses 1 and 2. Results for a third group, called the Course 3 cohort group, are also presented. The Course 3 cohort group consists of all students who completed the ATDQT pretest and the ATDQT posttest at the end of each of the three courses. The numbers of students in each cohort group are as follows:
One measure of the size of a treatment effect in a study with a pretest-posttest design is called the effect size which is defined as the difference between the pretest and posttest treatment means divided by the standard deviation of the pretest. Thus, the effect size is the number of standard deviations (of the pretest) that the mean changed from pretest to posttest. The standard deviation of the pretest, rather than posttest, is used because it is the best estimate of the variability of the group before it was affected by the treatment. One difficulty with that definition for a standardized test like ATDQT is that not all the growth can be attributed to the treatment. The norm group also grew from pretest to posttest and that average growth should be subtracted from the mean change from pretest to posttest. For a pretest mean at a particular beginning-of-year national percentile p, a good estimate of the posttest mean that reflects the average growth of the norm group is the standard score that corresponds to the same end-of-year percentile p. For our purposes, this standard score is called the "Norm Mean," and the following definition applies:
The "Adjusted Affect Size," then, for each cohort by treatment group is the number of pretest standard deviations the group grew from pretest to posttest, beyond the average growth of the ATDQT norm group. Table 3 summarizes the main results for Course 1 and 2 cohort groups by treatment and for the Course 3 cohort of CPMP students.
Table 3.
The adjusted effect sizes in the table show that the pretest to posttest gains of the cohorts of CPMP students are .26 to .43 standard deviations greater than the norm group's average gain. The gains for CPMP students are also greater than those for comparison students, and gains in Courses 2 and 3 are greater than those in Course 1. Concerning the statistical significance of the differences in CPMP and comparison group means, a CPMP versus comparison group analysis of covariance was conducted for the 11 schools with Course 1 comparison groups. With the Course 1 Pretest as covariate, the CPMP students' adjusted Course 1 Posttest mean was greater than that of the comparison students (p = .086). Similarly, an analysis of covariance was conducted for the five schools with Course 2 comparison groups. With the Course 2 cohort's Pretest school means as covariate, the CPMP adjusted Course 2 Posttest mean was greater than that of the comparison group (p = .027).
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
