Communication Center  Conference  Projects Share  Reports from the Field Resources  Library  LSC Project Websites  NSF Program Notes
 How to Use this site    Contact us  LSC-Net: Local Systemic Change Network
Educational Reform & Policy

Professional Development

Teaching and Learning

LSC Papers and Reports

Cross Site Reports

LSC Case Study Reports

Papers / Presentations authored by LSC Members

About LSC Initiatives and their impact

Bibliographies

Paper

  New!     

LSC Year Two Cross-Site Report

author: Iris R. Weiss, Kathleen A. Rapp, Diana L. Montgomery
published in: Horizon Research
published: 02/04/1998
posted to site: 02/04/1998

An Evaluative Look at Classrooms

Evaluators were asked to rate each lesson on specific indicators within five components of classroom practice: the quality of the lesson design; how well it was implemented; the quality of the science/mathematics content; the classroom culture; and the extent to which the lesson will enhance student capacity to engage in science and mathematics. Highlights of these ratings are described below.

Design

Table 17 shows the percent of observed lessons in Cohort 1 science, Cohort 2 science, and Cohort 2 mathematics classes that received high ratings (4 or 5 on a five-point scale) in each of a number of indicators within the design category. Note that the Cohort 1 science lessons, taught by teachers who had participated in LSC professional development, were more likely than the baseline Cohort 2 science lessons to receive high ratings on several design indicators: appropriateness of strategies for the purposes of the lesson; encouraging a collaborative approach to learning, and incorporating tasks, roles, and interactions, consistent with inquiry.

Baseline mathematics lessons had design ratings similar to baseline science lessons in some areas (encouraging collaboration, and incorporating inquiry strategies). They were rated as highly as Cohort 1 science lessons in appropriateness of strategies for accomplishing the purposes of the lesson, and higher than either of the science cohorts in the extent to which the instructional strategies appeared to meet the varied needs and learning styles of students.

Table 17
Lessons Rated 4 or 5 on Design Indicators
Percent of Lessons
Cohort 1 Science Cohort 2 Science Cohort 2 Mathematics
Strategies contributed to accomplishing purposes604656
Design encouraged collaborative approach483632
Design incorporated tasks, roles consistent with inquiry462116
Strategies appeared to meet varied student needs/learning styles414657

Figure 47 shows evaluators' holistic ratings of the design of the observed science and mathematics lessons. Note that Cohort 1 science lessons were considerably more likely to receive high ratings in the overall design than either baseline science or mathematics lessons.

Lessons with High Ratings for Design

Figure 47

Implementation

The percent of lessons receiving high ratings on each of a number of indicators of the quality of implementation is shown in Table 18. Note that the majority of teachers in each group were seen as confident in their ability to teach their subject. In contrast, relatively few teachers in any group received high marks for the extent to which their questioning strategies encouraged students' conceptual understanding.

Classes taught by LSC-participating teachers were considerably more likely than others to receive high ratings for incorporating inquiry-based strategies, into their instruction, while the Cohort 2 mathematics lessons received higher ratings than either science group on the appropriateness of the lesson pace.

Table 18

Lessons Rated 4 or 5 on Implementation Indicators

Percent of Lessons
Cohort 1 Science Cohort 2 Science Cohort 2 Mathematics
Teacher appeared confident676472
Pace of lesson was appropriate465259
Teacher attended effectively to student prior conceptions404534
Strategies consistent with inquiry402724
Questioning strategies enhanced conceptual understanding322030

Overall, one-third of the observed lessons received high marks (4 or 5) on implementation, including 28 percent of baseline science classes, 38 percent of baseline mathematics classes, and 41 percent of classes taught by teachers who had participated in LSC professional development. (See Figure 48.)

Lessons with High Ratings for Implementation

Figure 48

Content

As can be seen in Table 19, Cohort 1 science lessons were more likely than baseline science lessons to receive high ratings in each of the indicators of content quality. Differences were most pronounced for content accuracy and the extent to which concepts and processes were balanced appropriately.

Baseline mathematics lessons had content ratings similar to baseline science lessons in some content indicators, including appropriateness, balance of concepts and processes, and the extent to which the subject was portrayed as inquiry. Ratings on relevance and accuracy of content and the quality of the lesson closure were more similar to those of Cohort 1 science lessons.

Table 19
Lessons Rated 4 or 5 on Content Indicators

Percent of Lessons
Cohort 1 Science Cohort 2 Science Cohort 2 Mathematics
Teacher-presented information was accurate867084
Content appropriate for purposes of lesson665669
Content relevant to student needs/interests665668
Connections made to real-world applications524048
Concepts and processes appropriately balanced453132
Degree of closure appropriate383029
Science/mathematics portrayed as inquiry373021

Overall, 35 percent of baseline science classes, 38 percent of baseline mathematics classes, and 46 percent of Cohort 1 science classes received ratings of 4 or 5 for content. (See Figure 49.)

Lessons with High Ratings for Content

Figure 49

Classroom Culture

Evaluator ratings of a number of indicators related to classroom culture are shown in Table 20. Roughly two-thirds of the observed teachers in each Cohort received high marks for encouraging the active participation of all students. Other aspects of the classroom culture were not rated as highly, especially for baseline science and mathematics classes. For example, while evaluators assigned ratings of 4 or 5 to 55 percent of Cohort 1 science lessons for reflecting "collaborative working relationships among students," only 38 percent of Cohort 2 science lessons and 28 percent of Cohort 2 mathematics lessons were rated that highly.

Table 20
Lessons Rated 4 or 5 on Classroom Culture Indicators
Percent of Lessons
Cohort 1 Science Cohort 2 Science Cohort 2 Mathematics
Encourage active participation of all706568
Room arrangement facilitated interaction675054
Collaborative working relationship between students and teachers634452
High expectations for all students605655
Collaborative working relationship among students553828
Student ideas/conjectures encouraged493440
Students intellectually engaged with important ideas483733
Intellectual rigor and challenging of ideas encouraged353024

Overall, 32 percent of the observed lessons were rated at a level 4 or 5 for classroom culture, including 30 percent of baseline science lessons, 26 percent of baseline mathematics lessons, and 46 percent of Cohort 1 science lessons. (See Figure 50.)

Lessons with High Ratings for Classroom Culture

Figure 50

Enhancing the Ability of Students to Successfully Engage in Science and Mathematics

The final set of ratings on the classroom observation protocol asked evaluators to consider all of their previous ratings in assessing the likelihood that the lesson would enhance the ability of students. A number of key areas were specified, including understanding key science and mathematics concepts, understanding the nature of inquiry, being able to generalize their learning to real-life situations, and becoming self-directed learners. While evaluators often rated individual components of the lessons highly, their summary judgments of the likely impact of the lessons were generally less positive. For example, only 35 percent of the Cohort 1 science lessons, and even fewer Cohort 2 science and mathematics lessons, received high marks for the likelihood of enhancing students' understanding of the nature of inquiry. (See Table 21.)

Table 21
Lessons Rated 4 or 5 on Indicators of Enhancing Student Ability

Percent of Lessons
Cohort 1 Science Cohort 2 Science Cohort 2 Mathematics
Increase ability to become self-directed learners413039
Enhance conceptual understanding392938
Increase ability to apply knowledge to real-life situations353124
Enhance understanding of nature of inquiry352227

Overall, 27 percent of the observed lessons were rated a 4 or 5 in this area, including 23 percent of baseline science classes, 30 percent of baseline mathematics classes, and 36 percent of science classes taught by teachers who had participated in LSC professional development. (See Figure 51.)

Lessons with High Ratings for Likely to Enhance Student Ability

Figure 51

Overall Ratings of Observed Classes

As the final step in completing the observation protocol, evaluators were asked to indicate the "level" that best described each lesson they observed. Table 22 shows the percent of lessons in each Cohort considered effective instruction, beginning stages of effective instruction, and ineffective instruction. The shaded boxes contain examples of lessons that were assigned to each of these categories. Note that only 32 percent of Cohort 1 science lessons were rated as effective instruction, quite close to the 27 percent of Cohort 2 science lessons and 29 percent of Cohort 2 mathematics lessons that were rated that highly. The major difference between Cohort 1 and Cohort 2 science lessons is at the lower end of the scale, with lessons of treated teachers more likely to be considered in the beginning stages of effective instruction and less likely to be considered ineffective in structure.

Table 22
Overall Ratings of Observed Classes

Percent of Lessons
Cohort 1 Science Cohort 2 Science Cohort 2 Mathematics
Effective Instruction322729
Beginning stages of effective instruction453654
Ineffective Instruction293717

Lesson #1:
Ineffective Science Instruction-Passive "Learning"

The teacher stands at the front of a sparsely-equipped urban classroom and leads his third grade class, composed of mostly minority students, through a lesson about the life cycle of a butterfly. Using commercially-prepared packets, the teacher proceeds page-by-page, asking students to locate and hold up the correct page and assigning, round-robin fashion, the task of reading aloud. After reading, the teacher asks questions that require one-word answers, which he writes on the board, and students copy into their notebooks. The students then fill out worksheets. The information presented is limited to the names of the life stages; there is no reference to the processes of change and the lesson seems more like a reading/vocabulary lesson than science. Although desks are arranged in groups, there is no collaboration among students. The lesson appears too simple for the children, although they are eager to participate, to give the "correct" answers, and to fill in the blanks on their worksheets.
Lesson #2:
Ineffective Mathematics Instruction-Activity for Activity's Sake

As students in this seventh grade mathematics class arrive and take their seats, they open their notebooks and begin correcting their homework based on answers projected on the overhead screen. The teacher then turns off the overhead, and reviews previous work the class has done with measurement, especially alternative methods of measurement (such as arm span, finger length, stride length). She then instructs the class to open their math text to the page containing today's activity: measuring places on campus using alternative methods of measurement. Students are told to form groups of 2-4 and go outside to take measurements. There is little time for students to ask questions; some are clearly bewildered by the assignment, but the teacher is unaware of their reaction. Later, as students return from outside, they seem more focused on socializing than on math. The only closure at the end of the class is to tell students to present their measurements the next day.
Lesson #3:
Beginning Stages of Effective Mathematics Instruction

A kindergarten class engages in several activities to reinforce counting, adding, and subtracting skills. The teacher introduces the lesson by holding up Unifix cubes (two red, one white) and asking students the total number of cubes. When students offer answers, she asks them how they got their answers and reinforces their responses by reminding them there are different ways to arrive at answers. The lesson continues with a game using Unifix cubes and laminated game boards. The whole class sits in a circle, and students take turns rolling a die and adding or subtracting cubes on the game board with the teacher's help ("Lisa rolled a 5-how many would it take to get to 5 on your board if you already have 2 cubes on it?"). Students later pair off to play another game with the cubes. Students are engaged, and it is evident that the majority see the various ways to determine the total number of cubes (counting, adding, subtracting). However, little debriefing or discussion takes place after the activities.

Lesson #4:
Effective Science Instruction

As part of a plant unit that includes several related activities, including a visit to a neighborhood nursery, fourth grade students meet to discuss the progress of their extended experiments. Working in groups of four, they have previously planted six seed pots: three as a control group, and three with one variable changed (e.g., extra fertilizer, blue dye in water, watered with soda). The groups share the results of their experiments so far, demonstrating facility in working collaboratively, in reporting back to the class, and in using appropriately vocabulary words such as "control" and "variable." The teacher is confident and well-versed in science, but allows the students to lead discussions; all students, particularly girls, are noticeably at ease in their presentations, even when expressing some uncertainty or tentativeness about their ideas. To close the lesson, the teacher outlines the parameters for another experiment. Each group of students works together to choose their own variables to test, and they write a plan based on their discussion.

Lesson #5:
Effective Mathematics Instruction

To practice subtraction skills, a first grade teacher reintroduces a previously-played game involving a set number of Unifix cubes in a bowl or tub. Reviewing earlier lessons, the teacher asks for examples and students demonstrate various subtraction strategies including counting cubes, counting with fingers and elbows, and drawing pictures. Then students pair off to play several rounds of "The Tub Game": taking turns, one student hides some of the 12 Unifix cubes, and the partner determines the number of hidden cubes based on the number remaining in the tub. The student then writes a "number sentence" (equation), and checks with the partner to confirm that the equation matches the actual number of cubes taken from the tub. Students are engaged and collaborative, if somewhat competitive, and appear to be making the connection between the symbolic and the concrete. The lesson ends with a group discussion of difficulties, helpful strategies, and a few additional student demonstrations. The teacher's focus is on each student using the strategies that help her/him the most.

Summary

Teacher questionnaires and classroom observations provide a consistent picture of science and mathematics instruction in LSC districts. Teacher presentations are common, especially in mathematics classes, where 91 percent of teachers report presenting content information at least once a week and 89 percent of observed lessons included a formal presentation by the teacher. Hands-on and other investigative activities are also fairly common, with the majority of teachers in each group reporting that their classes do hands-on activities at least weekly, and most observed lessons including some kind of hands-on/investigative activity.

As is the case nationally, most teachers in LSC districts teach mathematics on a daily basis, but only about 1 in 4 has a science lesson each day. Instruction in these subjects has different emphases, as well. Mathematics lessons are more likely than science lessons to focus on reviewing concepts, using what has been learned to solve "real-world problems," having students explain concepts to one another and supply evidence to support their claims, and homework review. Science lessons are more likely than mathematics lessons to emphasize identifying prior student conceptions and developing conceptual understanding and more likely to use open-ended questions and have students write their reflections in a notebook or journal.

Cohort 1 science lessons, taught by teachers who had participated in LSC professional development, generally were more likely than baseline science or mathematics lessons to receive high ratings on lesson design, content, and implementation, as well as the quality of the classroom culture. Cohort 1 lessons were rated particularly high on the extent to which:

  • The discipline was portrayed as a process of inquiry, and the lesson incorporated tasks, roles, and interactions consistent with inquiry;
  • Concepts and processes were balanced appropriately;
  • The room arrangement encouraged student-centered interaction; and
  • Students were encouraged to generate ideas, conjectures, and propositions;
  • The degree of closure or resolution of conceptual understanding was appropriate for the purposes of the lesson.

However, when considering the lesson as a whole, evaluators were equally likely to give high ratings to baseline science and mathematics lessons as to science lessons taught by teachers who had participated in LSC professional development.


  1. These data have the advantage of reflecting the judgment of objective observers, rather than teachers' own interpretations of classroom practices. However the reader is reminded that the 1996 observation data reported here are limited to 273 classrooms (73 Cohort 1 science, 145 Cohort 2 science, and 52 Cohort 2 mathematics). As is the case with data provided in the teacher questionnaires, classroom observation data were weighted to represent the population of eligible teachers in each project (all targeted teachers for baseline data in Cohort 2 projects, and all treated teachers in Cohort 1 projects).
  2. "Ineffective instruction" includes lessons the observers categorized as level 1a, passive learning; level 1b, activity-for-activity's sake; and level 2, elements of effective instruction. Lessons rated a low or solid 3 are considered "beginning stages of effective instruction," while those rated a high 3, 4, or 5 are considered "effective instruction."

 to previous page   next page