 |
|
|
|
|
author:
|
Dan Heck, Horizon Research, Inc.
|
|
presented at:
|
Studying the Effects of the LSC on Students
|
|
published:
|
02/02/2001
|
|
posted to site:
|
02/02/2001
|
Proposal for Measuring the LSC Program's Effects on Student Achievement in Science
Slides Presented by Dan Heck of Horizon Research Inc.
"HRI has proposed a study to meet the challenge of investigating program-wide effects on student achievement in science, without imposing undue burden on projects."
(taken from "Horizon Reasearch Inc. "Proposal for Measuring the LSC Program's Effects on Student Achievement in Science" distributed by HRI in the conference materials binder.)
Slides of talk follow:
Why a K-8 Science Study?
- NSF has needs for program wide studies of LSC effects on students
- Many science projects have internal needs for studying effects on students
- Most science projects include grades 4-6
- Most science projects do not have access to science achievement data
Instrumentation
- Achievement test made up of multiple choice items taken from 4th and 8th grade NAEP and TIMSS
- Sub-scales: Nature of Science, Physical Science, Earth Science, Life Science
- Projects are encouraged to add open-ended and/or performance items to the achievement instrument
- Teacher survey:
- demographics
- background
- content coverage
- instructional materials
- professional development
- student demographics
- Student demographic survey
Sampling
- Project chooses one grade level
- (4th, 5th, or 6th)
- HRI randomly samples 4-6 classes
- About 120 students per project
- About 3500 students program wide
- Demographics of sample compared to demographics of districts to examine representativeness
- Individual projects may choose to include a larger sample (piggy-back) in order to study project level effects of the LSC on student achievement
Design
- Pre-test, Post-test during a single academic year
- Comparison groups
- Exposure to content area (life, physical, earth science)
- Exposure to content area based on LSC-designated instructional materials
- Teacher's overall LSC PD
- Teacher's LSC PD in each content area
- Disaggregation (Achievement "gaps")
- Socioeconomic status
- English-language proficiency
- Gender
- Race/ethnicity
Internal Validity
- Pilot testing the achievement instrument
- Confirm psychometric properties of items
- Investigate instructional sensitivity of items
- Examine representativeness of sample
- Compare demographics of sample to population (district demographics)
- Compare treatment levels (PD, use of instructional materials) to information from other studies of the LSC program
Analysis
- Descriptive statistics
- Information on the sample
- Basic information on status of grade 4-6 science achievement across the LSC program
- No project-specific statistics
- Inferential statistics
- Series of hierarchical linear models (HLM)
- Outcomes:
- Achievement in overall science
- Achievement in nature of science
- Achievement in earth science
- Achievement in physical science
- Achievement in life science
- Controlled for prior knowledge (pre-test)
- Achievement gains predicted by
- Exposure to content area
- Use of LSC-designated instructional materials
- Teacher's overall LSC PD
- Teacher's LSC PD in each content area
- Achievement gaps predicted by
|
|
