Page 20 - S44 Compendium

Disability • Economically Disadvantaged • Independent Measure
18
SAGINAW PUBLIC SCHOOLS, MI
STUDY PROFILE
Evaluation Period: 2011–2012
Grades: 4–8
Assessment: Comprehensive Test of Phonological Processing (CTOPP) Elision subtest, Test of Word Reading Ef ciency (TOWRE) Sight Word Ef ciency and Phonetic Decoding Ef ciency subtests, Test of Silent Reading Ef ciency and Comprehension (TOSREC), Scholastic Reading Inventory (SRI), Scholastic Phonics Inventory (SPI)
Participants: N=317
Implementation: 60 minutes daily (Stand-Alone)
OVERVIEW
Saginaw Public Schools (SPS) enrolls approximately 9,000 students in Grades PreK through 12. The majority of students in SPS are African American (65%), 20% are Caucasian, 13% are Hispanic, 1% are Asian/Paci c Islander, and less than 1% are American Indian/Alaskan Native. Eighty-one percent of students are eligible for free or reduced-price lunch.
During the 2011–2012 school year, students from 12 elementary schools and four middle and K–8 schools in SPS were selected to participate in a randomized controlled trial study led by a third-party research rm, RMC Research. In order to be eligible
to participate, students had to meet the following three criteria: 1) perform below the 50th percentile on the Michigan Educational Assessment Program (MEAP); 2) score below 600 Lexile (L) measures on SRI; and 3) demonstrate foundational reading de ciencies (Beginning or Developing Decoder) on SPI. Eligible students who were placed into the System 44 classrooms at SPS during the 2011–2012 school year were expected to receive 60 minutes of System 44 instruction daily.
RESULTS
Implementation Results
Overall, teachers expected System 44 to be more effective
than their prior year’s program in the ve foundational literacy skills listed above (phonemic awareness, phonics, vocabulary, uency, and comprehension). These expectations were realized in phonemic awareness, phonics, vocabulary, and uency according to Spring 2012 ratings of System 44 effectiveness. The differences between the perceived effectiveness of the prior program and the System 44 program, with respect to teaching phonemic awareness and phonics, were statistically signi cant.
Gold standard study reveals System 44 students outperform comparison group students
on measures of word reading uency and comprehension.
Impact Results Overall
System 44 students performed signi cantly better than control group students on two of the individual standardized tests of word- level reading: CTOPP Elision (effect size of .27) and TOWRE Sight Word Ef ciency (effect size of .16). This represents percentile gains of 11 points and six points, respectively. SPI and SRI outcomes also showed positive gains for the System 44 students over the control group students. The impact was signi cant on SRI (effect size of .32). This represents a percentile gain of 13 points (Graph 1).
Impact Results for Students With Disabilities
Main effects for disability were revealed. The positive impact
for students with disabilities was signi cantly larger than for the students overall on the CTOPP Elision (effect size of .36) and TOWRE Sight Word Ef ciency (effect size of .24). This represents percentile gains of 14 points and nine points, respectively. The positive impact was also signi cantly larger on SPI Sight Word Fluency (effect size of .28). This represents a percentile gain of 11 points. In addition, the impact was signi cant on SRI (effect size of .34). This represents a percentile gain of 13 points (Graph 1).
Impact Results for Middle School Students
The System 44 middle school students performed signi cantly better than the control group students on three of the individual standardized tests of word-level reading: CTOPP Elision (effect size of .30), TOWRE Sight Word Ef ciency (effect size of .24),
and TOSREC (effect size of .20). This represents percentile gains of 12 points, nine points, and eight points, respectively. When disaggregated by students with disabilities, the signi cance held for the CTOPP Elision (effect size of .12) (Graph 2). The impact was signi cantly greater for the System 44 middle school students than the control group middle school students on SRI (effect size of .49). This represents percentile gains of 18 points, 22 points, and 19 points, respectively. When disaggregated by students with disabilities, the signi cance held for SRI (effect size of .31) and SPI Sight Word Fluency (effect size of .28). This represents percentile gains of 12 points and 11 points, respectively.