Helping young children become proficient readers is a critical goal. Research tells us that students who experience difficulty reading in the early years of school often struggle to catch up (Stanley, Petscher, & Catts, 2018; Ozernov, , Palchik et al., 2016; Cunningham & Stanovich, 1997; Francis, Shaywitz, Stuebing, Shaywitz, & Fletcher, 1996). This study1 focuses on an innovative curriculum for kindergarten that closely integrates literacy instruction and science exposure. The research study combines a rigorous randomized controlled trial with in-depth cost and implementation studies to investigate impacts

Bowden, Brooks

Fink, Ryan

Gray, Abigail

Sirinides, Philip

English

ScholarlyCommons@Penn

University of Pennsylvania ScholarlyCommons CPRE Policy Briefs Consortium for Policy Research in Education (CPRE) 4-2020 Zoology One Efficacy Evaluation Summary of Findings (April 2020) Abigail Gray Consortium for Policy Research in Education Philip Sirinides Penn State Harrisburg Ryan Fink University of Pennsylvania, ryanfi@gse.upenn.edu Brooks Bowden University of Pennsylvania, bbowden@upenn.edu Follow this and additional works at: https://repository.upenn.edu/cpre_policybriefs  Part of the Educational Assessment, Evaluation, and Research Commons, and the Elementary Education Commons Recommended Citation Gray, Abigail; Sirinides, Philip; Fink, Ryan; and Bowden, Brooks. (2020). Zoology One Efficacy Evaluation Summary of Findings (April 2020). CPRE Policy Briefs. Retrieved from https://repository.upenn.edu/cpre_policybriefs/87 This paper is posted at ScholarlyCommons. https://repository.upenn.edu/cpre_policybriefs/87 For more information, please contact repository@pobox.upenn.edu. Zoology One Efficacy Evaluation Summary of Findings (April 2020) Abstract Helping young children become proficient readers is a critical goal. Research tells us that students who experience difficulty reading in the early years of school often struggle to catch up (Stanley, Petscher, & Catts, 2018; Ozernov, , Palchik et al., 2016; Cunningham & Stanovich, 1997; Francis, Shaywitz, Stuebing, Shaywitz, & Fletcher, 1996). This study1 focuses on an innovative curriculum for kindergarten that closely integrates literacy instruction and science exposure. The research study combines a rigorous randomized controlled trial with in-depth cost and implementation studies to investigate impacts. Keywords Zoology One Efficacy Evaluation Disciplines Education | Educational Assessment, Evaluation, and Research | Elementary Education This policy brief is available at ScholarlyCommons: https://repository.upenn.edu/cpre_policybriefs/87 IntroductionHelping young children become proficient readers is a critical goal. Research tells us that students who experience difficulty reading in the early years of  school often struggle to catch up (Stanley, Petscher, & Catts, 2018; Ozernov,, Palchik et al., 2016; Cunningham & Stanovich, 1997; Francis, Shaywitz, Stuebing, Shaywitz, & Fletcher, 1996). This study1 focuses on an innovative curriculum for kindergarten that closely integrates literacy instruction and science exposure. The research study combines a rigorous randomized controlled trial with in-depth cost and implementation studies to investigate impacts. The setting: An urban district focused on literacyThe study’s setting—the School District of  Philadelphia (SDP)—makes this inquiry particularly salient. SDP is a large, urban district serving a diverse and economically challenged student population. Decades-long trends of  underperformance in literacy have led SDP to prioritize and invest in early literacy. This study offers a comparison of  two approaches to evidenced-based literacy instruction implemented in SDP schools—a business-as-usual approach and a curriculum that includes science integration. 1 The research reported here was supported by the Institute of  Education Sciences, U.S. Department of  Education, through Grant R305A160109 to the University of  Pennsylvania. The opinions expressed are those of  the authors and do not represent views of  the Institute or the U.S. Department of  Education2  The program has been renamed ARC CORE  Research Design • Multi-site randomized controlled study investigated the impacts of  American Reading Company’s Zoology One: Kindergarten Research Labs2 on kindergarten students’ achievement, motivation, and learning behaviors, as compared with business as usual literacy instruction. • Students in the treatment condition experienced Zoology One in place of  regular literacy instruction. Students in the control condition experienced SDP’s regular program of  literacy instruction. • Data were collected from 71 classrooms (treatment and control) in 21 schools, encompassing 1,589 students in two kindergarten cohorts. • The majority of  the data were collected in 2016-17 and 2017-18 while students were in kindergarten. Some analyses examined longitudinal impacts. Additional investigation of  longitudinal impacts is forthcoming. • Baseline equivalence was established between treatment and control.• In the total sample, approximately 8% of  students had English-language learner designation; 75% were eligible for free/reduced lunch; 8% had IEPs; 15% spoke a language other than English at home. 50% of  the students were female.• Key measures were the Woodcock Reading Mastery Test (WRMT); the AIMSWeb curriculum-based assessment; the Kaufman Test of  Educational Achievement in Writing (KTEA); and the Kindergarten Reading Motivation Scale (KRMS). Celebrating Thirty-five Years of Research ExcellenceZoology One Efficacy EvaluationSummary of Findings (April 2020)Abigail Gray, Consortium for Policy Research in Education Philip Sirinides, Penn State University Ryan Fink, Consortium for Policy Research in Education Brooks Bowden, University of  Pennsylvania1,589 7121StudentsClassroomsSchools1. Zoology One students outperformed control students in reading comprehension. Impact models comparing students in the treatment and control groups on the Passage Comprehension WRMT subtest revealed that Zoo students scored significantly higher. The effect size for this difference is 0.17 standard deviations.2. Zoology One students outperformed control students in letter-naming fluency. Analysis of  SDP’s AIMSWeb Letter Naming Fluency data revealed a positive and significant group mean difference for treatment students. The effect size for this difference is 0.27 standard deviations.3. Zoology One students outperformed control students in motivation to read. Although the KRMS revealed high reading motivation across the sample, participating in Zoology One increased reading motivation by .32 standard deviations.4. Zoology One students scored no better or worse than control students in decoding overall. Students of  teachers who implemented Zoology One with high fidelity achieved significant impacts in decoding. The WRMT Word Attack (reading nonsense words) and Word Identification (sight word reading) revealed no significant differences between treatment and control in the overall sample. However, exploratory analysis compared literacy impacts for students of  high-fidelity implementers with those of  low-fidelity implementers (quartiles).  This contrast revealed statistically significant impacts on WRMT Word Attack and Word Identification subtests.   5. Zoology One students scored no better or worse in writing overall. Students of  teachers who implemented Zoology One with high fidelity achieved significant impacts in writing. Comparison of  group mean differences on KTEA Writing revealed no significant differences between treatment and control students in the overall sample. However, exploratory analysis compared literacy impacts for students of  high-fidelity implementers with those of  low-fidelity implementers (quartiles).  This contrast revealed statistically significant impacts on KTEA Writing.6. Zoology One students were rated higher than control students on five learning behaviors by their 1st and 2nd grade teachers. Former Zoology One students were categorized by their first- and second- grade teachers as either “Strongest in Class” or “Above Average” at significantly higher rates than former control students on reading independence, confidence, verbal expression, love of  learning, and interest in science.  7. Girls in Zoology One classrooms defy gender stereotypes for reading interests. Girls in treatment classrooms were statistically significantly less likely than girls in control classrooms to indicate that they preferred books with topics identified as stereotypically female (princesses, Barbies, etc). Treatment-group membership reduced the likelihood of  girls’ stereotypical book topic selection by 26%.8. Zoology One costs about $480 per student to produce these impacts. About two-thirds of  these costs were for Zoology One curricular materials and teacher coaching in school, and the remaining one-third of  costs reflect the home reading component. On average, three fewer supplemental literacy programs were used in Zoo classrooms, resulting in an average expenditure reduction of  $40 per student. 9. Teachers believe Zoology One impacts learning. A majority of  teachers who offered perceptions of  Zoology One’s impact on their students’ literacy reported that their students’ literacy improved more during Zoology One implementation than in past years.  Teachers who believed that their students’ reading had been impacted by the curriculum often referred to the value of  the independent reading time. 10. Teachers believe Zoology One changed their perspectives or beliefs as teachers. A majority of  teachers interviewed discussed ways that their pedagogy and beliefs were impacted by Zoology One. Themes included change in their own perceptions of  students’ abilities to engage with and learn from the curriculum, and their beliefs about how literacy should be taught.  Next StepsWe will continue to explore longitudinal impacts on reading, science, learning behaviors, and other outcomes. Funding for this work has been approved.Kindergarten Impacts On average, classrooms using Zoology One achieved the same or better results than control classrooms while using three fewer instructional programs. Zoology One Efficacy Evaluation, Summary of  Findings (April 2020)Consortium for Policy Research in Education | RB 2020 - 01

Zoology One Efficacy Evaluation Summary of Findings (April 2020)

Kosmopolis

University of Pennsylvania 
ScholarlyCommons 
CPRE Policy Briefs Consortium for Policy Research in Education (CPRE) 
4-2020 
Zoology One Efficacy Evaluation Summary of Findings (April 
2020) 
Abigail Gray 
Consortium for Policy Research in Education 
Philip Sirinides 
Penn State Harrisburg 
Ryan Fink 
University of Pennsylvania, ryanfi@gse.upenn.edu 
Brooks Bowden 
University of Pennsylvania, bbowden@upenn.edu 
Follow this and additional works at: https://repository.upenn.edu/cpre_policybriefs 
 Part of the Educational Assessment, Evaluation, and Research Commons, and the Elementary 
Education Commons 
Recommended Citation 
Gray, Abigail; Sirinides, Philip; Fink, Ryan; and Bowden, Brooks. (2020). Zoology One Efficacy Evaluation 
Summary of Findings (April 2020). CPRE Policy Briefs. 
Retrieved from https://repository.upenn.edu/cpre_policybriefs/87 
This paper is posted at ScholarlyCommons. https://repository.upenn.edu/cpre_policybriefs/87 
For more information, please contact repository@pobox.upenn.edu. 
Zoology One Efficacy Evaluation Summary of Findings (April 2020) 
Abstract 
Helping young children become proficient readers is a critical goal. Research tells us that students who 
experience difficulty reading in the early years of school often struggle to catch up (Stanley, Petscher, & 
Catts, 2018; Ozernov, , Palchik et al., 2016; Cunningham & Stanovich, 1997; Francis, Shaywitz, Stuebing, 
Shaywitz, & Fletcher, 1996). This study1 focuses on an innovative curriculum for kindergarten that closely 
integrates literacy instruction and science exposure. The research study combines a rigorous randomized 
controlled trial with in-depth cost and implementation studies to investigate impacts. 
Keywords 
Zoology One Efficacy Evaluation 
Disciplines 
Education | Educational Assessment, Evaluation, and Research | Elementary Education 
This policy brief is available at ScholarlyCommons: https://repository.upenn.edu/cpre_policybriefs/87 
Introduction
Helping young children become proficient readers is a critical 
goal. Research tells us that students who experience difficulty 
reading in the early years of  school often struggle to catch up 
(Stanley, Petscher, & Catts, 2018; Ozernov,, Palchik et al., 2016; 
Cunningham & Stanovich, 1997; Francis, Shaywitz, Stuebing, 
Shaywitz, & Fletcher, 1996). This study1 focuses on an 
innovative curriculum for kindergarten that closely integrates 
literacy instruction and science exposure. The research study 
combines a rigorous randomized controlled trial with in-depth 
cost and implementation studies to investigate impacts. 
The setting: An urban district focused on 
literacy
The study’s setting—the School District of  Philadelphia 
(SDP)—makes this inquiry particularly salient. SDP is a 
large, urban district serving a diverse and economically 
challenged student population. Decades-long trends of  
underperformance in literacy have led SDP to prioritize 
and invest in early literacy. This study offers a comparison 
of  two approaches to evidenced-based literacy instruction 
implemented in SDP schools—a business-as-usual approach 
and a curriculum that includes science integration. 
1 The research reported here was supported by the Institute of  
Education Sciences, U.S. Department of  Education, through Grant 
R305A160109 to the University of  Pennsylvania. The opinions 
expressed are those of  the authors and do not represent views of  the 
Institute or the U.S. Department of  Education
2  The program has been renamed ARC CORE 
 
Research Design 
• Multi-site randomized controlled study investigated the 
impacts of  American Reading Company’s Zoology One: 
Kindergarten Research Labs2 on kindergarten students’ 
achievement, motivation, and learning behaviors, as 
compared with business as usual literacy instruction. 
• Students in the treatment condition experienced Zoology 
One in place of  regular literacy instruction. Students in 
the control condition experienced SDP’s regular program 
of  literacy instruction. 
• Data were collected from 71 classrooms (treatment and 
control) in 21 schools, encompassing 1,589 students in 
two kindergarten cohorts. 
• The majority of  the data were collected in 2016-17 and 
2017-18 while students were in kindergarten. Some 
analyses examined longitudinal impacts. Additional 
investigation of  longitudinal impacts is forthcoming. 
• Baseline equivalence was established between treatment 
and control.
• In the total sample, approximately 8% of  students had 
English-language learner designation; 75% were eligible 
for free/reduced lunch; 8% had IEPs; 15% spoke 
a language other than English at home. 50% of  the 
students were female.
• Key measures were the Woodcock Reading Mastery Test 
(WRMT); the AIMSWeb curriculum-based assessment; 
the Kaufman Test of  Educational Achievement in 
Writing (KTEA); and the Kindergarten Reading 
Motivation Scale (KRMS). 
Celebrating Thirty-five 
Years of Research Excellence
Zoology One 
Efficacy Evaluation
Summary of Findings (April 2020)
Abigail Gray, Consortium for Policy Research in Education 
Philip Sirinides, Penn State University 
Ryan Fink, Consortium for Policy Research in Education 
Brooks Bowden, University of  Pennsylvania
1,589 
71
21
Students
Classrooms
Schools
1. Zoology One students 
outperformed control students 
in reading comprehension. 
Impact models comparing 
students in the treatment and 
control groups on the Passage 
Comprehension WRMT subtest 
revealed that Zoo students scored 
significantly higher. The effect size 
for this difference is 0.17 standard 
deviations.
2. Zoology One students 
outperformed control students 
in letter-naming fluency. 
Analysis of  SDP’s AIMSWeb 
Letter Naming Fluency data 
revealed a positive and significant 
group mean difference for 
treatment students. The effect size 
for this difference is 0.27 standard 
deviations.
3. Zoology One students 
outperformed control students 
in motivation to read. Although 
the KRMS revealed high reading 
motivation across the sample, 
participating in Zoology One 
increased reading motivation by 
.32 standard deviations.
4. Zoology One students scored 
no better or worse than control 
students in decoding overall. 
Students of  teachers who 
implemented Zoology One with 
high fidelity achieved significant 
impacts in decoding. The WRMT 
Word Attack (reading nonsense 
words) and Word Identification 
(sight word reading) revealed 
no significant differences 
between treatment and control 
in the overall sample. However, 
exploratory analysis compared 
literacy impacts for students of  
high-fidelity implementers with 
those of  low-fidelity implementers 
(quartiles).  This contrast revealed 
statistically significant impacts on 
WRMT Word Attack and Word 
Identification subtests. 
 
 
5. Zoology One students scored 
no better or worse in writing 
overall. Students of  teachers who 
implemented Zoology One with 
high fidelity achieved significant 
impacts in writing. Comparison of  
group mean differences on KTEA 
Writing revealed no significant 
differences between treatment 
and control students in the overall 
sample. However, exploratory 
analysis compared literacy impacts 
for students of  high-fidelity 
implementers with those of  low-
fidelity implementers (quartiles).  
This contrast revealed statistically 
significant impacts on KTEA 
Writing.
6. Zoology One students were 
rated higher than control 
students on five learning 
behaviors by their 1st and 2nd 
grade teachers. Former Zoology 
One students were categorized 
by their first- and second- grade 
teachers as either “Strongest in 
Class” or “Above Average” at 
significantly higher rates than 
former control students on reading 
independence, confidence, verbal 
expression, love of  learning, and 
interest in science.  
7. Girls in Zoology One 
classrooms defy gender 
stereotypes for reading 
interests. Girls in treatment 
classrooms were statistically 
significantly less likely than girls 
in control classrooms to indicate 
that they preferred books with 
topics identified as stereotypically 
female (princesses, Barbies, etc). 
Treatment-group membership 
reduced the likelihood of  girls’ 
stereotypical book topic selection 
by 26%.
8. Zoology One costs about $480 
per student to produce these 
impacts. About two-thirds of  
these costs were for Zoology 
One curricular materials and 
teacher coaching in school, 
and the remaining one-third of  
costs reflect the home reading 
component. On average, three 
fewer supplemental literacy 
programs were used in Zoo 
classrooms, resulting in an average 
expenditure reduction of  $40 per 
student. 
9. Teachers believe Zoology One 
impacts learning. A majority of  
teachers who offered perceptions 
of  Zoology One’s impact on 
their students’ literacy reported 
that their students’ literacy 
improved more during Zoology 
One implementation than in past 
years.  Teachers who believed 
that their students’ reading had 
been impacted by the curriculum 
often referred to the value of  the 
independent reading time. 
10. Teachers believe Zoology One 
changed their perspectives or 
beliefs as teachers. A majority 
of  teachers interviewed discussed 
ways that their pedagogy and 
beliefs were impacted by Zoology 
One. Themes included change in 
their own perceptions of  students’ 
abilities to engage with and learn 
from the curriculum, and their 
beliefs about how literacy should 
be taught.
 
 
Next Steps
We will continue to explore longitudinal 
impacts on reading, science, learning 
behaviors, and other outcomes. 
Funding for this work has been 
approved.
Kindergarten Impacts 
On average, classrooms 
using Zoology One achieved 
the same or better results 
than control classrooms 
while using three fewer 
instructional programs. 
Zoology One Efficacy Evaluation, Summary of  Findings 
(April 2020)
Consortium for Policy Research in Education | RB 2020 - 01


https://repository.upenn.edu/cgi/viewcontent.cgi?article=1088&amp;context=cpre_policybriefs

Zoology One Efficacy Evaluation Summary of Findings (April 2020)

Abstract

Similar works

Full text

Available Versions

ScholarlyCommons@Penn

Kosmopolis