Investing in after-school and summer programs leads to amazing things. At Afterschool Labs, we are partnering with school districts like yours to accelerate student recovery and achievement. Investing in our students boosts reading skills, and that is amazing.
AFTER-SCHOOL PROGRAMS
Introduction
This white paper presents a study on the impact of after-school extended learning programs on student reading competency in two Alabama school districts: Fort Payne City Schools and Walker County Schools. The study analyzed data from 1,815 and 928 students in these two school districts for the 2021-2022 school year. The results showed that students who participated in the afterschool program showed higher growth rates in their reading scores compared to non-participating
peers. The study concluded that the Afterschool Labs’ Extended Learning Program positively impacted student reading competency.
It has been a little over two years since the pandemic disruption to public education in the United States, and the data is in on the impact this disruption had on students. A recent Chalkbeat article titled The Nation’s report card: Massive drop in math scores, slide in reading linked to COVID disruption presented the below NAEP score Data over time for both fourth and eighth-grade students:
1 Barnum, Matt, (2022). Nation’s report card: Massive drop in math scores, slide in reading linked to COVID disruption.:
https://www.chalkbeat.org/2022/10/24/23417139/naep-test-scores-pandemic-school-reopening
The data points from the article highlight that while students continue to learn, they do so at rates much slower than in years prior to the pandemic. The article noted that “No state was completely spared from the pandemic’s academic wreckage. Every state in the country saw declines on at least one exam, and students in most states lost ground on multiple exams.”
Because of the pandemic’s impact, more and more schools are now turning to tutoring and extended learning programs. According to a recent Washington Post article titled Schools sink money into tutoring, but some programs fall short, schools see mixed results, particularly with online self-direct
tutoring for students. This is driven mainly by low participation rates. In-person tutoring, required byschool systems for underperforming students, is much more effective.
Data & Methods
Afterschool Labs K12, LLC (ALABS) is an educational services company providing comprehensive after-school and summer programs to accelerate student recovery and achievement. ALABS programs includes proprietary STREAM curriculum, customizable programming, and turn-key administration. ALABS commissioned a third party to review the after-school extended learning program to determine the impact of the program on participating students regarding reading competency.
Two Alabama school systems participated in the study: Fort Payne City Schools and Walker County Schools. Both school systems provided data for students for the 2021-2022 School Year, and the data provided was for both participating and non-participating students.
Fort Payne City Schools provided i-Ready Diagnostic Comprehensive Reading Score and Percentile Rank data taken at the start and end of the school year for grades kindergarten through fifth grade.
The i-Ready Diagnostic is an adaptive assessment that matches each question’s difficulty to each student’s ability. As the student answers questions correctly, the test gets more complicated. As the student answers questions incorrectly, the test gets easier. The assessment uses a vertical scale that compares growth within and across years. Students receive a scale score and percentile rank each time they take the diagnostic assessment i-Ready Diagnostic reading score, and percentile rank data was provided for 1,815 students, of which 158 attend the Afterschool Labs, Afterschool Program.
2 George, Donna St., (2023). Schools sink money into tutoring, but some programs fall short: https://www.washingtonpost.com/education/2023/01/14/highdosage-tutoring-opt-in-pandemic-learning-loss/
Walker County Schools provided Renaissance Star Reading Score and Percentile Rank data taken at the start and end of the school year for grades kindergarten through sixth. The Renaissance Star Reading assessment is designed to comprehensively test a child’s skills in reading. The test is adaptive, adjusting to each answer that the student provides. The test is designed to provide insights
into growth, high achievement, and areas in which students might struggle. Renaissance Star Reading Scores and Percentile Rank data was provided for 928 students, of which 208 attend the Afterschool Labs, Afterschool Program.
Each program was reviewed separately due to the differences in testing instruments.
All student data was sorted to remove students with less than 80% attendance in normal school and to remove students that did not have both scores due to leaving the school system. Students participating in Afterschool Extended Learning were sorted to remove students that did not have a minimum program attendance of 60%. This reduced Fort Payne City Schools Students to 1,442 nonparticipating and 118 participating. This reduced Walker County Schools Students to 417 nonparticipating and 128 participating. Test score outcomes were averaged for testing at the first and the end of the year, and an average growth rate was calculated. Non-participating student outcomes were compared to that of the participating students.
Results
In Fort Payne City Schools, on average the i-Ready reading scores of non-participating kindergarten through fifth grade students increased 8.7% from 449.33 to 488.84. Students that participated in the program showed an average score increase of 10.6% from 426.03 to 471.20.
In Walker County Schools, on average the Star reading scores of non-participating kindergarten through sixth grade students increased 33.15% from 48.98 to 65.22. Students that participated in the program showed an average score increase of 53.84% from 25.91 to 39.87.
In summary, students that participated in Afterschool Labs’ Afterschool Extended Learning Program overall saw higher growth rates than non-participating peers.
“ALABS has been a tremendous tool for Fort Payne City Schools to address our learning loss that occurred during the pandemic. For the first time our District has earned an overall grade of an A on the state report card. Additionally, our schools using ALABS increased their achievement data for every grade.“
Paula Muskett, Assistant Superintendent
Forth Payne City Schools
“The data reveal that students who participated in the ALABS program have shown academic growth and higher attendance rates. Our teachers and students are excited about the program. Plus, ALABS eliminates internal hassles that are associated with operating extended learning programs.“
Dr. Tanya Guin, Director of Elementary
Education, Walker County Schools
Summer Programs
Introductions
The study commissioned by Afterschool Labs showed that participating students in their Summer Extended Learning Program experienced less summer learning loss than non-participating students. Students in kindergarten to 2nd grade showed an average 1.5% increase in reading scores and a 15.39% increase in percentile rank, with 77.1% of students showing improvement. Third-grade students had an average 11.5% increase in reading scores and a 24.93% increase in percentile rank, with 80% showing improvement. The results indicate that the Summer Extended Learning Program effectively reduced summer learning loss and promoted reading proficiency in students.
Most educators have heard the term “summer slide” or “summer learning loss.” In 1996, researchers published the first comprehensive study on the effects of summer vacation on achievement test scores. The study showed that kids lose significant knowledge in reading and math over summer break, which tends to have a snowball effect as they experience subsequent skill loss each year. This has been collaborated by more recent studies. One published in 2020 by the American Educational Research Journal, entitled “School’s out: The role of summers in understanding achievement disparities,” found that the average student lost 17–34% of the prior year’s learning gains during summer break as well as that students who lose ground in one summer are more likely also to lose ground in subsequent summers.
Afterschool Labs is an educational services company providing comprehensive afterschool and summer programs to enhance student achievement and foster soft skills development. The company solicited a third party to statistically review their summer extended learning program to determine the impact on participating students regarding the core competency of reading.
3Cooper, H., Nye, B., Charlton, K., Lindsay, J., & Greathouse, S. (1996). The Effects of Summer Vacation on Achievement Test Scores: A Narrative and Meta-Analytic Review. Review of Educational Research, 66(3), 227–268. https://doi.org/10.3102/00346543066003227
4Atteberry, Allison, and Andrew McEachin. (2019). American Educational Research Journal. School’s Out: The Role of Summers in Understanding Achievement Disparities. (EdWorkingPaper: 19-82).
Data & Method
To understand the impact of Summer Labs, data was provided by Fort Payne City Schools for students in grades kindergarten through third for the 2021-2022 School Year Summer Program. This data was provided for a total of 127 students. For kindergarten through second-grade students, the school system provided i-Ready Diagnostic Comprehensive Reading Score data for participating students before and after the summer. The i-Ready Diagnostic is an adaptive assessment that matches each question’s difficulty to each student’s ability. As the student answers questions correctly, the test gets more complicated. As students answer questions incorrectly, the test gets easier. The assessment uses a vertical scale that compares growth within and across years. Students receive a scale score and percentile rank each time they take the diagnostic assessment. This data was provided for seventy-four students in grades kindergarten through second.
For third-grade students, the school system provided ACAP assessment Reading Scores before and after the summer for participating students that did not meet the required benchmarks. ACAP stands for the Alabama Comprehensive Assessment Program. It is an online assessment designed to provide student progress toward mastery of the Alabama Course of Study Standards. Students receive a scale score and percentile rank each time they take the assessment. ACAP reading score and percentile rank data were provided for fifty-three third-grade students.
Student data was sorted to remove students that did not have both scores due to leaving the school system and students that had a minimum of 87.5% attendance in the Summer Extended Learning program (a minimum of fourteen days of the sixteen-day program). This reduced the kindergarten through second grade students to forty-eight and resulted in third-grade students being reduced to twenty-five.
For kindergarten through second-grade students, the study compared each student’s reading score and percentile rank before and after summer. On average, the kindergarten through second-grade students’ scores increased 1.5%, and those students’ Percentile Rank increased 15.39%. Most students (77.1%) saw score and percentile rank increase. The average scores of these students increased by 3.3% and saw a percentile rank increase average of 44.72%. For the eleven students that did not show increased scores, the average decrease was 3.7%, significantly less of a decrease than those noted in prior studies.
For third-grade students, the study compared each student’s reading score and percentile rank before and after summer. On average, third-grade students’ scores increased 11.5%, and on average, those students also saw a percentile rank increase of 24.93%. Most students (80.0%) saw score and percentile rank increase. These students’ average scores increased by 12.6% and saw a percentile rank increase average of 67.2%. For the five students that did not show increased scores, the average decrease was 2.4%*, significantly less of a decrease than those noted in prior studies. 2 Evidence-Based Afterschool & Summer Programs Impact on Student Reading Competency 12 Data & Methods There was one outlier student within the five, this student was removed from the average. The outlying student had a 37.8% decrease.
*This was significantly higher than any other decrease across all students with a minimum attendance of fourteen days.
Results
In summary, students that participated in Afterschool labs’s Summer Extended Learning Program overall saw diminished summer learning loss with the majority showing 1.5 to 3.3% Reading score growth and 15.39 to 44.72% Reading Percentile Rank Improvement.
