Dr Kristina Milanovic, Research Data Scientist, RS Assessment
Katie Blainey, Director - Assessment Product, RS Assessment
This is the second in a series of blog posts which provide the latest updates in children’s attainment from the summer 2022 term. This blog post will review children’s attainment in Reading. Analysis was conducted on data uploaded to MARK from over 130,000 pupils sitting RS Assessment’s standardised termly Reading tests (New PiRA) between May and August 2022 at approximately 1,500 state Primary schools across England [1]. For the period of summer 2019 and 2021, the same data was used as in our earlier white papers, the full details on coverage and representativeness can therefore be found in the summer 2021 white paper on: risingstars-uk.com/whitepaper.
Performance from pre-pandemic to summer 2022
The impact of school closures over the course of 2020 and 2021 on children’s reading attainment test scores was not evenly distributed. Younger children were disproportionately impacted and this can still be seen in their most recent test results from summer 2022.
To help understand how differences in mean Standardised Score correspond to time spent learning, we translated effect sizes to months’ progress using a method developed by the Education Endowment Foundation (EEF) [2]. Last summer Reception and Year 1 were still 2-3 months behind the pre-pandemic cohort whereas later school years appeared to have caught up, this can be seen in Figure 1.
Figure 1: Effect sizes and months’ progress in reading (PiRA) for all school years summer 2019-21
Figure 1 uses data from PiRA tests while Figure 2 uses data from New PiRA tests. PiRA tests have recently been retired and so while we can talk about them in the context of New PiRA, the results are not directly comparable. For a full explanation of the differences please visit risingstars-uk.com/whitepaper.
To provide increased comparability between different groups and across time periods effect sizes are used to compare attainment levels. The larger the negative change in effect size, the larger the attainment gap and effect on learning is likely to be. Conversely, the larger the positive change in effect size, the more the attainment gap has been reduced and the more progress children have made over the period being looked at.
Figure 2 shows the effect sizes and months’ progress made by children sitting New PiRA tests in summer 2022 compared to summer 2021. The effect sizes for all school years can also be seen in the bar chart in Figure 3. As can be seen in both Figures 2 and 3, the most progress has been made by children in Reception and Year 1.
Figure 2: Effect sizes and months’ progress in reading (New PiRA) for all school years summer 2021-22
Looking at Figures 1 and 2, we can conclude that while it is likely that children in Key Stage 2 have caught up the pre-pandemic cohort, the same cannot be said for children in Key Stage 1. It is likely that these children may still be behind their pre-pandemic counterparts and will need more assistance in catching up going forwards. This is also clearly illustrated by Figure 3.
Figure 3: Effect size for all school years in reading (New PiRA) for summer 2021-22
Impact of Disadvantage
As we have highlighted previously, within each year group not all children have been affected equally and a particular concern is the impact of disadvantage on children’s attainment. There is already a difference between the average attainment of children who are eligible for Pupil Premium and those who are not. With Pupil Premium children on average obtaining lower scores than their peers. In reading the disadvantage gap between children eligible for Pupil Premium [3] and their peers remains substantial. It fell in summer 2022 (blue bars) compared to summer 2021 (orange bars) for almost all years except Year 2 where is increased slightly compared to the summer 2021 difference. This can be seen below in Figure 4.
Figure 4: Difference in Mean Standardised Score in reading (New PiRA) between Pupil Premium children and their peers for summer 2021 and 2022
To look at the level of in-school deprivation, we grouped children by the percentage of children receiving Free School Meals (FSM) at their school. This can be seen in Figure 5. (Please note, Reception and Year 6 have been omitted from Figures 4 and 5 because the group sizes were too small to be representative.) The blue bars represent children at schools where in-school deprivation was low (FSM <20%); the orange bars represent a medium level of in-school deprivation (FSM 20-35%) and the grey bars indicate a high level of in-school deprivation (FSM >35%).
Figure 5 shows two interesting trends, firstly the effect size for children in Year 1 in schools with high in school deprivation is more positive than for the other two groups, indicating children have made more progress in the last school year and in summer 2022 are obtaining higher scores than summer 2021. This may be because there have been additional programs within schools across England to help these children in their first year of schooling.
The second trend is that across England (on average) children in Year 2 in the low in-school deprivation category have negative effect size, indicating that the attainment gap has widened for these children and they obtained lower scores on average in summer 2022 than the previous year’s cohort in summer 2021. This effect size is even more negative than the one for children in the high in-school deprivation category and this may be the reason why the disadvantage gap has increased for Year 2.
Figure 5: Effect size by level of school deprivation for Reading (New PiRA) for summer 2021-22
Regional Analysis
Drilling down into the regions, Figure 6 shows the effect sizes for summer 2021-22 by region for all primary school years. All regions except the West Midlands show positive effect sizes which would indicate children have performed better in the summer reading tests in 2022 than in 2021 which is an encouraging sign.
Figure 6: Effect size by region for Reading (New PiRA) for summer 2021-22
Breaking this down by school years we can see that Year 2 made the least progress compared to last summer. This is primarily driven by children in the Midlands who obtained lower scores than last year (indicated by the negative effect size). (Please note, Reception and Year 6 have been omitted from Figure 7 because the group sizes were too small to be representative.)
Figure 7: Effect size in reading (New PiRA) by regional group across school years for summer 2021-22
In our previous white paper, which considered attainment up until spring 2022, children in Key Stage 2 in the North showed the most progress between spring 2021-22. Figure 7 also shows this for the most recent (summer 2022) term. Although attainment levels had decreased the most in the North due to school closures caused by the COVID-19 pandemic, over the course of the last school year, this is the region where children have bounced back the fastest, particularly in Key Stage 2.
Topic analysis of the primary school years did not reveal any specific trends. In particular for Year 2 where children’s scores were lower for all topics in summer 2022 compared to summer 2021. Further support will therefore be needed across all topics as these children progress through primary school.
Summary
Overall, it is encouraging to see that most Primary school years appear to be back on track and attaining levels close to or on par with pre-pandemic levels. However, Year 2 (this year’s Year 3) is a special case. Children in this year had their Early Years and Key Stage 1 education most strongly impacted by national lockdowns and we can see that the impact of this is still lingering in their performance. It will be important going forwards to provide additional assistance to these children, especially as they move into Key Stage 2.
We are always interested in receiving feedback on our work to make sure it stays relevant. If you would like to hear first about updates on the research, or if you are a senior leader in a school and you would like to work with us to ensure our analysis and outputs are providing insights that support you, please do get in touch with us at assessment@hodder.co.uk quoting Nuffield stating your interest in the project.
The project has been funded by the Nuffield Foundation, but the views expressed are those of the authors and not necessarily the Foundation. Visit www.nuffieldfoundation.org
[1] We have analysed only results from fully completed tests, with non-zero scores sat by a pupil within the correct age range. Tests sat at the wrong time of year have been omitted. An analysis of the coverage of types of schools included in all cohorts was broadly similar in that all regions and major school types were included. Additionally, to maintain representativeness, the minimum group size for analysis was 1000 pupils and results from groups with less than this number of pupils have been omitted.
[2] Effect sizes were calculated by dividing the difference in Standardised Score points between prior and current cohorts by the standard deviation of the prior cohort. These were converted to months progress using the EEF table, see: Education Endowment Foundation (EEF), (September 2021), “Teaching and Learning: Early Years Toolkit Guide”, EEF, London, pp. 6.
[3] Only pupils attending schools with overall Pupil Premium percentages in MARK that were broadly consistent with the proportion reported publicly for that school by the Department for Education were included in this analysis. Pupils with unknown Pupil Premium status were also excluded.
Tags
assessment,
attainment,
nuffield,
reading,
research