Dr Kristina Milanovic, Research Data Scientist, RS Assessment
Katie Blainey, Director - Assessment Product, RS Assessment
This is the last in a series of blog posts that provide the latest updates in children’s attainment from the summer 2022 term. This blog post will review children’s attainment in Maths. Analysis was conducted on data uploaded to MARK from over 120,000 pupils sitting RS Assessment’s standardised termly Maths tests (New PUMA) between May and August 2022 at approximately 1,500 state Primary schools across England [1]. For the period of summer 2017 to 2021, the same analysis criteria was used as in our earlier white papers, the full details on coverage and representativeness can therefore be found in the summer 2021 white paper on: risingstars-uk.com/whitepaper.
Progress in the last school year
Focusing on progress in the last school year and looking at only the New PUMA tests, Figure 1 shows the effect size and months’ progress for children in all Primary school years. To help understand how differences in mean Standardised Score correspond to time spent learning, we translated effect sizes to months’ progress using a method developed by the Education Endowment Foundation (EEF) [2]. It shows how children in Key Stage 1 have made the largest improvement this summer compared to last summer, making on average 1.5 months progress.
Figure 1: Effect sizes and months’ progress in maths (New PUMA) for all school years summer 2021-22
Year 6 in particular has made an improvement in the last year and this can be clearly seen in Figure 2, which shows the effect sizes for each school year. The larger the negative change in effect size, the larger the attainment gap and effect on learning is likely to be. Conversely, the larger the positive change in effect size, the more the attainment gap has been reduced and the more progress children have made over the period being looked at.
Figure 2: Effect size for all school years in maths (New PUMA) for summer 2021-22
The ‘old’ PUMA tests for maths used in previous white papers were updated and revised to adapt to changes in teaching that had occurred since their initial standardisation and publication. The termly papers for the New PUMA tests were standardised on a nationally representative sample and were used in schools in summer 2021. Before this time, only ‘old’ PUMA were available for analysis of pupils’ attainment. From autumn 2022 only New PUMA is available for testing in schools.
We can now see that children sitting New PUMA tests in summer 2022 are on average 1 month ahead of the children sitting tests in summer 2021. Our previous white papers have shown that children sitting PUMA tests in summer 2021 were on average 2.7 months behind their pre-pandemic counterparts. Children in Reception and Key Stage 1 had a larger drop in attainment than children in Key Stage 2. The effect sizes showing the changes in attainment from prepandemic (summer 2019) to summer 2021 for children sitting PUMA tests can be seen in Figure 3.
Figure 3: Effect size for all school years in maths (PUMA) for summer 2019-21
Although these tests (New PUMA and PUMA) are not directly comparable our analysis does suggest that on average children are likely still behind pre-pandemic attainment levels.
Impact of disadvantage
As we have seen in other subjects, within each year group not all children have been affected equally by pandemic school closures and the impact of disadvantage on children’s attainment remains a concern. Pupil Premium children on average obtain lower scores than their peers, so there is already a difference between the average attainment of children who are eligible for Pupil Premium and those who are not. In maths the disadvantage gap between children eligible for Pupil Premium [3] and their peers remains substantial and is roughly the same across all school years. Encouragingly it has reduced in summer 2022 (blue bars) compared to summer 2021 (orange bars) for all years. This can be seen below in Figure 4. (Please note, Reception and Year 6 have been omitted from Figure 4 because the group sizes were too small to be representative.)
Figure 4: Difference in Mean Standardised Score in maths (New PUMA) between Pupil Premium children and their peers for summer 2021 and 2022
Regional differences
Looking at the regional breakdown, Figure 5 shows the effect sizes for summer 2021-22 for all primary school years. Blue bars are regions in the North, orange the Midlands and grey the South. In the last year, children in all regions apart from the West Midlands have improved their maths scores. Children in the North East and South East have shown the most improvement overall in maths.
Figure 5: Effect size by region for maths (New PUMA) for summer 2021-22
Delving into the regional progress by school year, Figure 6 shows the effect sizes by school year for the three regional groups as defined above. Figure 6 shows how children in the North have made progress in all school years, with particularly strong progress in Year 1. Year 1 has also shown the most improvement in the South. Overall children in the Midlands have not shown improvement and this is driven by lower scores in Year 2 and 5 this summer compared to last summer. (Please note, Reception and Year 6 have been omitted from Figure 4 because the group sizes were too small to be representative.)
Figure 6: Effect size in maths (New PUMA) by regional group across school years for summer 2021-22
Summary
Overall children have shown improvement in maths in the last school year. The disadvantage gap has also reduced across Primary school. This is encouraging news. Provided children continue to improve as they did this year, with perhaps some additional support for some groups, it appears children are on track to get back to pre-pandemic attainment levels in the future.
One group who could benefit from additional support would be children in all school years in the Midlands. Another group is children who were in Year 4 and 5 last school year. These children showed the least improvement since the summer 2021 and since they are this year’s Year 5 and 6, more support will be needed for upper Key Stage 2. This will be particularly important as they start to prepare for National tests later this year.
We are always interested in receiving feedback on our work to make sure it stays relevant. If you would like to hear first about updates on the research, or if you are a senior leader in a school and you would like to work with us to ensure our analysis and outputs are providing insights that support you, please do get in touch with us at assessment@hodder.co.uk quoting Nuffield stating your interest in the project.
The project has been funded by the Nuffield Foundation, but the views expressed are those of the authors and not necessarily the Foundation. Visit www.nuffieldfoundation.org
[1] We have analysed only results from fully completed tests, with non-zero scores sat by a pupil within the correct age range. Tests sat at the wrong time of year have been omitted. An analysis of the coverage of types of schools included in all cohorts was broadly similar in that all regions and major school types were included. Additionally, to maintain representativeness, the minimum group size for analysis was 1000 pupils and results from groups with less than this number of pupils have been omitted.
[2] Effect sizes were calculated by dividing the difference in Standardised Score points between prior and current cohorts by the standard deviation of the prior cohort. These were converted to months progress using the EEF table, see: Education Endowment Foundation (EEF), (September 2021), “Teaching and Learning: Early Years Toolkit Guide”, EEF, London, pp. 6.
[3] Only pupils attending schools with overall Pupil Premium percentages in MARK that were broadly consistent with the proportion reported publicly for that school by the Department for Education were included in this analysis. Pupils with unknown Pupil Premium status were also excluded.
Tags
assessment,
attainment,
maths,
nuffield,
pira,
research