The decline in achievement rates since apprenticeship standards were introduced, driven by nearly half of all apprentices dropping out, is well documented. And, in March the ESFA admitted that they had incorrectly calculated achievement rates for 2019/20, and this lowered them further than previously reported (see here).

But for the 2021/22 achievement rates, due for publication next March, we could be facing a new problem that I am concerned the ESFA hasn’t considered.

It’s complicated, but let me explain (and hope that next March I don’t need to say “I told you so”).

In 2015, there was something of a scandal when the ESFA concluded many apprenticeship providers were inflating their achievement rates by leaving the learner record in the Individualised Learner Record (ILR) as either on a break in learning or simply continuing, rather than recording them as having withdrawn. As a result, these apprentices were not included as non-retained, something the DfE statisticians concluded was boosting some provider achievement rates by more than 20 percentage points!

To combat this abuse, the ESFA check multiple ILRs from different years, and where they find a break in learning or continuing record is well beyond it’s planned end date, they count them as withdrawn and thus non-retained for the purposes of achievement rate reporting.

For the 2021/22 achievement rate calculation, the ESFA rely on the 4th monthly ILR data return in 2022/23 from providers (known as R04) in order to check on the current status of continuing apprentices, or those on a break. The deadline for R04 is today (6 December).

The ESFA describe the importance of 2022/23 ILR R04 in their achievement rate business rules document (click here). For example, para 40 says “We treat learning aims as withdrawals for the overall QAR methodology where they have a ‘Completion status’ of 6 (‘Learner has temporarily withdrawn from the aimdue to an agreed break in learning’) and where the planned break recorded in the R14 ILR return for 2020 to 2021 has no corresponding restart record in the R04 ILR return of 2022 to 2023.”

Also, the ILR data collection timetable published by the ESFA (click here) makes it very clear that for all funding streams (not just apprenticeships) the ILR R04 is used to support their achievement rate calculation.

This automated checking of ILRs completely relies on providers making sure their apprentice records match-up in each ILR, else the check fails and the starter record is assumed to have withdrawn.

In order for providers to ensure their records match-up, many have become heavily reliant on using some software that for more than a decade the ESFA has paid KPMG to produce and make freely available to providers and auditors. Known as the Provider Data Self-Assessment Toolkit (click here). 

But, PDSAT has not been made available to provider yet, for this academic year. This is highly unusual, and relates to delays in rolling out a new online version of the PDSAT software, as announced by the ESFA last month (click here).

Somewhat surprisingly, the only plan B from the ESFA has been to release a few post-16 monitoring (FRM) reports a month early, to support with the data quality for R04. BUT…none of these FRM reports relate to checking apprenticeship data, such as breaks in learning. And the relevant Apprenticeship Accountability Framework reports are also still unavailable.

It also seems like the ESFA have forgotten about the importance of the R04 when it comes to their achievement rate calculation, giving it no mention at all in some recent guidance to the sector (click here).

My concern, is that in the absence of PDSAT combined with an ESFA that appears to have forgotten why R04 is important for their achievement rate calculation, the sector could be in for a shock once the draft achievement rates are shared with them in January.

If I am right, the ESFA will need to decide whether this constitutes a significant enough problem to try an correct. Typically, they would only correct a calculation error, not errors in the ILR.

Either way, the lack of PDSAT to help prepare for R04 is something the ESFA should not have allowed to happen and I hope I’m wrong when it comes to the implications to 2021/22 achievement rates.

Last week I delivered a webinar to help support providers prepare for R04 – without PDSAT – and the slides can be downloaded from here.