All apprenticeship providers produce data. But how can you use it to ensure you deliver quality training and demonstrate this to Ofsted inspectors? In a recent webinar, Paul Warner, R&D Director for AELP, spoke to former Ofsted HMI Specialist Advisor Dr Chris Jones, and Bud’s Head of Customer Success, Brad Tombling:
How can you prepare data for Ofsted inspections?
Delivering high quality, effective experiences for learners is a top priority for all apprenticeship providers. But with multiple learners – not to mention trainers, employers and programmes – to manage and oversee, how can you be sure that real progress is being made? And how can you prove it and excel when it comes to your Ofsted inspection?
The answer is by being intentional about the way you collect, interpret and, most importantly, use data within your organisation to inform curriculum development and ensure learners get the support they need to succeed.
How important is data within the Education Inspection Framework?
According to Dr Chris Jones, inspectors use validated data both before and during an inspection to assess the performance of a provider. This happens at an overall provider level but they will also drill down into individual subjects and types of provision.
“What the validated data does is that it allows inspectors to come up with the kind of questions that will inform them about what they need to do during the inspection,” says Dr Jones. “It will inform them about the conversation they would want to have with the nominee during the inspection planning call.”
Although judgements aren’t made on data alone, data does provide evidence of the judging impact of the quality of education. Being able to calculate your achievement rate, pass rates and retention rate is essential.
How can you use learner Data for curriculum development?
But data should do more than that, according to Dr Jones. It should inform curriculum development too.
“Through the Education Inspection Framework the larger focus for what inspectors will look at is the curriculum,” he says. “It’s about what we are doing in terms of intent and implementation to ensure that learners and apprentices achieve their best. That they take what you are giving them, the experiences that you provide, and help them to make the progress that they should.”
By being able to quickly and easily calculate your pass rates, retention rates and achievement rates, you can use that data to look at curriculum intent and implementation, to ensure that you’re doing what you need to do in order to maintain or improve performance.
Because inspectors will want to know what you have done to change and improve the curriculum, to improve implementation, upskill your staff and so on, in order to ensure that current learners continue to make the progress that they should. If there have been issues in the past, how are you using the data you have to mitigate those issues?
This may well be especially important now thanks to the changes that have needed to be made due to COVID-19.
“One of the things that we do know from the ESFA is that there will be no institutional level qualification achievement rates in the NART (National Achievement Rate Tables),” says Dr Jones.
“So we’re not going to get that individual institution level QA. Now that then means that you need to be in a position to have sensible data conversations with inspectors. And it may therefore be important for you to be able to look at your own QA data.”
How can you assess learner progression using data?
A key element of what inspectors are looking to understand during an Ofsted inspection is the progress learners have made. And that isn’t just the simple act of passing through various tasks and assessments.
“We have to remember that in terms of EIF the definition of progress is about knowing more and remembering more, it’s about knowledge, it’s about skill, application, it’s about recall,” says Dr Jones.
“So it’s not about progress through a series of units or test points; they’re easy to demonstrate in a data system. It’s about getting to that question of how do you know that those learners know more and remember more.”
And when it comes to individual challenging targets, the data should help provide flexibility by giving a picture of what is happening to learners over time.
Dr jones adds: “An inspector would be interested in how teaching changes, how support patterns change, in order to enable learners and apprentices to achieve those individual challenging targets.”
If progress is an important part of demonstrating the quality of learning that is being provided, then it is important to assess the skills, knowledge and behaviours that a learner brings with them to a programme.
From there, training providers need to be able to collect data about how quickly or slowly they learn, how quickly they acquire and develop their knowledge, and how quickly they convert knowledge into skills and behaviours.
This is where Bud’s flexible data collection, reporting and visualisation tools provide real value for our clients.
The importance of the right system to support effective data use
Many systems collect data but, as Dr Jones explained, the data in itself is not what inspectors are interested in. They are looking to see what training providers do with that data to improve learning. This means it’s vital that an apprenticeship management system is able to produce reports and displays that allow training providers to very quickly identify issues and address them.
During the webinar, Bud’s Apprenticeship Delivery Consultant, Brad Tomlin, demonstrated a couple of example learner journeys created with one of our visualisation tools, the Skill Scan report. This gives an overall score based on learning, skills and behaviours and can be segmented by trainer, employer, program or various other filters.
The first showed a learner who, at the beginning of their learning journey, scored 6 out of a possible 10 – where 8 suggests a readiness to pass their endpoint assessment. In this case, the training provider might consider whether the learner should have been on a more challenging programme from the start.
The second showed a group of learners with four months to go until the end of their programme. The data highlighted that they were underperforming, scoring just four out of 10. This knowledge would allow the training provider to put extra support in place for those individuals in order to help them get to the point they needed to reach.
This kind of reporting and insight is made possible because Bud builds reviews into the programme – they can’t be skipped. As a result, training providers always have a rich pool of data to draw from in order to ensure they are delivering the appropriate learning to each apprentice.
Predicted achievements, attainment data and fixed-time terminal examinations
In learning situations where fixed-time terminal examinations comprise the entire assessment of the course inspectors won’t look at internal progress and attainment data. In these circumstances, data is often used to create “predicted grades” but these aren’t always accurate when compared to grades attained. And with COVID-19 having interrupted studies this year, predictions may well be even less helpful.
So can apprenticeship providers ignore progress and attainment data? Not at all. Firstly, this data will inform gateway decisions. More importantly though, as Dr Jones stressed, it should inform the decisions you make about curriculum development.
“What are you doing with what you know and how does that impact upon your current learners?” he says. “You may well, as providers, generate, analyze and interpret performance data. But always inspectors will be looking for the curriculum response – your actions as teachers and leaders as a result of that data.”
At Bud curriculum intent is very important to us, to the point that we have embedded it within our workflow.
“To enroll a learner onto a program we make sure a program of learning is planned and set out, showcasing the clear structure as to how you intend to deliver a successful program from day one to endpoint assessment, with that focus really being on high quality teaching and learning from the outset,” says Brad.
“By baking this into our workflow, progress can be measured and monitored throughout the
entire program and clear plans are always at hand when you’re asked to showcase your curriculum intent, whether that be at audit or inspection.”
How can you share data with teachers, learners and employers?
Data is important to all stakeholders within an apprenticeship program setting. First the teachers, who should be using it to inform what they do, their curriculum implementation. Learners should have the opportunity to understand the progress they’re making against their personal and challenging targets.
And finally managers should be able to take all the data from and about their teachers and learners in order to make effective decisions about curriculum intent, implementation and impact. Data also allows managers to tailor staff development and timetabling in order to ensure that learners are taught by the most effective, knowledgeable and skilful teachers.
Bud caters to multiple stakeholders by bringing all user requirements and data together into one system. Using a trainer dashboard, for example, a trainer can instantly see what they need to do as soon as they log in, whether that’s responding to a message, seeing who’s made a submission or checking who has been inactive or has overdue tasks.
Equally an operations manager might use their dashboard to manage trainers and understand how they’re managing their caseloads. This gives them the opportunity to identify who is delivering high quality learning and where any potential risks might be.
And finally an employer might log onto their portal where, at the click of a button, they can track the progress of an applicant or sign any necessary documents.
With the right system in place, you will not just be able to track and access data but use it to gain genuine insights that lead to effective actions. In this way you can ensure you deliver consistently high quality learning experiences for all learners – and successfully demonstrate that fact when it comes to your Ofsted inspection.