Responding to Criticisms of the CASLO Approach (Report B): Conclusion and references
Published 18 November 2024
Applies to England
© Crown copyright 2024
This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at https://www.gov.uk/government/publications/responding-to-criticisms-of-the-caslo-approach-report-b/responding-to-criticisms-of-the-caslo-approach-report-b-conclusion-and-references
Conclusion
The AOs described complex constellations of mitigations and protective factors that are embedded in CASLO qualifications to ensure their quality and value. We observed instances of creative solutions to specific problems, some variability in the profile and plausibility of specific mitigations, some tensions between certain mitigations, and some differences in perceived seriousness of individual problems. The latter appeared to depend, at least in part, on different contexts and qualification uses, as well as on the perceived extent of responsibility of AOs in the educational ecosystem beyond solely assessment and certification.
While recognising many of the potential risks associated with the CASLO approach, and the resources required for its implementation, AOs were confident in the validity and value of their qualifications. They often invoked stakeholder approval and recognition for their CASLO qualifications, challenging the suggestion in the academic literature that the CASLO approach might be inherently sub‑optimal.
We have qualifications with external assessment in them. It’s a very different learning experience for students. I think, ultimately, our strong feeling is that CASLO qualifications, or our model of CASLO qualifications, work really well for our students. If someone introduced us to another model that we found to work better, we would explore that as well, but for us, at the moment, with our ethos and the learning experience that we’re trying to offer students in relation to vocational experience, this model works really well. […] creative arts learning and education is a community of its own, […] and they tell us that our qualifications serve their needs. So that’s really what informs us. […]. And obviously it’s difficult, but we do it and we believe we do it well.
Creative_L3
[…] I should say we have recognition by 5 or 6 different professional bodies for the qualifications, so they think that we have done it in a way that represents their needs.
Construction_L5
[…] I think it would be a sad day if we ever lost it [the CASLO approach] because I do think there are individuals out there that [have] been let down by the traditional system of education, so having a different approach to enable them to get the recognition for the work that they do, and also a route into other work that they want to do, I think we do need an alternative […].
End of life care_L2
CASLO qualifications can be absolutely fine. Yeah, they’ve got weaknesses […] but, equally, if they’re delivered by good teachers who know what they’re doing, with a good IQA and good support from the EQA, you can be pretty confident the learner’s going to get a good experience and get through it.
Adult care_L3
It is, perhaps, of note that the most recognised potential assessment problem involved inaccurate judgements, while the most recognised teaching and learning problem involved incoherent teaching programmes. Both of these problems relate to the defining properties of the CASLO approach and the ability of the transparency mechanism to support the functioning of CASLO qualifications by ensuring that teaching, learning and assessment domains (or curriculum, pedagogy and assessment) align appropriately and are interpreted consistently.
According to what many AOs told us, LO- and AC‑based specifications on their own are unlikely to ensure the transparency of this alignment entirely, and need further elaboration through syllabi, support and guidance, and other means. In one sense, this might be seen as an acknowledgement of the force of certain key criticisms in relation to current CASLO qualifications. Yet, in another sense, it can be seen as a reminder that the CASLO approach is adaptable, to achieve optimal functioning. For instance, it was clear from various AO comments and practices that the CASLO approach is not incompatible with or antithetical to specifying quite detailed syllabi and providing support and guidance to teaching and learning in addition to LOs.
The insights provided by our respondents seem to suggest that high quality CASLO qualifications should strive for an appropriate balance between ensuring educational value, reliability, validity, and manageability, bearing in mind the specific purposes these qualifications are intended to serve. Furthermore, given the extent of interdependence between teaching, learning and assessment in CASLO qualifications, it would stand to reason that ensuring a high level of alignment between the actors in these different areas would also help to ensure optimal functioning of these qualifications. Effective regulation of CASLO qualifications is likely to require a nuanced understanding of the abovementioned tensions, balances and domains of responsibility in different contexts. The current study has hopefully moved that understanding forward.
References
Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
De Ville (1986). Review of Vocational Qualifications in England and Wales. London: HMSO.
Gale, N.K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13(117), [not paginated].
Jessup, G. (1991). Outcomes: NVQs and the Emerging Model of Education and Training. London: The Falmer Press.
Newton, P.E. (2018). Grading Vocational & Technical Qualifications: Recent policies and current practices. Ofqual/18/6441/3. Coventry: Office of Qualifications and Examinations Regulation.
Newton, P.E. & Lockyer, C. (2022). How ‘CASLO’ Qualifications Work. Ofqual/22/6895. Coventry: Office of Qualifications and Examinations Regulation.
Ofqual (2008). Regulatory Arrangements for the Qualifications and Credit Framework. Ofqual/08/3726. London: Office of the Qualifications and Examinations Regulator.
Ritchie, J. & Lewis, J. (2003). Qualitative Research Practice: A guide for social science students and researchers. London: Sage.