left to my own devices – occasional thoughts on higher education

Image by Freepik

It’s difficult to accuse OfS of rushing to judgment, in some areas at least.

how soon is now?

In May 2022 OfS announced investigations of business and management courses at eight providers.  The University of Bolton was visited by an assessment team five times, ending in January 2023; London South Bank University three times, the final occasion in March 2023.  These are first two of the eight assessments to have their reports published.  Yesterday (by coincidence the day before the report of the House of Lords Industry and Regulators Committee’s inquiry into OfS).  And the press release is clear that although the reports have been published, OfS has yet to make any regulatory decisions based on them.

(Let’s not dwell on the contrast of these quality assessment timescales with Daily Mail article in April 2021 about marking practices in universities in relation to spelling, punctuation and grammar; the setting up in June that year of an OfS review of this issue; and the publication in October 2021 of the report of this review).

what the world is waiting for?

It’s good to see published some substantive outcomes from OfS quality assessment investigations, given OfS’s much commented on failure to date to do so before now.  As Jim Dickinson has written, it starts to give an indication of how OfS sees its new approach to quality and standards working out on the ground (though we need to take care of course not to over-interpret a very small dataset).

The reports and the process they capture are both recognisably in the tradition of QAA subject/internal periodic review, but also different given the emphases in the OfS’s quality and standards conditions compared to previous national quality requirements.  And this mix of continuity and change can also be seen in those areas identified as problematic in the Bolton report.

Issues such as personal tutoring and feedback on assessed work (identified in the Bolton report) echo common themes in external QA reports going back to the 1990s.  The willingness (again in the Bolton report) to flag concerns about resourcing levels and their impact on quality is, though, something that rarely featured in previous external QA processes.  QAA processes, by design and implementation, generally steered clear of such things. A decade ago a HEPI report by Bekhradnia and Brown identified this failure to consider ‘the way in which decisions on resource allocation and management interact with academic decisions and judgments’ [pp.5-6] as a significant gap in external QA. So it’s interesting to see resourcing feature in one of the reports, particularly as the link between resourcing and quality is an issue that is going up the agenda in an era when inflation is not so much eroding, more like washing away, the unit of resource for undergraduate education.

More than what these two reports say, though, what has struck me most has been some of the silences in the reports.  The things left unsaid and the voices not heard are as interesting as what’s actually in the reports.

your silent face

Jim Dickinson’s piece picks up on one of the important silences – we don’t really know the basis on which business and management was selected for OfS intervention, or why the eight providers in particular were chosen.  I’m not even sure we know who made that selection.  Yes it was the OfS, but that doesn’t reveal a great deal. Was it OfS officers?  Was it the Quality Assessment Committee?  The issue is important, not least as it raises the question of whether there was any expert academic judgment involved in these choices.

The WonkHE piece also points out that individual assessment teams have been making risk-based decisions on where they focus their attention within the business and management programmes at each provider.  We know that the teams making the decisions are made up of academic experts in the subjects under review.  But what’s interesting is that the reports are silent on who those reviewers were.  Under previous regulatory regimes external QA reports always said who the reviewers were.  It seems strange that OfS has chosen not to follow suit.

This secrecy though is consistent with the OfS approach as a whole to these visits.  As far as I’m aware there has been no operational description or method handbook published for these assessments.  Little wonder then that in a blog this week for HEPI on the experience of being assessed, London South Bank’s Vice-Chancellor commented that ‘it was unreasonable not to be given a clear idea of the initial lines of enquiry at an earlier stage as it left everyone feeling this was more of a ‘fishing trip’ than a review taken against a clearly defined risk-based framework’.

This perhaps should not be too surprising given OfS’s past practice.  When QAA was still the Designated Quality Body its teams would be asked to undertake Quality and Standards Reviews (QSR) where OfS had identified potential concerns (in the jargon, for the purposes of monitoring and intervention), but teams weren’t told what concerns had led to the QSR.

And of course there’s the most notable silence of all in the quality assessment process: the exclusion of students from the review team.  Students’ interests are meant to be at the heart of OfS’s work, and by extension of this process.  Yet their role in quality assessment reviews is as the congregation responding to the prompts of the academic priesthood that is the assessment team. Students watch from the other side of the screen, while the process of making quality assessment judgments is performed by a team with no student membership.  The student voice has a defined and limited place, and where judgments are made the direct student voice has been silenced.

Perhaps none of this should surprise us – after all, the House of Lords Industry and Regulators Committee has found OfS to be ‘controlling, arbitrary and unclear’.  But there are better ways, and one place to start is to restore to students their voice in these processes.