left to my own devices – occasional thoughts on higher education

Image AI generated, using Google Gemini

Recently I’ve written, twice and thereby at length, on Sir David Watson’s 2006 article Who killed what in the quality wars. Most recently I sought, in a piece intended for the Quality Strategy Network (QSN) Annual Conference, to pull out some of the key themes from Watson’s piece and consider their ongoing relevance.  And unsurprisingly given Watson’s acuity as an analyst of UK higher education, the salience of Who killed what persists.

However, even in that longer piece I couldn’t cover everything that I perhaps wished to, and there was one particular aspect where I wanted to make further comment but didn’t have the time/space to do so.

Leading to this post.

audit and assessment (again)

One of the key themes in my last post was the fundamental and ongoing tension between two rival approaches: audit, and assessment/review.  The high point of the latter was, of course, the programme of Teaching Quality Assessment (TQA), and subsequently Subject Review, undertaken between the early 1990s and 2001.

I joined the sector too late to have any direct involvement in this process, but it continued to cast a shadow over the sector into the early 2000s.  When I attended my first QSN Conference in 2006 attendees were still exchanging war stories about their TQA/Subject Review experiences, proudly displaying their medals from past campaigns.

One of the ongoing debates has been how far TQA/Subject Review was a mis-step by UK higher education.  For some it’s an unqualified yes; huge amounts of bureaucracy and wasted resource to no effect.  For others, a necessary evil to defend the sector from those who sought to argue that in an era of higher education expansion more must mean worse.

1066 and all that

To his credit, Watson both declares his interest (as chair of the HEFCE committee that oversaw TQA, before it moved to QAA in 1997) and addresses the issue directly [p.4]:

for all of its flaws, teaching quality assessment (TQA) produced over an eight-year period a comprehensive, peer-reviewed, account of the state of teaching and learning in UK higher education (including medicine). It is probably the most compelling empirical evidence of the controlled reputational range of which the UK sector used to be proud.  This Domesday Book inheritance is significantly under-valued by Brown. For all of its flaws, I know of no other record in the world which could match it … the record told us some interesting things about the post-binary inheritance, with ‘traditional’ universities out-scoring the former public sector in areas which reflect a more generous infrastructural inheritance (like ‘learning resources’), while the picture is reversed on ‘quality management’. It will be easier to see the Centres for Excellence in Teaching and Learning (CETL) and Higher Education Academy returning to the full set of HEFCE subject overviews (as did most of the Learning and Teaching Support Networks) than to the two editions of HEQC’s Learning from audit.

This is a point at which I think Watson’s analysis is less convincing.

learning from the past?

I think there’s a good basis for the argument (as Mike Ratcliffe framed it in a comment about my original post) that ‘the 1990s round of TQA/Subject Review was necessary to head-off the ‘more means worse’ crowd, to show that the quality of teaching was actually pretty good’.  The idea that TQA/Subject Review provided a ‘Domesday Book inheritance’, however, feels much less plausible.

All historical sources have to be weighed for their value, and what they can therefore tell us.  The contributions to and outputs of TQA/Subject Review aren’t really an ‘account of the state of teaching and learning in UK higher education’.  Instead they show us how universities chose to present and frame their education in order to obtain strong results, within a set of (dare I say arcane) bureaucratic rules and processes (using bureaucratic in a descriptive not pejorative sense).  There may be some nuggets in there for those interested in the history of education in universities, but I suspect anyone seeking them will need to do a lot of digging and panning and probably not to much profit.

The claims about the future use of the outputs of TQA/Subject Review are also not particularly convincing.  Attempts were made to make use of these – e.g. the Subject Overview Reports produced by QAA.  Unfortunately, and I realise that this is a very personal perspective, although I worked in the area of quality assurance for a number of universities in the 2000s, I never encountered a single instance of a subject area drawing on the learning from either their own Subject Review (other than to tick the box of addressing the report recommendations); or drawing on practice at other universities identified by Subject Review.  And having worked at universities that participated in three CETLs, never saw their work being informed by Subject Review.

It is of course unsurprising that some aspects of a piece written 20 years ago perhaps hold up less well than others.  (I’m just glad that my own meanderings on this blog will not be around in 2045 for someone to dissect).

However, even in this respect there may perhaps be a different type of currency and relevance of Watson’s analysis for higher education in the 2020s.

lessons for the future?

Since TEF was inflicted on the sector a decade ago, there has been a small but growing cottage industry of analyses and syntheses seeking to look at a TEF submissions and outcomes in order to draw out such perennials as ‘good practice’ and ‘best practice’.

Many, but not all, are sound pieces of work in their own right.  But how far do they really provide any value for universities considering the ongoing development of their educational approaches and portfolios? 

Or is this the sector on autopilot, seeking to make a silk purse out of the pig’s ear that is TEF simply because the process and its collateral (TEF narratives; student submission; outcome reports) are there?  Kidding ourselves that it’s possible to derive some overarching value from a process that, despite OfS’s determination to double down on it, is fundamentally flawed and of no real value or use?

Perhaps the lesson we need to take from the lack of ongoing, enhancement value of TQA/Subject Review is that as a sector we’re on a hiding to nothing trying to find such value in the inputs and outputs of an externally imposed regulatory regime.  And stop pretending that the pill of having to continue to do TEF can be sweetened in that way.