left to my own devices – occasional thoughts on higher education

In 2016, at a meeting about this new-fangled thing called TEF, I remember being told (by someone who wasn’t at the DfE but was close to those circles) that Jo Johnson, then Minister for Universities and Science, had said that he saw TEF becoming one of the enduring legacies of his political career.

I’ve no idea if this was true.  It doesn’t really matter, as I think it speaks to a broader sense at the time that TEF would be a significant, permanent and consequential change in English higher education.  Not only would TEF medals sort the sheep from the goats in terms of quality of education; they would also be the mechanism for unlocking increases in undergraduate fees based on judgments about quality. And soon we would be moving to subject-level TEF assessment to give applicants the information they needed about the quality of courses.

old TEF

Well we got the medals, as crude as they were (and sorry, I still can’t think of TEF ratings without The Beautiful South’s words coming to mind to mock them – ‘I don’t want Silver, I just want Gold;  Bronze is for the sick and the old’).  We also got a rise in fees.

The fee increase was, however, a one-off.  The Conservative’s poor showing at the 2017 general election was partly attributed to the alleged Youth Quake, leading to a determination not to repeat that fee rise.  And then the planned introduction of subject-level TEF was repeatedly postponed, before rightly being killed off in January 2021 on the back of the Independent Review.

new TEF

The future that was being shaped in 2016 is now here, in the shape New TEF.  But it’s different to what was envisaged, and as is often the case it feels a bit underwhelming.  The view that TEF has been and is worth the effort would be a minority (and probably a very small minority) view in the sector.

So why write about this now, not least given that I’ve already written about TEF earlier this year?

Well, below the surface things are stirring.  The TEF timeline said that provisional outcomes would be shared with HEPs in July and August.  Right now senior teams across universities are staring at these provisional outcomes, deciding whether they’re happy with what they’ve been told and if not whether to ‘make representations’ (i.e. appeal).

new boss, same as the old boss?

A key question for the sector is whether the New TEF currently underway, is a better process than the pre-2020 Old TEF.  Of course it’s too early be definitive on this, but it does feel that there are some improvements.

The removal of Old TEF’s mechanistic Initial Hypothesis based on a limited interpretation of the core metrics is a step forward, as is the introduction of a more nuanced rating of those core metrics through the statistical uncertainty measures evident on the TEF Dashboard.  Talking to colleagues at multiple universities this change, and the resulting increased emphasis on the additional evidence in the Provider and Student submissions, appears to have led to a feeling that outcomes are less pre-determined than was the case before.

I’m a bit more sceptical about one of the other significant changes in New TEF: the recommendation in the Pearce Review, accepted by the government and now embodied in OfS’s TEF Guidance, to clarify that TEF’s primary purpose was to promote quality enhancement.

Enhancing the educational experience is already central to what we do in universities. This is intrinsically part of our nature as academic communities, our organisational character and our professional practice.  I’m unconvinced that TEF is needed to encourage us to (in OfS’s words) ‘deliver excellence above minimum requirements’ [p.6].  As just one example, see the wealth of curriculum review work currently underway that has been highlighted on WonkHE.

Of course that leaves open the question of how effectively we all enhance our educational provision.  Does TEF provide a valid assessment of this?  Possibly.  It certainly might, through the assessment by expert peers of a combination of core metrics and additional evidence in Provider and Student submissions.  The task facing these expert assessors, however, is huge.  I also worry that there is a flaw in the process.

It’s right that providers are able to provide additional evidence to supplement the core metrics.  And there is rigour in the TEF process, through the provisions to request from providers evidence that verifies the claims in their submissions.

However, the verification process is limited.  I would be astonished if any university made any claim in their Provider Submission that was not evidence-based, and could not be verified.  But what about the danger of ‘cherry picking’ – submitting the evidence that demonstrates excellence and success, but remaining silent about internal evidence that shows a more complex or challenging picture?  The opportunities to examine this are limited (though perhaps – and only perhaps – the introduction of the independent student submission might help with this).  Might this limit the validity/usefulness of the TEF outcomes ?

That leads to one final thought.  OfS’s Guidance states that ‘TEF ratings will create this incentive [to enhance provision, quoted above] by putting a spotlight on the quality of providers’ courses, influencing providers’ reputations and informing student choice’ [p.6].  In other words, enhancement and excellence as measured by TEF will matter to us as it will affect recruitment.  This runs counter to the findings of research  from UCAS commissioned for the Independent Review, which suggested that applicants did not regard TEF ratings as an important factor in making decisions on where to study.  It remains to be seen whether New TEF changes this.  Again, I’m sceptical.

change and continuity

So there’s much to continue to think about, but one closing comment.  In 2016 TEF was the future, and we’re now living in a version of that future that’s significantly different to the one originally envisaged.  But of course change sits alongside continuity. There are strong elements of continuity between Old and New TEF. But the continuities are deeper than just that.  When I joined the sector in the early 2000s the then Vice-Chancellor at the University of York, Sir Ron Cooke, was leading two pieces of work at a national level:  one on quality enhancement, the other on published information about educational provision to meet the needs of stakeholders.  Some issues are always with us.