left to my own devices – occasional thoughts on higher education

Obviously there was only one story last week for those interested in higher education, and its regulation.  Actually, there’s probably only going to that one story for a few weeks to come.  And in many ways rightly so, given the comprehensive victory for Sussex and the devastating implications for OfS of both the outcome and, perhaps even more so, what the detailed judgment reveals about its culture and operation.

There was, though, another regulatory story last week, less fanfared but also important in many ways.

regulatory action

The QAA Scotland report National Review of Awarding Arrangements: Guide for Institutions was the latest publication resulting from the investigation and response to the tragic circumstances where the misapplication of assessment and award regulations at Glasgow University contributed to a student taking their own life.  It sets out the progress to date of the Review, and the planned next steps (required by the Scottish Funding Council (SFC), and taken forward by QAA) across the Scottish sector following on from the specific investigation that was undertaken into Glasgow.

A key element of the National Review to date has been the development of a methodology for ‘deep-dive reviews’ on assessment and awarding regulation, policy and practice in the country’s universities. This has led to the development of a number of focused lines of enquiry [para. 12 in the report]; essentially a series of challenging questions for universities about assessment and awarding practices. Phase 3 of the National Review will involve a sample of universities, identified using criteria (set out in para. 13 of the report), that will now be subject to these ‘deep-dive reviews’.

QAA is at pains to state that selection for a deep-dive review ‘does not mean that a concern has been identified’.  That may be some relief to those universities selected, many of whom are perhaps thinking that the awarding mistakes that happened at Glasgow couldn’t happen at their university.

Which reminded me of something.

the account

In a number of my roles across various universities, I’ve been involved in academic appeals: sometimes as a processor of appeals; other times as an adviser to decision makers; and on occasion as the decision-maker.

There was one appeal at one of these universities that made its way to me, where an otherwise strongly performing undergraduate had been awarded a pass/ordinary degree because they had failed their dissertation.  Failing the dissertation could not be compensated as the department had designated passing as essential to the programme learning outcomes, so the board of examiners awarded a pass/ordinary degree even though the student’s overall mark was still in the upper second range.

The student appealed, from memory on the grounds that this was unfair, and at the initial stages of the appeal it was rejected.  Fairness was not the issue; they had not met the requirements of the programme regulations for award of an honours degree.

Eventually the appeal file made its way to me.  The student had failed the dissertation, but the programme regulations did permit the dissertation to be compensated.  The student was entitled to an upper second.

As can be imagined, pointing this out made me incredibly popular with the department in question.  So we did that dance well known to many in professional services.

The department said they were right, so the line had to be held that they weren’t.  They then switched tack, to argue that the programme regulations had been amended in error by the institutional quality team.  We told them that would not be the case.  They didn’t believe us.  We dug into the archive (and this is long enough ago it was literally a case of working through the paper in the archive boxes), and produced the audit trail showing that the department had requested to remove the requirement that the dissertation must be passed to be awarded an honours degree. Huffing and puffing, the department conceded the point.

The student’s appeal was upheld.

We also asked the registry to go back and check all pass/ordinary degree awards made by that department in the years since it had changed its programme regulations.  I can’t remember exactly how many students who were entitled to honours degrees had left with ordinary/pass degrees, but it was somewhere between five and nine and we insisted that these students’ degrees be reissued with the correct award and classification.

the implication

The point of this war story?

The university where this happened was a good university.  Clear and robust academic regulations.  Strong on process.  Centralised IT systems for recording and processing marks. Systems that presented these to boards of examiners in concise, clear and standard ways.

A student being denied the academic award their work merited?  It couldn’t happen here.

But it did.

Of course when it did we took action to right the wrongs done to these specific students.

And in future years there were developments that would have reduced the chances of it happening again. While that incident was not the only, or perhaps even the main, reason for doing so, we started to require that all chairs and secretaries of boards of examiners undertake specific training for their roles; and introduced the expectation that following this they would attend annual refresher sessions.

But when I left that University would I have been confident we had done enough?  At the time? Probably; but I never asked myself that question then.  In retrospect? Possibly; but this a question I’m only asking myself now, having read the QAA Scotland reports.

And having done that now, I’m fairly clear that the questions about assessment and awarding practices asked of Scottish universities in the QAA Scotland report are ones that every PVC Education and Academic Registrar (or equivalents) at every UK university, not just those in Scotland, should be asking themselves.  And actually, more than asking themselves. They should be actively assuring themselves of the answers to these questions. Anything less risks impacts on students that, even though we can only hope they will not be as catastrophic as occurred at Glasgow, are still hugely, negatively consequential.

Because this is one of the things that institutional autonomy means; seeing a potential issue and acting. Not because your regulator tells you to (which it hasn’t in England); but because your sense of responsibility to your students, and your role as a degree awarding body, mean it’s the right thing to do. And make sure this kind of thing doesn’t happen here.

And just for completeness, these are the questions set in the QAA Scotland report, framed there as lines of enquiry:

qaa’s lines of enquiry

• Are the institution’s current assessment regulations effective to ensure standards are maintained consistently (and awards made) for the provision it delivers?
• How does the institution ensure consistency of application, interpretation and understanding of assessment regulations (across all courses and its internal structure such as schools or faculties) and is this effective?
• How does the institution review its assessment regulations and procedures and is that review process effective?
• How does the institution ensure that award outcomes (including degree outcomes) are calculated and recorded consistently and is this effective?
• How does the institution assure itself that intended learning outcomes are met when applying its assessment regulations and is this effective? (e.g. ensuring intended learning outcomes are met when awarding credit).
• Where an institution operates both local arrangements (e.g. within a school, faculty, or similar) and an institution wide approach for the setting and implementation of regulations, how does the institution ensure that requirements for award of credit are in line with the institution-wide regulations or requirements and is this effective?
• If the institution works in partnership with other organisations, are the arrangements and responsibility for awarding arrangements clear and effective?

National Review of Awarding Arrangements: Guide for Institutions, para. 12