Academics and AI marking: a welcome aid for overworked staff or a sloppy shortcut?

Armin Alimardani has blue hair, is fresh faced, wears a smart watch and describes himself as “a tech enthusiast”.

He is an unlikely sentinel, perhaps, to warn that a drift towards academics using generative artificial intelligence to help mark students assignments could pose an existential threat to higher education.

The Western Sydney University senior lecturer in law and technology has been studying and experimenting with AI for years – he is speaking from Palermo in Italy, where he is attending a series of university workshops on the technology. He says, colleagues have been asking him lately if it could help lighten the burden of grading. Alimardani finds this talk “extremely concerning”.

“We’ve got to be really careful where we are stepping here,” he says. “Because I think there’s always a risk, at the moment, for higher education to collapse”.

AI being used to mark assignments is no theoretical prospect – but a fork in the road at which universities have suddenly found themselves confronted.

Alimardani’s is among several universities in Australia that allow staff to use AI in marking assessments and giving students feedback.

Though limited, fellow critics say such use of AI risks universities replacing the work of human academics on campuses with algorithms in datacentres – and that it could create a so-called “slop-cycle”, in which students submit AI generated assignments that academics mark with AI.

“Students are already financially struggling,” Alimardani says. “If they are not receiving proper feedback, if they feel like their degree is not as meaningful for employers and if they think that having a university degree doesn’t help them that much with guaranteeing a job – why would they come to university, really?”

The University of Newcastle, also in New South Wales, Deakin University and RMIT in Victoria and South Australia’s University of Adelaide all give similar allowances to WSU. In all those cases, it is a qualified green light for AI. Newcastle, for example, gives students the right to opt out and Deakin says staff can use AI to “improve efficiency and support assessment and administrative activities” but not “assign grades”.

A spokesperson said WSU took “a human-centred approach to generative AI use in assessment” in which “the awarding of marks, grades and feedback must also always be the responsibility of academic staff”.

“Academic staff may use AI tools to support certain aspects of assessment and feedback, however responsibility for academic judgment must always remain with university teaching staff,” the spokesperson said.

Alimardani says AI does have the potential to improve the quality of marking by acting as a “second pair of eyes” – he is working to create models that can pick out inconsistencies in human assessment. And the promise of AI assistance comes at a time when many university staff are overworked to “breaking point”, he notes, with marking taking up a large portion of academics’ time.

But claims of the potential benefit AI would bring to marking, he says, are largely untested, while the risk it poses is so great as to behoove a “really slow” and considered evaluation of those claims. Alimardani is not sure how AI could be used at scale to speed up assessments at all, without inviting disaster.

Imagine 100 academics are given an AI marking tool but told to verify its assessments, Alimardani says.

“Let’s say 80 do exactly as advised, there are going to be 20 who do not”.

He calls this “verification drift”: after a few AI generated assessments appear fine, the human charged with its oversight – often overworked themselves – stops being as vigilant and AI hallucinations slip through as facts.

Such scandals, as have been seen in courtrooms over recent years, would be almost inevitable, he says, tarnishing university reputations and fostering a sense of unfairness among students that would see them question the value of higher education.

Which may explain several high-ranking universities – including University of NSW, the University of Melbourne and the University of Sydney – draw the line at using generative AI in marking, despite enthusiastically allowing it in other aspects of the classroom.

Jury still out

For all those that have adopted a position on the matter, a number of universities say they are considering how they might use AI as a marking tool.

A Monash spokesperson said “at this time, AI is not used in the marking process”, but that the university was “transforming assessment through a programmatic approach” in which “AI is integrated into teaching and assessment to support students’ learning”.

Wollongong and James Cook universities responded similarly. Others appeared to dodge the question entirely.

A University of Queensland spokesperson said its academics are “required to confirm that all specified learning outcomes are achieved and that grades awarded reflect the level of student attainment”.

“UQ maintains an expectation that academics assess their students’ work and is actively developing policies to maintain this expectation in the age of AI,” the spokesperson said in response to a direct question.

Similar jargon was deployed by the Queensland University of Technology in an email sent to staff in late August advising them of its “assessment uplift project”.

“The project is a whole-of-university initiative to transform QUT’s assessment ecosystem so that assessment remains authentic, trusted, scalable and future-focused in an increasingly AI-enabled learning environment,” it read.

“It is currently progressing through initiation, with broad consultation underway to understand current experiences, challenges, and opportunities across the assessment lifecycle.”

Such “corporate speak” prompted one QUT academic to quip that it sounded like AI was writing to inform academics it was taking their job.

While a senior colleague, who also spoke on the condition of anonymity, said the reaction among staff was one of “widespread concern”.

Though “worryingly vague”, the academic said the email “certainly seems to be flagging that we will use less human marking if we can”.

They said sessional academics often rely on marking as a primary source of income in an already precarious line of work.

“People are concerned their job might be made redundant,” they said.

Why mark with AI if students can’t write with it?

As well as some staff having concerns about the environmental and ethical considerations, many are a long way from being convinced that AI could reliably and accurately assess work, they said, given the poor quality of AI generated assignments they were seeing some students try to pass off as their own.

“We are telling students all the time that there are integrity issues with their use of AI in assessment,” the academic said.

“What message does it send if we then say: but we are going to use it to mark you?”

For QUT creative and professional writing student, Alex Cameron, the message would be clear.

“At that point, we might as well just have AI everything,” he says. “Just get our degrees through AI and have AI write our assignments”.

An editor at the QUT’s Glass Magazine, Cameron says his three years of higher education have been an “awesome” time in which he has been surrounded by like-minded and interesting people – “the whole point of universities”.

But Cameron says he “wouldn’t pay $1000” for a degree in which he was marked with AI – and doesn’t think many of his cohort would be happy with the prospect.

“I would be irate,” he says. “I would be absolutely bloody incensed.

“We like to do human writing, we like to have humans look at it – and we like to get feedback from humans”.