Accessibility navigation


A real-world test of artificial intelligence infiltration of a university examinations system: a “Turing Test” case study

Scarfe, P. ORCID: https://orcid.org/0000-0002-3587-6198, Watcham, K., Clarke, A. and Roesch, E. ORCID: https://orcid.org/0000-0002-8913-4173 (2024) A real-world test of artificial intelligence infiltration of a university examinations system: a “Turing Test” case study. PLoS ONE, 19 (6). e0305354. ISSN 1932-6203

[img]
Preview
Text (Open Access) - Published Version
· Available under License Creative Commons Attribution.
· Please see our End User Agreement before downloading.

2MB
[img] Text - Accepted Version
· Restricted to Repository staff only

2MB

It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

To link to this item DOI: 10.1371/journal.pone.0305354

Abstract/Summary

The recent rise in artificial intelligence systems, such as ChatGPT, poses a fundamental problem for the educational sector. In universities and schools, many forms of assessment, such as coursework, are completed without invigilation. Therefore, students could hand in work as their own which is in fact completed by AI. Since the COVID pandemic, the sector has additionally accelerated its reliance on unsupervised ‘take home exams’. If students cheat using AI and this is undetected, the integrity of the way in which students are assessed is threatened. We report a rigorous, blind study in which we injected 100% AI written submissions into the examinations system in five undergraduate modules, across all years of study, for a BSc degree in Psychology at a reputable UK university. We found that 94% of our AI submissions were undetected. The grades awarded to our AI submissions were on average half a grade boundary higher than that achieved by real students. Across modules there was an 83.4% chance that the AI submissions on a module would outperform a random selection of the same number of real student submissions.

Item Type:Article
Refereed:Yes
Divisions:Life Sciences > School of Psychology and Clinical Language Sciences > Department of Psychology
ID Code:116685
Publisher:Public Library of Science

Downloads

Downloads per month over past year

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation