Navigating the fog: the effectiveness of personalised conversational GenAI models for supporting ancient language learning

Ross, Edward A. S.; Baines, Jackie

Navigating the fog: the effectiveness of personalised conversational GenAI models for supporting ancient language learning

Lists

Tools

Ross, E. A. S. ORCID: https://orcid.org/0000-0003-4174-835X and Baines, J. (2025) Navigating the fog: the effectiveness of personalised conversational GenAI models for supporting ancient language learning. AI & Antiquity, 1 (1). pp. 35-52. ISSN 3081-4553

[thumbnail of 2025-09-24 - AI & Antiquity Ross-Baines-35-52.pdf]

Text - Published Version
· Restricted to Repository staff only
· The Copyright of this document has not been checked yet. This may affect its availability.
· Available under License Creative Commons Attribution Non-commercial.
1MB

It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

To link to this item DOI: 10.64946/aiantiquity.v1i1.002

Abstract/Summary

Hallucinations (misleading, inaccurate predicted text presented as fact) are a critical problem for using generative artificial intelligence (GenAI) tools to support ancient language teaching and learning. For a teacher, significant editing time is required to correct any inaccuracies or misrepresentations prior to making use of AI-generated content to support their teaching practice. For students, these convincing errors may not be recognised, and this may lead to misconceptions in their knowledge formation. OpenAI and Google released public-facing, customizable conversational AI models which allow users to upload their own datasets to create personalised AI chat agents, known as GPTs (2023) and Gems (2024) respectively. This presents an opportunity for teachers to personalize their own models to streamline their students’ experiences. However, can personalised conversational AI tools provide a fine-tuned experience that reduces the major, problematic ancient history and ancient language hallucinations that we see in standard ChatGPT and Gemini outputs? This paper discusses the creation of a personalised Latin Tutor GPT and Gem through the development of a series of exhaustive Latin vocabulary spreadsheets. We tested these personalised tools against their standard GenAI counterpart to determine if personalisation improved their efficacy and efficiency for supporting ancient language learning. The development of the spreadsheets and testing process both closely addressed current GenAI ethical issues, including copyright, environmental impact, and content restrictions. The results of these tests found that personalised GPTs and Gems made small efficacy and efficiency improvements, but the time and energy required greatly outweighed the results.

Item Type:	Article
Refereed:	Yes
Divisions:	Arts, Humanities and Social Science > School of Humanities > Classics
ID Code:	124613
Uncontrolled Keywords:	Ancient Language Learning, Generative Artificial Intelligence, Latin, OpenAI, Gemini, AI Ethics
Publisher:	Center for Innovation in Ancient Worlds

Altmetric

Deposit Details

References

Ali, M. (2023) ‘A Beginner’s Guide to Using the ChatGPT API’, datacamp, 14th December. Available at: https://www.datacamp.com/tutorial/a-beginners-guide-to-chatgpt-api (Accessed 04 May 2025). Baines, J., Ross, E. A. S., Hunter, J., McRitchie Pratt, F. and Patel, N. (2024) Digital Tools for Learning Ancient Greek and Latin and Guiding Phrases for Using Generative AI in Ancient Language Study. 3rd edn. Available at: https://doi.org/10.6084/m9.figshare.25391782.v3 (Accessed: 04 May 2025). Bogmans, C., Gomez-Gonzalez, P., Melina, G. and Thube, S. (2025) ‘AI Needs More Abundant Power Supplies to Keep Driving Economic Growth’, IMF Blog, 13th May. Available at: https://www.imf.org/en/Blogs/Articles/2025/05/13/ai-needs-more-abundant-power-supplies-to-keep-driving-economic-growth (Accessed: 20 May 2025). Creative Commons. (2013) CC BY-SA 4.0: Attribution-ShareAlike 4.0 International – Deed. Available at: https://creativecommons.org/licenses/by-sa/4.0/ (Accessed: 20 April 2025). De Freitas, I. (2025) ‘GenAI and copyright: three key cases’, Farrer & Co, 9th July. Availabe at: https://www.farrer.co.uk/news-and-insights/genai-and-copyright-three-key-cases/ (Accessed 10 July 2025). Department of Education. (2025) Generative artificial intelligence (AI) in education. Available at: https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education/generative-artificial-intelligence-ai-in-education (Accessed: 22 April 2025). Google. (2024a) Gems (13 March 2025 version) [computer software]. Available at: https://gemini.google.com/gems/create. Google. (2024b) ‘New in Gemini: Custom Gems and improved image generation with Imagen 3’, The Keyword, 28th August. Available at: https://blog.google/products/gemini/google-gemini-update-august-2024/ (Accessed: 04 May 2025). Google. (no date) ‘Policy guidelines for the Gemini app’, Gemini. Available at: https://gemini.google/policy-guidelines/?hl=en-GB# (Accessed 19 April 2025). Mollick, E. and Mollick, L. (2023) ‘Assigning AI: Seven Approaches for Students with Prompts’, The Warton School Research Paper. Available at: https://dx.doi.org/10.2139/ssrn.4475995 (Accessed 10 March 2025). O’Donnell, J. and Crownhart, C. (2025) ‘Everything you need to know about estimating AI’s energy and emissions burden’, MIT Technology Review, 20th May. Available at: https://www.technologyreview.com/2025/05/20/1116331/ai-energy-demand-methodology/ (Accessed: 20 May 2025). OpenAI. (2023a) GPTs (21 February 2025 version) [computer software]. Available at: https://chatgpt.com/gpts/editor. OpenAI. (2023b) ‘Introducing GPTs’, OpenAI, 6th November. Available at: https://openai.com/index/introducing-gpts/ (Accessed: 04 May 2025). OpenAI. (2025) ‘Usage Policies’, OpenAI, 29th January. Available at: https://openai.com/policies/usage-policies/ (Accessed: 19 April 2025). Pogrebna, O. (2024) ‘AI is a multi-billion dollar industry. It’s underpoinned by an invisible and exploited workforce’, The Conversation, 8th October. Available at: https://theconversation.com/ai-is-a-multi-billion-dollar-industry-its-underpinned-by-an-invisible-and-exploited-workforce-240568 (Accessed 20 April 2025). Randieri, C. (2025) ‘Bias and Corruption in Artificial Intelligence: A Threat to Fairness’, Forbes, 14th March. Available at: https://www.forbes.com/councils/forbestechcouncil/2025/03/14/bias-and-corruption-in-artificial-intelligence-a-threat-to-fairness/ (Accessed 14 March 2025). Romanou, A., Foroutan, N., Sotnikova, A., Chen, Z., Nelaturu, S. H., Singh, S., Maheshwary, R., Altomare, M., Haggag, M. A., Snegha, A., Amayuelas, A., Amirudin, A. H., Aryabumi, V., Boiko, D., Chang, M., Chim, J., Cohen, G., Dalmia, A. K., Diress, A., Duwal, S., Dzenhaliou, D., Florez, D. F. E., Farestam, F., Imperial, J. M., Islam, S. B., Isotalo, P., Jabbarishiviari, M., Karlsson, B. F., Khalilov, E., Klamm, C., Koto, F., Krzemiński, D., de Melo, G. A., Montariol, S., Nan, Y., Niklaus, J., Novikova, J., Ceron, J. S. O., Paul, D., Ploeger, E., Purbey, J., Rajwal, S., Ravi, S. S., Rydell, S., Santhosh, R., Sharma, D., Skenduli, M. P., Moakhar, A. S., Moakhar, B. S., Tamir, R., Tarun, A. K., Wasi, A. T., Weerasinghe, T. O., Yilmaz, S., Zhang, M., Schlag, I., Fadaee, M., Hooker, S., and Bosselut, A. (2024) ‘INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge’, arXiv preprint. doi: https://doi.org/10.48550/arXiv.2411.19799. Ross, E. A. S. (2023) ‘A New Frontier: AI and Ancient Language Pedagogy’, The Journal of Classics Teaching 24(48), pp. 143-161. doi: https://doi.org/10.1017/S2058631023000430. Ross, E. A. S. (2025) ‘Introductory Latin Personalized GenAI Tool Dataset’. Version 2. doi: https://doi.org/10.6084/m9.figshare.29261450.v2 (Accessed: 06 June 2025). Ross, E. A. S. and Baines, J. (2024) ‘Treading Water: New Data on the Impact of AI Ethics Information Sessions in Classics and Ancient Language Pedagogy’, The Journal of Classics Teaching 25(50), pp. 181-190. doi: https://doi.org/10.1017/S2058631024000412. Ross, E. A. S., Baines, J., Hunter, J., McRitchie Pratt, F., Patel, N., Richards-Fowkes, E. and Tandy, H. (forthcoming) ‘Taking the Helm: Collaborating with Students to Navigate Generative AI for Ancient Language Learning’ To be published in Digital Culture & Education. Saw, C. L. and Tan, B. Z. Y. (2025) ‘Unpacking copyright infringement issues in the GenAI development lifecycle and a peek into the future’, Computer Law & Security Review 58, pp. 1-17. doi: https://doi.org/10.1016/j.clsr.2025.106163. Shabanov, A. (2025) ‘Gemini users may soon share Gems as Google begins testing’, TestingCatalog, 8th May. Available at: https://www.testingcatalog.com/gemini-users-may-soon-share-gems-as-google-begins-testing/ (Accessed: 15 May 2025). Vayani, A., Dissanayake, D., Watawana, H., Ahsan, N., Sasikumar, N., Thawakar, O., Ademtew, H. B., Hmaiti, Y., Kumar, A., Kukreja, K., Maslych, M., Al Ghallabi, W., Mihaylov, M. M., Qin, C., Shaker, A. M., Zhang, M., Ihsani, M. K., Esplana, A. G., Gokani, M., Mirkin, S., Singh, H., Srivastava, A., Hamerlik, E., Izzati, F. A., Maani, F. A., Cavada, S., Chim, J., Gupta, R., Manjunath, S., Zhumakhanova, K., Rabevohitra, F. H., Amirudin, A. H., Ridzuan, M., Kareem, D. N. A., More, K. P., Li, K., Shakya, P., Saad, M., Ghasemaghaei, A., Djanibekov, A., Azizov, D., Jankovic, B., Bhatia, N., Cabrera, A., Obando-Ceron, J., Otieno, O., Farestam, F., Rabbani, M., Ballah, S., Sanjeev, S., Shtanchaev, A., Fatima, M., Nguyen, T., Kareem, A., Aremu, T., Xavier, N. A. Z., Bhatkal, A., Toyin, H. O., Chadha, A., Cholakkal, H., Anwer, R. M., Felsberg, M., Laaksonen, J., Solorio, T., Choudhury, M., Laptev, I., Shah, M., Khan, S., and Khan, F. S. (2025) ‘All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages’, Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 19565-19575. doi: https://doi.org/10.48550/arXiv.2411.16508. Wikimedia Foundation. (2023) Wikimedia Foundation Terms of Use. Available at: https://foundation.wikimedia.org/wiki/Policy:Terms_of_Use (Accessed: 20 April 2025).

University Staff: Request a correction | Centaur Editors: Update this record

University of Reading

CentAUR: Central Archive at the University of Reading

Accessibility navigation

Navigating the fog: the effectiveness of personalised conversational GenAI models for supporting ancient language learning

Abstract/Summary

Page navigation

See also

Footer navigation