Assessing the impact of a university transition online course on student continuation using statistical matching methods

Wong, B. ORCID: https://orcid.org/0000-0002-7310-6418 and Fletcher, L. (2025) Assessing the impact of a university transition online course on student continuation using statistical matching methods. Evaluation Review, 49 (6). pp. 969-999. ISSN 1552-3926 doi: 10.1177/0193841X251339686

Abstract/Summary

This study demonstrates how to evaluate a university-wide online course designed to support student transition into university by using Propensity Score Matching (PSM) and Doubly Robust Estimation (DRE). Using data from seven academic years, from 2016/17 to 2022/23, with more than 28,000 students, we examine whether enrolment in this optional pre-arrival course affects first-year pass rates. We also conducted additional analyses to compare outcomes from the year before and after the course’s implementation, as well as to examine these patterns across recent cohorts to potentially account for contextual changes over time. Results indicate that enrolled students show a 6.2 percentage point increase in the likelihood of passing Year 1, controlling for factors including sex, domicile, age, ethnicity, disability and socioeconomic status. We demonstrate how utilising existing institutional data can potentially strengthen evidence of impact for centralised initiatives and conclude with reflections on the use of such institutional data and matching techniques and their viability for future evaluations.

Item Type	Article
URI	https://centaur.reading.ac.uk/id/eprint/122406
Identification Number/DOI	10.1177/0193841X251339686
Refereed	Yes
Divisions	Arts, Humanities and Social Science > Institute of Education > Improving Equity and Inclusion through Education
Publisher	Sage
Download/View statistics	View download statistics for this item