Gergo Pinter¹
Daniel Zahra², Steven Burr¹, Thomas Gale², Jolanta Kisielewska¹, José Miguel Gomes Moreira Pêgo³ and Nuno Santos⁴
1 University of Plymouth
2 Peninsula Medical School
3 University of Minho
4 iCognitus4ALL - IT Solutions, Braga

Peninsula Medical School (PMS) is pioneering the introduction of Content Adaptive Progress Testing (CAPT) [1,2] for new students in 2023. Unlike traditional Computerised Adaptive Testing (CAT) [2] assessments that adapt during the exam by adjusting difficulty levels, CAPT adopts a new approach by adapting the content of questions in between assessments, tailoring each test to individual students based on their past performance. The CAPT approach was specifically designed to improve longitudinal progress testing for students and educators, meeting national licensing exam requirements. 

Each CAPT assessment consists of 125 personalised non-negatively marked multiple-choice questions. Questions are mapped to one of over 600 topics aligned with the UK General Medical Council (GMC) framework (gmc-uk.org/mla), promoting a comprehensive understanding of medical knowledge. Question selection rules prioritise fairness, balance, and attainability while quantifying student performance based on the number of unique correctly answered topics, eliminating norm-referencing. 

We present our experience of developing and delivering cohesive and student-centric CAPT assessments which build on the benefits of traditional identical question tests such as bidirectional navigation and a frequent but low-stake nature by eliminating compensation within and across assessments. CAPT's asynchronous delivery capability also offers a distinct advantage over non-adaptive methods with rising student numbers. However implementing fair question setting and progression decisions required new, more complex processes. 

Truly personalised assessments benefit students by providing incredibly detailed progression information, feedback and remediation options to guide their learning. Educators also benefit from greater teaching insights and reduced question-setting workloads. Finally, the general public also stands to gain from the CAPT method since it promotes the training of safe and generalist foundation doctors. 

The personalised CAPT approach gives rise to a plethora of benefits for students and educators alike alongside superior alignment to learning outcomes goals and is a compelling candidate for longitudinal assessments. 

References (maximum three) 

[1] Burr SA, Gale T, Kisielewska J, Millin P, Pêgo JM, Pinter G, et al. A narrative review of adaptive testing and its application to medical education [Internet]. Vol. 13, MedEdPublish. F1000 Research Ltd; 2023. p. 221. Available from: http://dx.doi.org/10.12688/mep.19844.1 

[2] Burr SA, Kisielewska J, Zahra D, Hodgins I, Robinson I, Millin P, et al. Personalising knowledge assessments to remove compensation and thereby improve preparation for safe practice - developing content adaptive progress testing [Internet]. Research Square Platform LLC; 2022. Available from: http://dx.doi.org/10.21203/rs.3.rs-1977511/v1 

[3] Rice N, Pêgo JM, Collares CF, Kisielewska J, Gale T. The development and implementation of a computer adaptive progress test across European countries [Internet]. Vol. 3, Computers and Education: Artificial Intelligence. Elsevier BV; 2022. p. 100083. Available from: http://dx.doi.org/10.1016/j.caeai.2022.100083