Vikki O'Neill¹
James Scholar¹, Kathy Cullen¹ and ^S. Helen Oram²
1 Queen's University Belfast
2 King's College London

An advantage to computer delivered assessment, is the ability to gather real-time information about candidate interaction with individual questions and assessments as a whole (“clickstream” data) [1]. Selection of items, question order navigation, view time, bookmarking and repeated viewing, may all be interrogated. 

Understanding student exam strategies, also known as “testwiseness”, is vital, as whilst exams aim to quantify knowledge, factors such as test anxiety, motivation, and strategic approaches will also impact test outcomes. Testwiseness will not only effect scores, but also play a secondary role in reducing anxiety and sustaining motivation [2,3]. 

Summary of work 

Clickstream data from an online invigilated Single Best Answer (SBA) Progress Test, given to medical students in their first 3 years (870 students, 100 items) was analysed. Employing classification techniques in R, six groups were formed based on students structural strategies, such as question skipping and reviewing. 

Results 

Results indicate similar levels of strategies across the year groups, with the most popular strategy being to review all questions once (39.85% Y1, 41.97% Y2, 42.37% Y3). Reviewing all 100 questions as a second pass improves a score by 0.31% (p=0.0516). Students’ scores were found to be significantly different across the strategies for year 3 only (p=0.0014). A small proportion of students were found to have a true non-sequential approach, apparently selecting questions at random, which bears further study. 

Discussion
Distinct reviewing strategies were noted, with no variation in proportions by year group. Reviewing questions at least once each was of some benefit to students. These strategies provide insights into how we might best support borderline students and those entitled to additional exam time. 

Conclusions/Take-home messages
Clickstream data offers an opportunity to better understand students’ testwiseness. Exam strategies benefit students, and consequently we should consider teaching students how to utilise these to their advantage. 

References (maximum three) 

1. McManus, I.C., Chis, L., Ferro, A., Oram, S.H., Galloway, J., O'Neill, V., Myers, G. and Sturrock, A., (2023). “Visualising candidate behaviour in computer-based testing: Using ClickMaps for exploring ClickStreams in undergraduate and postgraduate medical examinations”. medRxiv, pp.2023-06. 

2. Mahamed, A., Gregory, P. A., & Austin, Z. (2006). "Testwiseness" among international pharmacy graduates and Canadian senior pharmacy students. American journal of pharmaceutical education, 70(6), 131. https://doi.org/10.5688/aj7006131 

3. Russo, James A. (2019) "The impact of a short test-wiseness intervention on standardised numeracy assessment scores: A cautionary tale about using NAPLAN growth data to evaluate primary schools," Networks: An Online Journal for Teacher Research: Vol. 21: Iss. 2. https://doi.org/10.4148/2470-6353.1301