Mike Tweed¹
Robin Willink¹ and Tim Wilkinson¹
1 University of Otago

In clinical practice, the certainty a clinician has in their clinical-decisions, including whether they need to seek further resources, is important. Such self-monitoring can be assessed in medical students using response certainty in MCQs, but how might it be followed over time? 

With each answer on MCQ progress tests, medical students provided their certainty of its correctness. We have proposed aspects of self-monitoring as including insightfulness (increasing correctness with rising certainty), safety (high probability correctness for ‘high certainty’ responses) and efficiency (low probability correctness for ‘no certainty’ responses). Each could be classified as present, absent, or undetermined. A tracking system was developed using first principles and data from one cohort of students, and a dataset from a second cohort was used as an independent check. 

The patterns of aspects of self-monitoring were consistent across both cohorts. Nearly all the students met the criteria for insightfulness on all tests. Most of the students met the criteria for efficiency, with the highest prevalence mid-course, whereas the absence of efficiency increased later. Most safety results were undermined, but when a definitive result was obtained, it was more likely to be absent mid-course, and present later in the course. 

Throughout the course, students showed reassuring levels of insightfulness. The results suggest that students may balance safety with efficiency. This may be explained by students learning the positive implications of decisions earlier, becoming more efficient; and negative implications later, becoming more cautious and safer. 

Analysis of correctness for different levels of certainty allowed for the tracking of self- monitoring as students progressed through the course, and revealed differences in insightfulness, safety, and efficiency. 

Take-home messages
Item response certainty has the potential to introduce self-monitoring and track it. 

Johnson WR, Durning SJ, Allard RJ, Barelski AM, Artino Jr AR. A scoping review of self‐monitoring in graduate medical education. Medical Education 2023. 

McConnell MM, Regehr G, Wood TJ, Eva KW. Self-monitoring and its relationship to medical knowledge. Advances in Health Sciences Education 2012; 17(3): 311-23. 

Tweed M, Purdie G, Wilkinson T. Low performing students have insightfulness when they reflect‐in‐action. Medical Education 2017; 51(3): 316-23.