Neville Chiavaroli¹
Jacob Pearce¹
 1 Australian Council for Educational Research 

Marking schemes are essential for guiding assessors to evaluate candidate responses in line with learning objectives and the requirements of constructed-response tasks. They contribute to the validity of written assessments, as reflected in contemporary validity frameworks which recognise the importance of the scoring phase of assessment (Cook et al, 2015). In health professions assessment contexts, such schemes can be quite prescriptive, allocating specific marks to relevant and correct statements by candidates. Such ‘points-based’ schemes reflect the common focus on knowledge of typical written examination questions. However, for more complex questions aiming to elicit higher-order reasoning, such as diagnostic reasoning or justifications for proposed management, marking schemes need to enable and guide suitable assessor judgement. 

Discussion 

‘Levels-based’ marking schemes require assessors to identify broad levels of anticipated responses according to the accuracy, defensibility and/or overall quality of the response. These levels are ideally based on an underlying principle for discriminating between responses, along with a suitable range of marks allocated to responses judged at each level (Ahmed&Pollitt, 2011). A common concern among assessors is that such schemes are inherently subjective and therefore open to interpretation. Yet the alternative – drafting and applying points-based schemes to higher-order questions for the sake of ‘objectivity’ – risks undermining the validity of those assessment tasks. 

While the appeal of points-based schemes is understandable in terms of efficiency and perceived objectivity, the ultimate criterion for the choice of marking scheme is its alignment with the nature of the question and expected performance. Levels-based schemes, supported through appropriate marker training and calibration, provide more appropriate guidance to support marker appraisal of candidate responses to higher-order questions. As in clinical assessment more broadly, we need to accept and value the necessary subjectivity in assessment which comes with expert judgement (ten Cate & Regehr, 2019). 

References (maximum three) 

Ahmed A, Pollitt A. 2011. Improving marking quality through a taxonomy of mark schemes. Assess Educ. 18(3):259-278. 

Cook DA, Brydges R, Ginsburg S, Hatala R. 2015. A contemporary approach to validity arguments: a practical guide to Kane's framework. Med Educ. 49(6):560-575. 

ten Cate O, Regehr G. 2019. The Power of Subjectivity in the Assessment of Medical Trainees. Acad Med. 94(3):333-337.