Vicki Jones¹
Sharmyn Turner¹ and Nathan Ryckman¹
1 University of Auckland 

Critical reflection is central to professionalism (MCNZ, 2021). In years four and five (Phase 2) of the medical programme, students' reflective practice skills are assessed using a student-led written portfolio. Portfolios are marked and moderated using a descriptive rubric with eight (numerically graded) criteria, 

To test the reliability and defensibility of grading, the Phase 2 results data for 2021-2022 was statistically analysed. Outcomes were compared to a ‘blind’ benchmarking exercise that utilised the current marking and moderating process. 

Results 

Despite numerical grades values for ‘pass’ and ‘distinction’ outcomes overlapping, the numerical grades are significantly higher for ‘distinction’ compared to ‘pass’ outcomes. Further, the distribution of ‘distinction’ outcomes became less spread over time as assessors gain experience in utilising the qualitative assessment rubric. 

Summative assessment works to engage students’ reflective practice skills (Driessen et al., 2005). However, setting rigid assessment cut-points where qualitative assessment tools are used can serve to diminish the richness of the assessed content. Delineating between ‘pass’ and ‘distinction’ grades requires assessor flexibility and may not always be a direct reflection of a derived numerical assessment outcome (Biggs, 1992). 

There appears to be little statistical basis to support a particular cut-point for a ‘distinction’ over a ‘pass’. This may be desirable considering the qualitative nature of the assessment. Rather than adhering to a rigid numerical structure to delineate between ‘pass’ and ‘distinction’, the PPS markers and moderator are encouraged to base their outcomes on the overall synthesis of quality and depth observed in each portfolio. Harnessing the experience of a stable marking team is important to optimise assessment reliability. 

A revised rubric incorporating a te ao Māori perspective is in development. The final version will be analysed and benchmarked to ensure future grading remains robust and defensible. 

References (maximum three) 

Biggs, J. B. (1992). A qualitative approach to grading students. Herdsa News, 14(3), 3-6. 

Driessen, E.W., van Tartwijk, J., Overeem, K., Vermunt, J.D., ven der Vleuten, C.P.M. (2005). Conditions for successful reflective use of portfolios in undergraduate medical education. Medical Education, 39, 1230 – 1235. https://doi.org/10.1111/j.1365-2929.2005.02337.x 

Medical Council of New Zealand. (2021). Good Medical Practice. https://www.mcnz.org.nz/assets/standards/b3ad8bfba4/Good-Medical-Practice.pdf