Presentation Description
Matthew Sibbald1
Tom Alexander1, Haroon Yousuf1, Deborah Azim Fleming1 and Adrian Alexander2
1 Michael G. DeGroote School of Medicine, McMaster University, Hamilton, CANADA.
2 Alfred Health, Melbourne, VIC, AUSTRALIA
Tom Alexander1, Haroon Yousuf1, Deborah Azim Fleming1 and Adrian Alexander2
1 Michael G. DeGroote School of Medicine, McMaster University, Hamilton, CANADA.
2 Alfred Health, Melbourne, VIC, AUSTRALIA
Background:
Narrative assessments are ubiquitous in medical student assessments.1 There is a paucity of research into the validity, reliability or quality of narrative assessments particularly for undergraduate medical students.1 There is some preliminary research and emerging evidence on what constitutes quality in narrative assessments.1
Importance:
There is little research into how student assessment in general; and narrative assessments in particular, varied in quality during the “far”pandemic/virtual state as compared to the “near” pre-pandemic/in- person state.
Method:
Faculty raters applied the 7-point narrative quality index drawn from Chakroun et al.comprehensive scoping review while blinded to whether the de-identified and randomly drawn narrative assessment from our database, was created in the pandemic/virtual vs. pre-pandemic/in-person states and whether they resulted in a satisfactory result or a provisional satisfactory result requiring remediation. In addition to assigning quality estimates to the narrative assessments, faculty raters guessed the state in which the narrative assessment was assigned and whether the result was satisfactory or provisional satisfactory requiring remediation. Our database was analyzed for main effect statistical differences and correlations of quality across a number of predictive variables between the pandemic and pre-pandemic states.
Workshop format:
Active learning workshop format using interactive participant e- polling platforms sequenced with progressive reveal of our results to promote engagement and discussion.
Who should participate:
Learners and faculty interested in medical student assessment and medical education research.
Level of workshop:
Intermediate.
Workshop outcomes:
1) Review the literature on narrative assessments for medical students
2) Describe 3 challenges for faculty assessors in writing narrative assessments for medical students in the pre-clerkship.
3) Compare estimates of narrative assessment quality from your setting and context with our data during a progressive reveal of study results.
4) Implement applications of virtual vs. in-person assessment in your setting with greater specificity based on our literature review and study results.
Narrative assessments are ubiquitous in medical student assessments.1 There is a paucity of research into the validity, reliability or quality of narrative assessments particularly for undergraduate medical students.1 There is some preliminary research and emerging evidence on what constitutes quality in narrative assessments.1
Importance:
There is little research into how student assessment in general; and narrative assessments in particular, varied in quality during the “far”pandemic/virtual state as compared to the “near” pre-pandemic/in- person state.
Method:
Faculty raters applied the 7-point narrative quality index drawn from Chakroun et al.comprehensive scoping review while blinded to whether the de-identified and randomly drawn narrative assessment from our database, was created in the pandemic/virtual vs. pre-pandemic/in-person states and whether they resulted in a satisfactory result or a provisional satisfactory result requiring remediation. In addition to assigning quality estimates to the narrative assessments, faculty raters guessed the state in which the narrative assessment was assigned and whether the result was satisfactory or provisional satisfactory requiring remediation. Our database was analyzed for main effect statistical differences and correlations of quality across a number of predictive variables between the pandemic and pre-pandemic states.
Workshop format:
Active learning workshop format using interactive participant e- polling platforms sequenced with progressive reveal of our results to promote engagement and discussion.
Who should participate:
Learners and faculty interested in medical student assessment and medical education research.
Level of workshop:
Intermediate.
Workshop outcomes:
1) Review the literature on narrative assessments for medical students
2) Describe 3 challenges for faculty assessors in writing narrative assessments for medical students in the pre-clerkship.
3) Compare estimates of narrative assessment quality from your setting and context with our data during a progressive reveal of study results.
4) Implement applications of virtual vs. in-person assessment in your setting with greater specificity based on our literature review and study results.
References (maximum three)
1) Chakroun, Molk et al., Quality of Narratives in Assessment: Piloting a List of Evidence- Based Quality Indicators, Perspectives in Medical Education, 12(1) May 26, 2023
2) Hatala, Rose et al. Using in-training evaluation report(ITER) qualitative comments to assess medical students and residents: a systematic review. Academic Medicine, 92(6) 2017, p. 868- 79.
3) Chan, Teresa, Monteiro, Sandra et al., The Quality of Assessment of Learning(QuAL) score: Validity evidence for a scoring system aimed at Rating short, workplace based comments on trainee performance, Teaching and Learning in Medicine, an International Journal Vol. 32, Issue 3. 2020.