Simone Roberts 

What learning experiences contribute to the preparedness for practice of Monash University medical students transitioning into their internship? 

This PhD study looks at preparedness for practice (P4P) through a social constructivist lens. P4P has been conceptualised as experience, knowledge, confidence, self-awareness, skills and competence (1). A contribution analysis methodology was chosen to describe the interacting factors influencing a student’s P4P (2). Application of contribution analysis to medical education research has not been widely conducted, however it has been proposed as a model that may be able to help shift evaluation from outcomes-based methods towards models that account for complexity (2). 

A Theory of Change (TOC) will be developed using literature review, data from a Monash medicine exit-level student P4P survey, review of key Monash MD documentation and input from course leaders and a new medicine graduate, on why graduating students feel prepared, or not. 

Qualitative data will be collected from key stakeholders identified in the TOC to review and revise the TOC and develop the contribution story. This data will be analysed using a thematic analysis approach (3). Stakeholders include medical students, new graduates, teaching staff and employers and supervisors of Monash medicine students. Data will be compiled to develop the final contribution story. 

The student survey shows that factors such as assessment methods, practice opportunities, placement experiences, curriculum and personal experiences impact on preparedness. My thesis will explore why these factors contribute to P4P. 

1. Are there limitations to the contribution analysis approach in medical education research and how might we mitigate these? 

2. Are there other evaluation approaches that would be superior? 

References (maximum three) 

1. Ottrey E, Rees CE, Kemp C, Brock TP, Leech M, Lyons K, et al. Exploring health care graduates’ conceptualisations of preparedness for practice: A longitudinal qualitative research study. Med Educ. 2021;55(9):1078-90. 

2. Van Melle E, Gruppen L, Holmboe ES, Flynn L, Oandasan I, Frank JR, International Competency-Based Medical Education Collaborators. 2017. Using contribution analysis to evaluate competency-based medical education programs: it’s all about rigor in thinking. Acad Med. 92(6):752–758. 

3. Clarke V, Braun V. Thematic analysis. The journal of positive psychology. 2017;12(3):297- 8. 

Michelle Hannington¹
1 University of Cape Town

Research Question?
How do we ensure valid assessment practice for the profession of OT in the SA context? 

Thesis methodology
The thesis is adopting a four phase approach:
1) A scoping review on the assessment literature in OT education globally, to identify the current landscape and a critical discourse analysis to analyse influencing factors.
2) A narrative enquiry of assessment practice in SA, to identify and critique the factors that influence assessment practice locally, using a historical epistemology approach.
3) The development of a ‘Philosophy of Assessment' for the profession of OT in SA, using a constructivist grounded theory (CGT) approach. The third phase will focus on semi-structured interviews and focus groups to explore the current espoused assessment philosophies and to generate values, belief and attitudes underpinning a philosophy of assessment for OT education in SA. 
4) The final phase will continue the CGT, using nominal group techniques to generate a philosophy of assessment. 

3. Findings so far
The scoping review has been conducted and identified two broad influencing factors: contextual factors (influence of the global north) and professional factors (influence of medical education). These findings highlight power dynamics that have resulted in a ‘lift and shift’ phenomenon. 

4.) Questions for discussion
Due to the history of South Africa and the development of OT in SA, it is relevant to operate within a critical paradigm when engaging in the first two phases of the study, in order to critique and identify the factors influencing practice. However, for the CGT a constructivist paradigm is more appropriate, as the intention is to generate a philosophy, which is pluriversal in nature. The factors identified in the first two phases would be used as sensitising concepts in the second two phases. Is it possible to straddle two paradigms and what considerations should be made? 

References (maximum three) 

Tavares, W., Pearce, J., & Eva, K. (2022). Tracing Philosophical Shifts in Health Professions Assessment. In Applied Philosophy for Health Professions Education: A Journey Towards Mutual Understanding (pp. 67–84). Springer Nature. https://doi.org/10.1007/978-981-19- 1512-3_6 

St-Onge, C., Young, M., Eva, K. W., & Hodges, B. (2017). Validity: One word with a plurality of meanings. Advances in Health Sciences Education, 22(4), 853–867. https://doi.org/10.1007/s10459-016-9716-3 

Varpio, L. & MacLeod, A. (2020). Philosophy of Science Series: Harnessing the Multidisciplinary Edge Effect by Exploring Paradigms, Ontologies, Epistemologies, Axiologies, and Methodologies. Academic Medicine, 95 (5), 686-689. doi: 10.1097/ACM.0000000000003142. 

David Kok^1,2,3
Kristie Matthews2, Steve Trumble³ and Caroline Wright²
1 Peter MacCallum Cancer Centre
2 Monash University
3 University of Melbourne

Background 
Assessment tasks in Health Professional Education (HPE) require periodic renewal to stay contemporary. This is particularly true in recent years with the educational landscape seeing notable advances in pedagogical theory, educational technology and the influence of Covid- 19. However, there is little data about current renewal rates, renewal practices and the effectiveness of these practices. 

Research Question(s)

1. a) How regularly are HPE Educators performing assessment renewal? b) To what degree are assessments being modified in these assessment renewals? 

2. What processes are HPE educators currently utilising when performing assessment renewal? 3. What changes to these processes may improve HPE assessment renewal practices?

Thesis methodology
1. Review of all Australian postgraduate HPE teaching qualifications from 2018 onwards. Assessments were compared between 2018 and 2023, differences quantified, pedagogical drivers for change identified, and trends in practice analysed.

2. Mixed methods. A) Survey of HPE educators on the processes of curriculum renewal they have utilised. B) targeted interviews of selected HPE educators to deep-dive into these processes. C) Perform process mapping and trend identification from these results. 

3. Method TBD depending on results of Q2. 

Q1: (Complete – abstract submitted to this conference!) 
421 assessments from 77 subjects were reviewed. Significant changes in assessment practice occurred in Australian HPE teaching courses with 48.5% of subjects undergoing assessment changes between 2018 and 2023. Significant drivers of assessment change included adaptations to online assessment and evolutions in assessment philosophy. 

Questions for discussion with participants
Currently refining methodology around Q2: 

What section of HPE educators should be targeted for this project (Australian only? Medical educators?) 

Is the current mixed methods proposal the optimal way of mapping current renewal processes? (and what Qs should be asked in this?) 

Discussion of best methodology to answer research Q3 Supervisor will be in attendance 

References (maximum three) 

Cox, M., Cox, M., & Irby, D. M. (2007). Assessment in Medical Education. The New England Journal of Medicine, 356(4), 387–396. 

Eraky, M. A. (2021). Evolving trends for assessment in the new norm of medical education. South‐East Asian Journal of Medical Education, 15, 3. 

Harris, P., Bhanji, F., Topps, M., Ross, S., Lieberman, S. A., Jason R. Frank, Frank, J. R., Linda Snell, Snell, L., & Sherbino, J. (2017). Evolving concepts of assessment in a competency-based world. Medical Teacher, 39(6), 603–608.