Christian Gray¹
1 Academy for Medical Education, The University of Queensland Medical School 

As more countries move towards the need for standardised medical licencing exams, students are finding it difficult to simultaneously navigate learning for their own medical school’s curriculum and prepare for licencing exams. In partnership with the University of Queensland, MD students from the Ochsner Health System (New Orleans, LA) complete their first two years in Australia, before completing two additional years of medical training in the USA. Students must attempt the first of three National Board of Medical Examiners (NBME) United States Medical Licensing Examination (USMLE) Step exams (Step 1) prior to starting year 3 (1). A dedicated USMLE preparation course was established to support students’ preparation for Step 1. 

The purpose of this study was to evaluate discipline areas within clinical sciences that predict success in Step 1. Longitudinal performances in clinical science multiple choice questions in years 1 and 2 of the MD program were correlated with NBME Comprehensive Based Science Exam (CBSE) performance. 

The CBSE significantly correlates with performances in Step 1 and is used to assess students’ readiness to take the Step 1 exam (1). within the program. Students who perform above average in the cohort within clinical sciences, were significantly more likely to pass the CBSE, with higher correlation shown in performances in anatomy and pathology. 

Students report feeling overwhelmed and begin to not trust their medical education, which is often the best learning resource (2). There is compelling evidence that performances in clinical science strongly corelates with success in Step 1. Anatomy in year 1, followed by strong performances in Pathology provides a foundation of knowledge that are key to success in Step 1. 

A foundation in Anatomy learning followed by strong performance in Pathology provides the bases for success in medical licensing exams. 

References (maximum three) 

1. Johnson TR, Khalil MK, Peppler RD, Davey DD, Kibble JD. Use of the NBME Comprehensive Basic Science Examination as a progress test in the preclerkship curriculum of a new medical school. Adv Physiol Educ. 2014 Dec;38(4):315-20. 

2. Khalil MK, Wright WS, Spearman KA, Gaspard AC. Relationship between students' perceptions of the adequacy of M1 and M2 curricula and their performance on USMLE step 1 examination. BMC Med Educ. 2019 Sep 14;19(1):358 

Christian Gray¹
 1 Academy for Medical Education, The University of Queensland Medical School 

In partnership with the University of Queensland, MD students from the Ochsner Health System (New Orleans, LA) complete their first two years in Australia, before completing two additional years of medical training in the USA. Students must attempt the first of three National Board of Medical Examiners (NBME) United States Medical Licensing Examination (USMLE) Step exams (Step 1) prior to starting year 3 (1). A dedicated USMLE preparation course was established to support students’ preparation for Step 1, including access to a Step 1 questions bank. 

The purpose of this study was to evaluate student performance in Step 1 in relation to timing of completion of USMLE Step 1 questions. NMBE Comprehensive Based Science Exam (CBSE) exams were used to identify readiness to take Step 1. Completion of Step 1 questions were monitored through a question bank (UWorld). 

The CBSE significantly correlates with performances in Step 1 and is used to assess students’ readiness to take the Step 1 exam (1). The number of practice questions completed prior to Step 1 significantly correlated with Step 1 results. Early and persistent engagement with practice questions is correlated with success, with numbers of questions answered correctly correlating even more strongly with exam performance. 

Early and persistent engagement with Step 1 practice questions is an effective strategy to ensure students are read to sit Step 1 (2). Students who choose to “save questions” believing they are not ready are negatively impacting their progress. The strong correlation between questions answered correctly and performance suggests that an understanding of the content of the question is important, supported through a solid foundation in clinical sciences. 

Early, consistent, and correct completion of Step 1 questions are crucial factors associated with success in medical licensing examinations. 

References (maximum three) 

1. Johnson TR, Khalil MK, Peppler RD, Davey DD, Kibble JD. Use of the NBME Comprehensive Basic Science Examination as a progress test in the preclerkship curriculum of a new medical school. Adv Physiol Educ. 2014 Dec;38(4):315-20. doi: 10.1152/advan.00047.2014. PMID: 25434014. 

2. Andre D Kumar and others, preparing to take the USMLE Step 1: a survey on medical students’ self-reported study habits, Postgraduate Medical Journal, Volume 91, Issue 1075, May 2015, Pages 257–261 

Brooke Flew¹
Lucy Chipchase², Darren Lee³ and Jodie McClelland¹
1 La Trobe University
2 Flinders University
3 Australian Physiotherapy Council

Currently, to work in Australia, overseas-qualified physiotherapists (candidates) must pass an In-person Clinical Competency Assessment (ICCA) to gain registration in Australia (APC, 2021). However, the requirement to be in-person presents challenges for candidates not residing in Australia and could not be conducted during the COVID pandemic. An alternative Online Clinical Competency Assessment (OCCA) was developed and determined feasible. We then sought to examine the accuracy of the OCCA to distinguish between competent and non- competent performance. 

Candidates on the waiting list for ICCA assessment between March-June 2023 were invited to participate. Participants completed both OCCA and ICCA and were assessed for competency on the same criteria by a cohort of experienced assessors. 

Sixty-three candidates (mean (SD) age of 34.3 (4.8) years, 71.4% female, trained as a physiotherapist in 22 different countries) were recruited. The pass rate was higher for the ICCA (68.3%) than the OCCA (54.0%). Forty candidates (63.5%) scored the same on both assessments. Of the remaining 23 candidates, 16 (70%) failed the OCCA and passed the ICCA. 

There were similar pass rates between the OCCA and the ICCA, with just under 2/3rds of candidates receiving the same outcome on both assessments. The unfamiliarity of the online environment, or the additional difficulty of needing to describe rather than perform hands-on techniques explain the tendency for higher failure rates in the OCCA. 

Further analysis of the results is required to investigate if the OCCA is a suitable alternative to the ICCA. 

References (maximum three) 

Australian Physiotherapy Council. (2021). Standard Assessment Pathway. Retrieved 6 May 2021 from https://physiocouncil.com.au/overseas-practitioners/standard-assessment-pathway/ 

Minna Lladó¹
Henriette Lorenzen¹
1 University College Copenhagen 

Traditionally, some Biomedical Laboratory Science students have been interns on international clinical internships in Europe and Australia. Little is known about how international internships foster development of intern’s professional competencies.
 The aim of this study is therefore to identify the impact of formative assessment practice in international clinical research departments on employability skills of Biomedical Laboratory Science interns. 

Quantitative data (n=22) from internship evaluations (questionnaire) were collected from 2015- 22. We conducted focus groups with 12 former interns (1-7 years post-graduating) and interviewed 4 international research departments (AU, FI, DE, FO) and 1 core healthcare employer. Data was thematically analyzed. 

On a 5-point Likert scale, interns experienced a practice enhancing student achievement (average: 4.9) and obtaining profession-relevant skills (average: 4.6).
 Qualitative data elucidated following key formative assessment components (Black & William, 2009): Clinical supervisors clarify the significance of intern’s work, engineer effective trouble shooting laboratory meeting discussions, initiate dialogues that induce reflection and create belief in the intern’s ability to succeed (self-efficacy). 

Qualitative data also showed that employer experienced 10 employability skills (Jackson, 2015) when employing former interns, particularly self-management, professionalism and problem-solving. 

Through continuous formative assessment practice, interns in fact develop all the highly desired employability skills and are simultaneously kept motivated and persistent (Tinto, 2017) because clinical supervisors clarify learning intentions & criteria for success, elicit intern understanding and provide encouraging feedback, and as a result activate interns as owners of their own learning. 

Our study showed that formative assessment practice through work-integrated learning developed all 10 employability skills, especially self-management, professionalism and problem-solving which are desired competencies for employer. 

Take-home message: 

Continuous formative assessment practice embedded in authentic international clinical settings is a powerful tool in developing employability skills. 

References (maximum three) 

Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. 

Jackson, D. (2015). Employability skill development in work-integrated learning: Barriers and best practice. Studies in Higher Education, 40(2), 350–367. 

Tinto, V. (2017). Through the eyes of students. Journal of College Student Retention: Research, Theory and Practice, 19(3), 254–269. 

Julie Gustavs¹
Amanda Barnard, Viren Naik², Sidonie Frerotte¹, Kim Ashwin¹, Jen Desrosiers¹ and Vinita Rane³
1 Australian Medical Council
2 Medical Council of Canada
3 The University of Melbourne

Many countries have experienced long term growth in healthcare demand which has outstripped workforce growth. International Medical Graduates comprise over 30 percent of the Australian and over 25 percent of the Canadian medical workforces and are the mainstay of the medical workforce in rural and remote areas. 

Australia and Canada, along with other OECD countries, experience barriers to IMG assessment with individual and system level impacts. There are opportunities to learn from innovations across the two countries so that International Medical Graduates are assessed and transition effectively and efficiently into the health workforces of their chosen country. 

Drawing on qualitative research and review of literature, we investigate the experiences of International Medical Graduates during their immigration process. We examine the factors that influence their decision to migrate, the barriers they face in certification, and the strategies they employ to overcome these challenges. 

We share project results including analysis of complex regulatory requirements, language proficiency exams, limited recognition of qualifications, and cultural adaptation. The findings also identify opportunities for change such as improved integration programs, streamlined assessment processes, and enhanced support networks for International Medical Graduates. 

The discussion explores the impact of healthcare system challenges: the potential loss of skilled healthcare professionals; and the implications for patient care; and the importance of fostering a more inclusive and equitable immigration process that recognises the diverse experiences and qualifications of International Medical Graduates. 

This presentation shares international insights into assessment innovations through building on humanistic design exploring the challenges and opportunities for change. 

Take-home messages recognise the unique skills and experiences of International Medical Graduates, improving assessment processes to facilitate their integration, and strengthening support networks to enhance their cultural adaptation. Further research focuses on evaluating current policies, exploring innovative solutions, and advocating for evidence-based reforms. 

References (maximum three) 

Al‐Haddad, M., Jamieson, S., & Germeni, E. (2022). International medical graduates' experiences before and after migration: A meta‐ethnography of qualitative studies. Medical education, 56(5), 504-515. 

Gilles, M. T., Wakerman, J., & Durey, A. (2008). " If it wasn't for OTDs, there would be no AMS": overseas-trained doctors working in rural and remote Aboriginal health settings. Australian Health Review, 32(4), 655-663. 

Hollett, A., Hann, S., & Bradbury, C. (2008). A qualitative study of the international medical graduate and the orientation process. Canadian journal of rural medicine, 13(4), 163. 

Helen Kelly¹
1 The Royal College of Surgeons in Ireland

Background:
Much has been published on factors affecting international learners in the medical education landscape with limited studies on inclusive education solutions for this group of learners (Murray & Muller, 2019). This discussion relates to the rationale for a more uniform and equitable approach to all aspects of international learners needs, with a particular focus on assessment approaches in a multicultural learning context for international non-native English- speaking medical students (Bennett J.M. & Bennett M.J. 2004: Coleman, J. A 2006; Little. D 2007). 

Summary of work:
This oral presentation will discuss the findings of an evaluation of an integrated approach to assessment of non-native speaker international undergraduate medical students as part of their medical degree at the Royal College of Surgeons, University of Medicine & Health Sciences, Dublin, Ireland. This study hypothesises that a more holistic approach is worth considering to evaluate to what extent it allows international students a more supportive and positive learning environment where their language skills are encouraged to evolve with their broader learning needs in a framework of assessment as/of/for learning. 

Summary of results/discussion:
This mixed methods study uses quantitative and qualitative analysis to evaluate the outcomes of this approach to inclusive learning and assessment. The key findings of this study suggest that a culturally integrated approach to medical education which includes a considered communication skills focus, intercultural adaptations and which promote autonomous and formative forms of assessment methodologies are most effective in supporting a more inclusive learning programme for international medical students. 

Conclusion: 
The research aims to provide future recommendations for consideration of optimal learning and assessment environments adapted for non-native English speaking international medical and health science students to participate effectively in higher education. 

References (maximum three) 

Jesper Dalum¹
Gunilla Sandborgh Englund¹, Anders Själander² and Magnus Hultin²
1 Karolinska Institutet
2 Umeå University

Background
In 2016, a common pathway for international healthcare professionals (IHPGs) from outside the European Union was established in Sweden.[1] The candidate has to pass a theoretical and a practical test, 3-6 months of internship, a course in Swedish statutes regarding health and medical care, and provide evidence of Swedish at the C1-level. The largest groups of IHPGs passing the system are medical graduates (IMGs) and dentistry graduates (IDGs). 

Summary of work
The proficiency tests consist of a theoretical and a practical test developed and delivered by contracted Swedish universities and in line with the requirements for licensure[2] and fair testing.[3] The tests are taken in Swedish. Swedish law mandates that the tests become public once graded. 

For IMGs and IDGs, SBA questions are used for the theoretical test and OSCE for the practical. Results 

Four healthcare professions, medical, dentistry, pharmacy and nursing, represent over 90% of the IHPGs taking the tests. 

As of mid-2023, 851 IMGs and 243 IDGs have passed the respective Proficiency test. This is in sharp contrast to the high number of graduates that have made at least a first attempt at the theoretical test (2051 IMGs and 874 IDGs). In fact, the average pass rate on the theoretical test has been 23.3% for IMGs and 28.5% for IDGs. 

Discussion
A single and systematic approach for testing the proficiency of IHPGs moving to Sweden has been established. A relatively low throughput indicates problems with the tests or the candidates' readiness to take the test. There are indications that language proficiency might be an issue. 

Conclusions
The new system testing IHPGs coming to Sweden has a low pass rate but might still be fair. Take-home messages/implications for further research or practice
Further research is needed to determine the cause of the relatively low throughput. 

References (maximum three) 

1. The National Board of Health and Welfare. Application for a license in health care. 2016. https://legitimation.socialstyrelsen.se/en/licence-application/
 2. Swedish Government. The Higher Education Ordinance (1993:100). 1993. https://www.uhr.se/en/start/laws-and-regulations/Laws-and-regulations/The-Higher- Education-Ordinance/ 

3. Fairness in testing. In: STANDARDS for Educational and Psychological Testing. 2014. 

Maxine Te¹
Darren Lee¹
1 Australian Physiotherapy Council

1. Background
The Australian Physiotherapy Council implemented a remote model of its Written Assessment in July 2020, in response to the COVID pandemic. The Written Assessment is a 120-question MCQ exam that assesses application of knowledge, ordinarily conducted in-person in a examination venue. The remote model assessments can be completed at the candidate’s home or other venue and are remotely proctored by an invigilator. 

2. Summary of work
The Council was keen to evaluate the performance of overseas qualified physiotherapists (candidates) completing the remote online assessment, to understand whether different contexts impacted assessment outcomes. Assessment outcomes (n = 1795) of candidates who had completed the remote and in-person assessments were compared. A two sample-t test was used to compare mean scores between in-person and remote assessments for candidates in the Standard Assessment Pathway. 

3. Results
The difference between mean scores of candidates in the Standard Assessment Pathway were not statistically significant (t = 0.7338, p = 0.46). 

4. Discussion
Performance of candidates in the Standard Assessment Pathway were comparable between in- person and remote assessments. The results suggest that the context of the assessment (in- person or remote) may not have an impact on candidate test performance. These results provide evidence to support the use of remote delivery in high-stakes pre-registration examinations. 

5. Conclusions
Assessment bodies can consider the use of remote delivery models where in-person assessments are unviable. 

Further research around candidate experience and satisfaction would help inform improvements to remote delivery models. There is also an opportunity to explore how the efficiency and frequency of high-stakes remote assessments can be increased to offer candidates improved access to examinations. 

References (maximum three) 

NIL