Hayley Croft¹
Josephine Maundu², Kirstie Galbraith³, Balakrishnan R (Kichu) Nair AM¹, Glenys Wilkinson² and Bronwyn Clark²
1 University of Newcastle
2 Australian Pharmacy Council
3 Monash University 

Background
In Australia, pharmacy graduates must complete a period of supervised practice (internship) as part of the requirements for registration. New workplace-based assessment (WBA) tools have been developed and introduced by the Australian Pharmacy Council (APC) to facilitate effective assessment of pharmacy interns. The tools include entrustable professional activities (EPAs), case-based discussion, and in-training assessments on health promotion and reflective practice(1). 

Summary of Work
APC is conducting an evaluation study to gather experiences from stakeholders through focus group, interviews, and an anonymous cross-sectional survey. We seek to gather feedback on the content, structure, format and understandability of the tools and supporting material; and perspectives on the impact of the tools on intern learning, intern/preceptor relationships and workload. 

Results
Five focus groups were conducted in May 2023 with Interns (n=11) and pharmacists (n=13). Feedback indicates that the WBA tools are beneficial in supporting workplace learning and feedback, enhance communication between preceptors and interns and optimise role allocation for interns. Increased workload, lack of time and poor WBA literacy are the key challenges expressed by both interns and pharmacists. Preliminary findings from other components of the study will be shared. 

Discussion
Competency based assessment approaches are increasingly being implemented in pharmacy education(2) and involve the use of WBA tools(3). Though not without its challenges competency-based assessment and the associated use of WBA is a valuable addition to pharmacy intern training. 

Conclusions
The introduction of WBA is providing a nationally consistent approach for Australian pharmacy intern training. Findings from the evaluation will inform improvements to the current tools and future expansion of WBA. 

Take home messages:
Our experiences on introducing a WBA for pharmacy intern training at a national level involving multiple sites in diverse settings can inform local and international health programs seeking to introduce WBA. 

References (maximum three) 

Asela Olupeliyawa¹
Sam Monk¹, Helen Wozniak¹, Anna Kull¹, Christine Devine¹, Christy Noble¹, Kym Ward² and Justine Gibson³
1 Academy for Medical Education, Medical School, University of Queensland
2 Medical School, University of Queensland
3 School of Veterinary Science, University of Queensland 

Background
Workplace-based assessments (WBAs) provide evidence of medical students’ clinical and non- clinical skills development in clinical settings. Learning analytics, extensively used to explore student engagement in classroom-based settings, are now emerging for WBA-related insights. Applying learning analytics in WBAs empowers educators and institutions with valuable insights to improve the learning experience through data-informed decisions. 

Summary of work
In 2022, we introduced a digital platform to the Workplace Learning Portfolio (WLP), a longitudinal course where students develop their skills through regular completion of WBAs in the final two years of the 4-Year medical program. While this learning is student-led, submission targets and deadlines encourage ongoing engagement. Mid-year and end-of-year reviews assess these targets, guiding progression decisions and supporting students review their learning opportunities. We conducted an analysis of the Year 3 data set (23920 submissions, 320 students) to identify patterns of engagement and associated factors. 

Results
Students with higher cumulative WBA completions show early and consistent engagement. Tracking average WBA submissions over the year revealed that students with lower prior semester GPA (<5) were slower to accumulate submissions and meet course requirements, while students with a higher GPA submitted more tasks, more frequently, and met course requirements earlier. This gap in completed tasks is visible by Week 7 of Semester 1. Placement type also matters, with students completing more WBAs during generalist placements. 

Discussion and conclusion
Prior GPA and lower WBA completion may inform identification of at-risk students who require more support to transition to workplace-based learning, where deliberate practice should be promoted over just in time completion. Resources to support students as they move between diverse clinical placements may assist to manage expectations around WBA opportunities. 

Take-home messages
Data-driven approaches to understand how and when students meet WBA requirements inform a more personalised approach to guiding learning in the workplace. 

References (maximum three) 

Wilson, K., & Lizzio, A. J. (2012) Engaging students who are at risk of academic failure: Frameworks and strategies. Invited Paper, 15th First Year in Higher Education Conference 2012, Brisbane. 

Piotrkowicz, A., Wang, K., Hallam, J. et al. (2021) Data-driven Exploration of Engagement with Workplace-based Assessment in the Clinical Skills Domain. International Journal of Artificial Intelligence in Education, 31, 1022–1052. https://doi.org/10.1007/s40593-021- 00264-0 

Stephen Tobin¹
Caroline Joyce², Carl Parsons³, Margaret Schnitzler⁴, Venessa Tsang⁴ and Nidhi Garg⁴
1 ANZAHPE, ASME, WSU Medicine
2 Western Sydney Univiersity
3 Western Sydney University
4 Sydney University

There is extensive literature about workplace-based assessment (WBA) and entrustable professional activities (EPA), much of which is positive. There is no doubt that WBAs/EPAs ask more of the student or trainee, provide a structure for feedback, help determine actions for better performance (i.e., patient care) and provide a record of longitudinal progress. As multi- competency tasks, they cover much of clinical practice and can be mapped to graduate outcomes. With all this effort and information provided, could or should they be used to replace formal clinical examinations such as Objective Structured Clinical Examinations (OSCE) and structured vivas or long-cases? What are the trade-offs of attempting to increase authenticity through use of WBA? Challenges include, but are not limited to, assessor training and calibration, volume of assessment, and ‘failure to fail’. 

Formal clinical examinations, such as the OSCE, have been used for decades, and until recently accepted as the norm. They certainly provide assessment of learning, drive student behaviours and require significant logistical effort. Yet senior medical students contribute to patient care in clinical settings. Demonstration of performance (entrusted by supervision levels) allows assessment ‘for learning’ and ‘of learning’ around authentic tasks. As a result, there has been a shift from structured clinical assessment formats to a portfolio of assessments completed in the workplace in some medical programs. However, there remains a need to evaluate such programs of assessment, considering advantages and disadvantages when compared to more traditional formats, particularly when the accrued data is used for progression decisions. 

We do not pretend this is clear-cut and encourage you to come along, contribute and enjoy the debate(s). 

4. Take homes 

We will tailor the discussion around attendee discussion points. Possible outcomes to include shared resources, comparison between systems and more. Collaboration through testing these approaches in other contexts 

Molloy E, Lew S, Woodward-Kron R, et al. Medical student clinical placements as sites of learning and contribution. Melbourne: University of Melbourne; 2018. 

Edmiston, N., Hu, W., Tobin, S.et al.“You’re actually part of the team”: a qualitative study of a novel transitional role from medical student to doctor.BMC Med Educ2023; 23: 112-121 

Hobday PM, Borman-Shoap E, Cullen MJ, et al. The Minnesota Method: A Learner-Driven, Entrustable Professional Activity-Based Comprehensive Program of Assessment for Medical Students. Acad.Med. 2021; 96: S50-55 

Josephine Maundu¹
Hayley Croft², Kirstie Galbraith³, Steven Walker³ and Peter Halstead¹
1 Australian Pharmacy Council
2 University of Newcastle
3 Monash University

Pharmacy education in Australia has embraced an outcome-based approach, evident through the integration of accreditation standards and a performance outcomes framework developed by the Australian Pharmacy Council (APC)(1). In response to the accreditation standards, education providers are required to use diverse, contemporary, and evidence-based assessment methods within academic, practical, and work- integrated learning (WIL) environments. This provides assurance that graduates, and prospective pharmacist registration applicants (pharmacy interns) have achieved the performance outcomes necessary to serve and safeguard the public. Addressing the need for comprehensive pharmacy intern assessments, the APC developed and introduced a suite of workplace-based assessment (WBA) tools in partnership with Intern Training Program (ITP) providers(2). These WBA tools support intern learning in the workplace, observation of intern performance and provision of structured feedback. Most training sites, including community and hospital pharmacies, are using WBA tools for the first time in 2023 across Australia. Further, a small-scale pilot of these tools was conducted in the context of WIL for pharmacy students in their later years of their undergraduate degree programs. This work explored the applicability of EPAs for supporting feedback and assessment of professional tasks related to medicine dispensing and patient counselling. 

Why is this topic important?
Competency-based education and the use of WBAs have been well established in medical training and expanding into other healthcare professions, including pharmacy(3). The advantage of WBA is that it enables the assessment of clinical reasoning, decision making, communication skills and professional judgement in real working environment. WBAs have been demonstrated to be a more effective evaluation of competency in comparison to traditional assessments such as written or oral examinations. We will share our learnings from the development and introduction of the WBA tools and describe challenges and enabling factors. 

Take home messages.
This is the first program to introduce and evaluate WBA in the pre-registration training of pharmacy graduates at a national level involving multiple sites in diverse settings in Australia. 

This is also informing strategies for WIL in pharmacy undergraduate programs. These lessons can inform other health professional training programs seeking to incorporate WBA. 

References (maximum three)