Aisha Mansfield¹
Clare McNally² and Claire Mustchin³
1 University of Melbourne
2 Melbourne Dental School, The University of Melbourne

Background:
Cultural safety (CS), with specific reference to Aboriginal and Torres Strait Islander people, is a key requirement for accreditation of dental programs in Australia. It is taught and assessed across the Melbourne Dental School (MDS). However, we have not been assessing the student’s clinical competence in cultural safety, rather focusing on their awareness and ability to be sensitive to the overall healthcare needs of Aboriginal and Torres Strait Islander communities. 

Summary of work:
In 2023, we invited an experienced Aboriginal dental professional (AM) to jointly write the CS OSCE station with an academic staff member (CM). The case information was jointly discussed, and what constituted CS in the encounter was determined by AM and is also reflected in the literature (1). 

Discussion:
This collaboration led to the creation of an OSCE station that assessed CS and history taking in a dental appointment. The station will be trialled in the final year Bachelor of Oral Health OSCE in September 2023, with AM participating as the simulated patient and co- examiner. This new collaboration will give insight into future work so that it can be developed further and implemented across MDS programs. 

Conclusion:
Collaborative partnerships between Universities and Aboriginal and Torres Strait Islander communities will improve CS practices and lessen the cultural load on individuals. Taking an iterative approach to this work and not rushing this collaboration and process is important. Significant investment is required to ensure the expansion of a sustainable Aboriginal and Torres Strait Islander CS assessment program. 

References (maximum three) 

1. Lin I, Green C, Bessarab D. “Yarn with me”: applying clinical yarning to improve clinician- patient communication in Aboriginal health care. Australian Journal of Primary Health [Internet]. 2016 Dec 15 [cited 2023 Aug 9];22(5):377–82. Available from: https://research.ebsco.com/linkprocessor/plink?id=7bdc015c-36e5-36cb-b4c3-f8976e416993 

Paul Edelblut¹
1 Vantage Labs

Assessment is often viewed as the final pillar of the triad of Curriculum, Instruction and Assessment providing feedback on the extent to which educational objectives have been achieved. When assessment is aligned with curriculum and instruction educators measure student progress accurately, reliably, and in a meaningful manner. Assessments that reflect the content that is covered provide a fair representation of a student's knowledge and skills. This tight alignment of curriculum, instruction and assessment is a ideal state not always achieved. 

Often, when assessments are not well aligned with either or both the curriculum and instruction, there is a tendency to look for the point of failure. All too often this begins with scrutiny of the students. While the students are inextricably connected to this educational lifecycle we have, utilizing an AI technology, identified that often a breakdown in student performance on an assessment is related to a disconnect elsewhere. In working with more than 30% of allopathic and osteopathic schools in the USA we have found a large number of instances where the curriculum is created and codified by one group, the instruction is provided by a second group and the assessments may be created by a third group. With the volume of content, the blend of medicine as both science and art, and humans relatively low reliability we must look first to the alignment of an assessment with the curriculum and instruction. 

In this session we will review how Artificial Intelligence is used by US medical schools to align this triad of instructional elements for student success. We will review specific examples and explain how the AI acts like a reference librarian with unlimited memory to unify, data across assessment and instructional platforms. 

References (maximum three) 

Lynn Lewandowski 

David Squires (2012)Curriculum Alignment Research Suggests That Alignment Can Improve Student Achievement,The Clearing House: A Journal of Educational Strategies, Issues and Ideas,85:4,129-135,DOI: 10.1080/00098655.2012.657723 

Glatthorn, Allan A (Fall 1999) Curriculum alignment revisited. Journal of Curriculum and Supervision ; Alexandria Vol. 15, Iss. 1, (Fall 1999): 26. 

Steketee, C. (2015). Prudentia: A medical school’s solution to curriculum mapping and curriculum management. Journal of University Teaching & Learning Practice, 12(4). https://doi.org/10.53761/1.12.4.9 

. 

Danica Sims¹
Francois Cilliers²
1 University of Oxford
2 University of Cape Town

Background:
Interventions to enhance the learning effects of assessment cannot take place without the assessor. Before assessment practice may be changed, first the how and why of assessor behaviour needs to be understood. This study was interested in using Health Behaviour Theory (HBT) to explore the factors that influence assessment practice in varied low-income ‘Southern’ contexts. 

Summary of work:
Thirty-one clinician-educators responsible for designing and implementing assessment were interviewed in South Africa and Mexico. 

Results:
Interacting personal and contextual factors positively and negatively influencing assessor behaviour were identified. These included perceived vulnerability and consequential magnitude of assessment in/actions (impact appraisal); assessment-related response efficacy, costs and value attached to expected outcomes (response appraisal); attitudes and perceived self-efficacy towards assessment; normative assessment beliefs and motivation to comply (interpersonal factors); assessment-related events and interventions, access to assessment information, products, services and facilities, and institutional assessment culture (physical and organisational environment); and the distal context in which assessment takes place. Assessment intention appeared to be mediated by personal competency and environmental affordances and constraints. 

Discussion:
A model of assessor behaviour was developed. While previous research has typically explored factors in isolation, this study used HBT to describe and explain assessment practice systematically and coherently. These findings add a contextual perspective to understanding assessment practice, while resonating with and extending existing work that is predominantly emanates from high-income ‘Northern’ contexts. Future research will explore the utility of this model for targeted (multi-factorial) and more effective (multi-level) faculty development and institutional change. 

Conclusions:
HBT offers a compelling theoretical framework for understanding, and changing, assessor behaviour.


References (maximum three) 

Cilliers, F., Schuwirth, L., & van der Vleuten, V. (2015). Health behaviour theories: a conceptual lens to explore behaviour change. Researching Medical Education, 141-153. 

Ronald Harden¹
Jeni Harden²
1 Medical Teacher
2 University of Edinburgh Medical School 

There is a need for a greater level of education literacy at all levels in a medical school, with a better understanding of the important role students can play as assessors (Harden & Harden 2024). 

This presentation illustrates how students can make a valuable contribution to assessment beyond that which can be offered by the teacher. This may seem as an inherent contradiction with regard to students as objects of assessment and students engaged with assessment policy and practice. 

Students’ involvement in assessment can enhance the effectiveness and quality of the education programme. They have a unique role to play particularly in two areas – peer assessment and self assessment. 

Peer assessment provides a valid assessment of students’ professionalism and contribution to group work, and serves as a means of encouraging teamwork and collaborative skills. As a student-centred approach it gives students an insight into the curriculum and expected learning outcomes. How the associated challenges can be overcome will be described. 

A student’s role in assessing their own competence is important, with critical self-appraisal a requirement of a medical professional. If we graduate students from our school still dependent on others to tell them when they are adequate, good or excellent, then we have missed the whole point of what education is about. Self-assessment is not an innate skill – it is a habit that needs to be acquired by students. It should be recognized as a cyclical process in which self- assessment judgements are gradually refined. Student self-assessment is linked to the move to “assessment for learning”. 

The question is no longer whether students should be engaged with assessment but how students can have a voice in relation to assessment policy and decisions taken through the lens of a student perspective. 

Harden J & Harden R M (2024) The Changing Role of Medical Students. Elsevier. 

Jemma Skeat¹
Sue Garner¹ and Melissa Wos-Oxley¹
1 Deakin University

Background:
To maximize the benefits of workplace-based assessments (WBAs), students and assessors must comprehend their purpose and significance. Students need to select assessors who provide meaningful feedback, complete a diverse range of assessments, and utilize feedback for improvement (Winstone et al., 2017). Assessors also need to comprehend the purpose and value of the assessment, and provide meaningful feedback to support student learning (Marcotte et al., 2019). However, challenges arise, including inadequate interaction with support materials, reliance on inaccurate peer information, and skepticism towards the assessment process. Often, completed assessments feature generic feedback, diminishing their value. 

Summary of work:
In collaboration with nine student partners funded through the Deakin Students as Partners program, we aimed to enhance student learning from WBAs. This involved creating resources (e.g., guidance and training materials) and supports (e.g., peer-led activities). The process included: 1) a needs analysis; 2) co-designing resources and supports; 3) evaluating these materials with other students and assessors. 

Results:
Working with academic staff (authors), students were supported to identify needs with respect to learning from WBAs. Group discussions yielded themes, and potential resources/supports were suggested. Smaller groups of students each collaborated with an academic staff member to co-design these materials. Drafts were shared with peers and assessors to assess their acceptability, usability, and effectiveness. 

Conclusions:
After several years of using usual university methods - ie, Unit and assessment guides/handbooks, orientation, etc- to attempt to support students through a complex assessment process, this project allowed students to engage with creating their own resources to support learning from WBAs. Student co-design is a well-developed concept, and this approach can be easily integrated to enhance engagement in learning, offering a fresh perspective and an effective strategy for supporting students in navigating intricate assessment processes. 

References (maximum three) 

Marcotte, L., Egan, R., Soleas, E., Dalgarno, N., Norris, M., & Smith, C. (2019). Assessing the quality of feedback to general internal medicine residents in a competency-based environment. Canadian medical education journal, 10(4), e32. 

Winstone, N. E., Nash, R. A., Rowntree, J., & Parker, M. (2017). ‘It'd be useful, but I wouldn't use it’: barriers to university students’ feedback seeking and recipience. Studies in Higher Education, 42(11), 2026-2041. 

Helen Wozniak¹
Justine Gibson², Christy Noble¹, Christine Devine¹, Shari Bowker³ and Anna Kull¹
1 Academy for Medical Education, Medical School, The University of Queensland
2 School of Veterinary Science, The University of Queensland
3 Institute for Teaching and Learning Innovation (ITaLI) The University of Queensland. 

Learning in the workplace is complex (1). Learners must adapt swiftly to the challenging environment, maximize learning opportunities, and seek actionable feedback from various health professionals as they complete their work-based assessments (WBAs). 

Summary of work

A collaborative co-design project at The University of Queensland involving diverse fields (Medicine, Veterinary Science, Learning Analytics, eLearning Design) is reimagining WBAs for workplace learning, harnessing expertise from key stakeholders (assessment leaders, clinical staff, and student partners representing urban, rural, and international cohorts). Co- design, from the human-centred design technology space, involves stakeholders having a ‘seat at the table’ so their collective creativity actively guides design decisions (2). 

Thematic analysis of co-design session recordings, transcripts and co-created artefacts revealed two outcomes: firstly design parameters for creating whole-of-cohort and individual digital dashboards for learners to monitor feedback and longitudinal WBA progress; and secondly a framework for optimising workplace learning considering socio-cultural and contextual factors (3). Strategies for learners and supervisors were identified to enhance WBA experiences prior to and during workplace learning, including options for initiating WBAs, gaining actionable feedback, and guiding learner-centred progress reviews. 

The co-design process integrated varied stakeholder perspectives to identify WBA engagement factors. This collaborative approach foregrounds the role of the learner and supervisor whist being cognisant of the university and workplace culture. It also illuminates how to embed technological options to support learning and ongoing monitoring of progress. 

Involving stakeholders in the co-design process provides an essential lens to ensure that the rhythm and flow of the workplace culture informs strategies for optimizing the learning value of WBAs and utilisation of information from digital dashboards. 

Understanding workplace learning and assessment requires ongoing inclusion of all stakeholder viewpoints from design to implementation in the assessment lifecycle. 

References (maximum three)