Louise Shaw¹
Kayley Lyons¹ and Mady Mani²
1 University of Melbourne 

Fast Healthcare Interoperability Resources (FHIR) standard has a crucial role in Australia’s digital health transformation. Australia needs significant upskilling of the health workforce to develop, implement, and maintain FHIR technologies [1]. It was necessary to determine the best choice of training programs for the differing roles of FHIR users, that could be feasibly delivered in one year. 

We conducted a general needs assessment (step 1) to establish training needs, required skills and knowledge, followed by a targeted needs assessment (step 2) to apply knowledge learned [2]. To determine the ideal approach, we gathered evidence aligned with a decision-making framework: 

Several sources of data were collected and analysed according to the framework, including expressions of interest surveys, post course surveys, individual and focus group interviews with key informants, and a market scan of existing training programs. One year after the program launch, we evaluated needs assessment estimates of how many learners would express interest in each proposed course.

We recommended nine FHIR fundamentals courses, which varied based on target audience, use case, length, and delivery medium. One-year follow up of calculated predictions of expressions of interest for each proposed course, was within 10%. Our decision-making framework offers educators and researchers an effective approach to needs assessment with the goal of enhancing the content and effectiveness.

Conducting needs assessments can focus the goals, objectives, educational strategies, and evaluation plan for a novel country-wide training program. Evaluating a needs assessment is rarely done but allows researchers to improve needs assessment approaches.


References (maximum three) 

Australian Digital Health Agency, Australia’s National Digital Health Strategy. 2020. 

2. Thomas, P.A., et al., Curriculum development for medical education: a six-step approach. 2022: JHU press. 

Martin Klasen

 Lina Vogt, Tobias Martin, Andreas Follmann, Michael Czaplik and Sasa Sopka 

Over the past years, telemedicine has become increasingly important in everyday clinical practice. In Germany, however, this is not reflected in the curricula for students of human medicine, who currently receive little or no training in telemedicine. Thus, there is a need for the development of a targeted telemedicine curriculum for medical students. 

Summary of work

Aim of the present study was to assess relevant learning goals for a telemedicine curriculum for medical students in Germany. For this purpose, we collected expert assessments on the topic in a group concept mapping (GCM) study. Briefly, GCM is a mixed methods approach that combines qualitative data collection with quantitative analysis (Jackson & Trochim, 2002; Kane & Trochim, 2007; Rosas & Kane, 2012). GCM allows for a collection of statements and their clustering into thematic fields, as well as for rating them according to their importance and ease of implementation. 

Results 
In total, the expert group identified 47 relevant learning goals for the curriculum, which could be clustered into 6 thematic domains (Basics of telemedicine; Implementation of telemedicine; Legal principles; Communication in telemedicine; Application of technology; Technological principles). The more important a learning goal was perceived, the easier the perceived implementation. 

The identified domains and their learning goals cover theoretical knowledge as well as practical skills. Whereas the content of some domains seems to be universal (e.g., Basics of telemedicine), the content of others may depend more on local circumstances (e.g., Legal principles). 

Conclusions
The study results can form the basis for a telemedicine curriculum for students of human medicine. Congruent ratings for importance and ease of implementation may guide the process by showing domains that should be prioritized. 

References (maximum three) 

Jackson KM, Trochim WMK. Concept Mapping as an Alternative Approach for the Analysis of Open-Ended Survey Responses. Organ Res Methods. 2002;5(4):307-336. doi:10.1177/109442802237114 

Kane M, Trochim WMK. Concept Mapping for Planning and Evaluation. Thousand Oaks: SAGE Publications, Inc.; 2007. https://www.amazon.com/Concept-Mapping-Planning- Evaluation-Research/dp/1412940281. 

Rosas SR, Kane M. Quality and rigor of the concept mapping methodology: A pooled study analysis. Eval Program Plann. 2012;35(2):236-245. doi:10.1016/j.evalprogplan.2011.10.003 

Brian Kwan^1,2
Leslie Oyama^1,2, Christopher Longhurst^1,2, James Murphy^1,2 and Michelle Daniel³
1 University of California San Diego School of Medicine
2 University of California San Diego Health
3 AMEE

Proficiency in procedures is a core competency for graduate medical education, but reporting of procedures often relies on manual logging workflows that are duplicative, inefficient, and generate data with questionable validity and accuracy. Failure to accurately gather trainee procedural completion data slows learner progression, delays procedures, and negatively impacts patient care. 

Summary of Work
We built a novel automated system that extracts procedural completion data for emergency medicine resident (trainee) physicians directly from an electronic health record (EHR) system in common use and uploads this data via an application programming interface (API)-based approach to a learning management system (LMS) used by many training programs for accreditation purposes. We also designed and built dashboards to facilitate data dissemination to stakeholders. 

Results
Over approximately one year, the automated system extracted 7,617 procedures from EHR data compared to 4,291 manually logged procedures in the same timeframe, representing an increase of 78%. The system uploaded procedures to the LMS with a recall/sensitivity of 97.3%, specificity of 100%, and overall accuracy of 99.5%. 

Discussion
Use of our automated system to document procedure completion resulted not only in a substantial increase in the number of procedures logged, but also provided data with greater validity and accuracy. Since data extraction and upload is automatic, this system facilitates frequent evaluation of procedural completion, enabling more timely intervention if learners require additional instruction or experience to advance in training. Limitations of our work include performance at a single site with a single EHR system and LMS. 

Conclusions
Reliance on manual procedure completion workflows results in significant procedure under- reporting compared to using EHR data recorded at the point-of-performance and can negatively impact learner progression and patient care. 

Implications for Further Research
We plan to expand our system into other training programs, use cases, and possibly other institutions. 

References (maximum three) 

1. NEJM Group. Exploring the AGME Core Competencies: Patient Care and Procedural Skills (Part 3 of 7). Published September 8, 2016. Accessed March 17, 2023. https://knowledgeplus.nejm.org/blog/patient-care-procedural-skills 

2. Triola MM, Burk-Rafel J. Precision Medical Education. Acad Med. 2023;98(7):775-781. doi:10.1097/ACM.0000000000005227