David Rojas¹
1 University of Toronto 

Background
The evaluation of the Competence By Design (CBD) training model in Canada so far has focused on the process of implementation. Evidence shows strengths around the creation of Competency Committees, but also challenges associated with the completion of EPA assessments, cultural change, and time and resources required to implement and sustain the model. Currently, there is urgent interest in capturing and understanding the value that the CBD model has offered to postgraduate training (1). 

Summary of work
Building on principles of Developmental, Value-based and Utilization-focused evaluation(2), we designed a 5-year plan to evaluate CBD in Canada, addressing the current evidence gaps. This work will guide the Royal College of Physicians and Surgeons of Canada (RCPSC) evaluation of CBD moving forward. 

Results 
The new evaluation approach focuses on capturing the expected value, added value, and created value (3) generated by the CBD model; moving away from process metrics like readiness and fidelity of implementation. The “value” construct is context-dependent (i.e., value for whom) and is composed of the effort, process, cost, and results experienced by each training institution (3). This evaluation plan is underpinned by a value-based, complexity-driven framework (2). 

Discussion
The new evaluation plan will help guide the CBD evaluation efforts toward better understanding the CBD contributions to the medical education field. This could, in turn, help inform future evolutions of CBD. 

Conclusions
We proposed going from a linear process-focused evaluation to an iterative, value-based, complexity-driven model that would help identify and clarify the contributions of CBD. Understanding the value of CBD will be the priority of evaluation efforts in Canada over the next 5 years. 

Take-home messages 
A complex educational training model, like CBD, requires a complex evaluation approach to be able to identify the added value of the model, as well as the unintended processes and effects. 

References (maximum three) 

1. Van Melle, E., Hall, A. K., Schumacher, D. J., Kinnear, B., Gruppen, L., Thoma, B., Caretta-Weyer, H., Cooke, L. J., & Frank, J. R. (2021). Capturing outcomes of competency-based medical education: The call and the challenge. Medical Teacher, 43(7), 794–800. https://doi.org/10.1080/0142159X.2021.1925640/SUPPL_FILE/IMTE_A_1925640_S M4648.PPTX