Presentation Description
Holly Caretta-Weyer1
1 Stanford University School of Medicine
1 Stanford University School of Medicine
Background:
CBME has emerged as the future of health professions education across the continuum. However, significant barriers exist particularly around programmatic assessment, including the challenges of implementing data-driven summative entrustment decision-making processes. Clinical competency committees (CCCs) are designated to review trainee assessment data and formulate recommendations for progression. However, the data input, group decision-making processes, and output of CCCs remain highly variable.
CBME has emerged as the future of health professions education across the continuum. However, significant barriers exist particularly around programmatic assessment, including the challenges of implementing data-driven summative entrustment decision-making processes. Clinical competency committees (CCCs) are designated to review trainee assessment data and formulate recommendations for progression. However, the data input, group decision-making processes, and output of CCCs remain highly variable.
Summary of Work:
To better understand how CCCs utilize data and formulate summative entrustment decisions, we performed a contribution analysis of current CCC processes and assessment data usage, group decision-making processes, and analysis of the ideal future state such that CCCs may defensibly derive summative entrustment decisions that inform graduation and initial certification decisions.
To better understand how CCCs utilize data and formulate summative entrustment decisions, we performed a contribution analysis of current CCC processes and assessment data usage, group decision-making processes, and analysis of the ideal future state such that CCCs may defensibly derive summative entrustment decisions that inform graduation and initial certification decisions.
Results:
Our contribution analysis created an impact pathway that identified assumptions and risks with CCC outputs, proximal outcomes, and distal outcomes in the assessment data usage and summative entrustment decision-making processes. By observing CCCs, we have learned how different groups grapple with assessment data and make decisions about resident progression. We have additionally explored what assessment gaps must be filled and data visualization required to better support the work of CCCs in making summative entrustment decisions to mitigate some of the risks and assumptions identified.
Our contribution analysis created an impact pathway that identified assumptions and risks with CCC outputs, proximal outcomes, and distal outcomes in the assessment data usage and summative entrustment decision-making processes. By observing CCCs, we have learned how different groups grapple with assessment data and make decisions about resident progression. We have additionally explored what assessment gaps must be filled and data visualization required to better support the work of CCCs in making summative entrustment decisions to mitigate some of the risks and assumptions identified.
Discussion/Conclusions:
By performing a contribution analysis, it is clear that aligning CCC decision-making with assessment data fit for purpose, desired outcomes, learner preparedness, and patient/societal needs is a key next step to mitigate risks in the impact pathway. Only then can the principles of CBME truly be achieved.
By performing a contribution analysis, it is clear that aligning CCC decision-making with assessment data fit for purpose, desired outcomes, learner preparedness, and patient/societal needs is a key next step to mitigate risks in the impact pathway. Only then can the principles of CBME truly be achieved.
Implications:
We intend to elaborate upon the assessment gaps identified in order to further develop programmatic assessment, mitigate the risks identified in the contribution analysis, and continually reevaluate CCC group decision-making using a realist lens to determine how to optimize the process and ensure robust summative entrustment decision-making within competency-based systems.
We intend to elaborate upon the assessment gaps identified in order to further develop programmatic assessment, mitigate the risks identified in the contribution analysis, and continually reevaluate CCC group decision-making using a realist lens to determine how to optimize the process and ensure robust summative entrustment decision-making within competency-based systems.
References (maximum three)
1. Van Melle E, Gruppen L, Holmboe ES, Flynn L, Oandasan I, Frank JR, et al. Using contribution analysis to evaluate competency-based medical education programs: it’s all about rigor in thinking. Acad Med. June 2017;92(6):752-8.