Presentation Description
Noa Gazit
Gilad Ben-Gal and Ron Eliashar
Gilad Ben-Gal and Ron Eliashar
Background.
The current selection for surgical training is based on ineffective methods. The use of virtual reality and game-based simulation assessments for selection is promising since they offer many advantages over traditional assessment methods: they provide real-life-like experiences, collect rich data and eliminate the bias associated with human assessment.
The current selection for surgical training is based on ineffective methods. The use of virtual reality and game-based simulation assessments for selection is promising since they offer many advantages over traditional assessment methods: they provide real-life-like experiences, collect rich data and eliminate the bias associated with human assessment.
Summary of Work.
Two gamified virtual reality assessment tests were developed – one for the assessment of technical skills and one for the assessment of cognitive abilities and personality characteristics. The technical skills test is performed on a laparoscopic simulator and includes 10 different gamified tasks. The non-technical skills test is performed on a computer and includes three gamified tasks. Based on these tasks 13 competencies are evaluated. A sample of 150 candidates and 30 expert surgeons took the test to evaluate the feasibility and validity of these tests for selection for surgical training. Participants gave their feedback regarding the tests and item analysis was conducted for each test separately.
Two gamified virtual reality assessment tests were developed – one for the assessment of technical skills and one for the assessment of cognitive abilities and personality characteristics. The technical skills test is performed on a laparoscopic simulator and includes 10 different gamified tasks. The non-technical skills test is performed on a computer and includes three gamified tasks. Based on these tasks 13 competencies are evaluated. A sample of 150 candidates and 30 expert surgeons took the test to evaluate the feasibility and validity of these tests for selection for surgical training. Participants gave their feedback regarding the tests and item analysis was conducted for each test separately.
Summary of Results.
Candidates and expert surgeons considered the tests to be relevant and suitable for selection. Item analysis showed positive psychometric properties of the tests such as reliability (0.83 for the technical skills tests and 0.70 for the non-technical skills test) and discrimination of test items. Items that were too difficult were omitted from the tests and changes in test structure and instructions were implemented according to the participants’ feedback.
Candidates and expert surgeons considered the tests to be relevant and suitable for selection. Item analysis showed positive psychometric properties of the tests such as reliability (0.83 for the technical skills tests and 0.70 for the non-technical skills test) and discrimination of test items. Items that were too difficult were omitted from the tests and changes in test structure and instructions were implemented according to the participants’ feedback.
Discussion and Conclusion.
Initial evidence regarding the validity of these innovative tests for selection for surgical training was obtained. Further evidence (e.g., relations with criterion) is needed to establish the validity of these tasks for selection.
Initial evidence regarding the validity of these innovative tests for selection for surgical training was obtained. Further evidence (e.g., relations with criterion) is needed to establish the validity of these tasks for selection.
Take Home Messages.
Virtual reality and game-based simulation are promising tools that have the potential to improve surgical residents' selection.
Virtual reality and game-based simulation are promising tools that have the potential to improve surgical residents' selection.
References (maximum three)
1. Wanzel KR, Ward M, Reznick RK. Teaching the surgical craft: From selection to certification. Curr Probl Surg. 2002;39(6):583-659. doi:10.1067/MOG.2002.123481
2. Louridas M, Szasz P, Fecso AB, et al. Practice does not always make perfect: Need for selection curricula in modern surgical training. Surg Endosc. 2017;31(9):3718-3727. doi:10.1007/s00464-017-5572-3
3. Bann S, Darzi A. Selection of individuals for training in surgery. Am J Surg. 2005;190(1):98-102. doi:10.1016/j.amjsurg.2005.04.002