Presentation Description
Richard Arnett1
John Hines2 and Peter Brennan2
1 RCSI University of Medicine & Health Sciences
2 Surgical Royal Colleges of the United Kingdom and in Ireland
John Hines2 and Peter Brennan2
1 RCSI University of Medicine & Health Sciences
2 Surgical Royal Colleges of the United Kingdom and in Ireland
1. Background
Membership of the Royal Colleges of Surgeons (MRCS) is an intercollegiate (hosted by the 4 Surgical Royal Colleges in the UK and Ireland) postgraduate surgical qualification required for entry into higher surgical training. Part A consists of 300 MCQ-type questions and Part B is an OSCE assessment. Part A is offered 3 times a year and attracts over 10,000 candidates each year. It has recently migrated from paper-based to an online assessment hosted in 400+ commercial test centres in 150+ cities around the world.
Membership of the Royal Colleges of Surgeons (MRCS) is an intercollegiate (hosted by the 4 Surgical Royal Colleges in the UK and Ireland) postgraduate surgical qualification required for entry into higher surgical training. Part A consists of 300 MCQ-type questions and Part B is an OSCE assessment. Part A is offered 3 times a year and attracts over 10,000 candidates each year. It has recently migrated from paper-based to an online assessment hosted in 400+ commercial test centres in 150+ cities around the world.
2. Summary of work
Analysis identified a small number of candidate pairs with unusually similar response patterns. This similarity was crudely characterized by evaluating the option choices of candidate pairs namely the ratio of ‘Exact Errors In Common (EEIC) to ‘Differences’ (D). High EEIC/D values were flagged for further investigation.
3. Results
A very small number of candidate pairs have been identified in several diets. High EEIC/D values are not detected in historic paper-based results and do not occur in candidate pairs from different test centres. Flagged pairs are followed up with the relevant test centres to ascertain seating plans and the availability of any additional evidence.
4. Discussion
Due to the circumstantial nature of the evidence, it is important to balance the validity requirements of the awarding body with candidate reputation. Where evidence suggests a particular result might be in question, results are withheld, and candidates are offered an opportunity to resit at the next available opportunity.
5. Conclusions
It is likely that there are isolated incidences of collusion taking place which need to be dealt with.
6. Take-home messages/implications for further research or practice
Even negligible incidences of potential cheating in an assessment programme threaten its validity and require investigation. The underpinning process needs to be robust, transparent & visible.
Even negligible incidences of potential cheating in an assessment programme threaten its validity and require investigation. The underpinning process needs to be robust, transparent & visible.
References (maximum three)
Jennings JS, Harpp DN, Hogan JJ. Crime in the Classroom: Part II. An Update. J Chem Educ. 1996;73(4):349.
Harpp DN. Crime in the Classroom: Conclusions after 27 Years. J Chem Educ. 2018;95(10):1900–1