Presentation Description
Marina Sawdon1
John McLachlan2
1 University of York, Hull York Medical School
2 University of Central Lancashire
John McLachlan2
1 University of York, Hull York Medical School
2 University of Central Lancashire
Background
Conjunctive hurdles for OSCEs are a widespread practice in healthcare professions’ assessments(1), avoiding compensation between stations, which may be a challenge to patient safety. In addition, conjunctive approaches can only operate with integer numbers; a rule that candidates should pass 75% of stations is easy to apply when there are 16 stations, but not when there are 15.
Conjunctive hurdles for OSCEs are a widespread practice in healthcare professions’ assessments(1), avoiding compensation between stations, which may be a challenge to patient safety. In addition, conjunctive approaches can only operate with integer numbers; a rule that candidates should pass 75% of stations is easy to apply when there are 16 stations, but not when there are 15.
Summary of work
Permission for this retrospective analysis of anonymised data was granted by the University Ethics Committee. Anonymised candidate total scores from several medical undergraduate OSCEs were analysed. Cut scores were determined using Borderline Regression.
Method
Part 1. The mean number of stations passed for each 5% score band were plotted with scores as the abscissae and average number of stations as the ordinate.
Part 2. A normal distribution ogive was fitted to the data, minimising the differences between data set and curve.
Results
The curves generated in Part 1 are indicative of an ogive of a normal distribution. The mean and standard deviation of this gives evidence of the level of difficulty and the discrimination of each OSCE, providing a good descriptor of the performance of the OSCE as a whole. In addition, any desired conjunctive can now be read off as a score.
Discussion
This normal distribution curve ogive provides a unified description of the difficulty and discrimination of the OSCE circuit as a whole.
Conclusions
This eases concerns about reliability. In addition a single cut score integrating the conjunctive condition and the station scores can be calculated.
This eases concerns about reliability. In addition a single cut score integrating the conjunctive condition and the station scores can be calculated.
Take home message
This avoids the possibility that a candidate might ‘pass’ on the cut score but fail on the conjunctive condition, and allows the same conjunctive condition (e.g. that 75% of stations must be passed) to be employed, no matter how many stations are present in the OSCE circuit.
References (maximum three)
1. Ben-David MF. 2000. AMEE Guide No. 18: Standard setting in student assessment. Med Teach. 22(2):120–130