ePoster
100% Page: /
Presentation Description
Edward Li1
Annette Mercer2 and Julie Willems2
1 Pearson VUE
2 Monash University
Annette Mercer2 and Julie Willems2
1 Pearson VUE
2 Monash University
Background:
The UCAT examination was first administered in July 2006 in the United Kingdom and other countries. In 2019 the UCAT exam was used for the UCAT_ANZ examination, which is administered in Australia and New Zealand, as well as other countries, to support applications to undergraduate medical/dental schools in Australia and New Zealand. This study specifically focuses on the cognitive sections of the exam.
The UCAT examination was first administered in July 2006 in the United Kingdom and other countries. In 2019 the UCAT exam was used for the UCAT_ANZ examination, which is administered in Australia and New Zealand, as well as other countries, to support applications to undergraduate medical/dental schools in Australia and New Zealand. This study specifically focuses on the cognitive sections of the exam.
Summary of work:
The cognitive sections were analysed using the Rasch model, with a common-item equating design employed to equate various forms to the base metric. The study examined the exam-level performance, including reliability and dimensionality, and item-level performance, such as item parameter drift and differential item functioning (DIF).
The cognitive sections were analysed using the Rasch model, with a common-item equating design employed to equate various forms to the base metric. The study examined the exam-level performance, including reliability and dimensionality, and item-level performance, such as item parameter drift and differential item functioning (DIF).
Results:
The findings revealed that all four cognitive sections displayed high levels of reliability and unidimensionality, indicating their suitability for measuring cognitive abilities. At the item level, parameters remained stable, with minimal DIF observed across gender groups.
The findings revealed that all four cognitive sections displayed high levels of reliability and unidimensionality, indicating their suitability for measuring cognitive abilities. At the item level, parameters remained stable, with minimal DIF observed across gender groups.
Discussion:
To enhance the inclusivity of the exam, considerable efforts were made to diversify the exam content to encompass a wide range of currencies, names, geographies, and cultures. UCAT_ANZ results indicated a similarity in the functioning of test items and the overall exam compared to UCAT, with ANZ candidate performances demonstrating comparable characteristics to those of UCAT candidates.
To enhance the inclusivity of the exam, considerable efforts were made to diversify the exam content to encompass a wide range of currencies, names, geographies, and cultures. UCAT_ANZ results indicated a similarity in the functioning of test items and the overall exam compared to UCAT, with ANZ candidate performances demonstrating comparable characteristics to those of UCAT candidates.
Conclusion:
It suggests that the cognitive sections of the exam measure equivalent constructs across the two testing populations. As a reliable instrument for candidate selection based on defined constructs, the UCAT_ANZ exhibits fairness and consistency in its assessment approach.
It suggests that the cognitive sections of the exam measure equivalent constructs across the two testing populations. As a reliable instrument for candidate selection based on defined constructs, the UCAT_ANZ exhibits fairness and consistency in its assessment approach.
Take-home messages / implications for further research or practice:
While the exam demonstrates equivalence in measuring cognitive abilities, future research should investigate the constructs' relevance in light of emerging technological advancements, such as the growing use of artificial intelligence (AI) in the medical field.
While the exam demonstrates equivalence in measuring cognitive abilities, future research should investigate the constructs' relevance in light of emerging technological advancements, such as the growing use of artificial intelligence (AI) in the medical field.
References (maximum three)
1. Patterson F, Knight A, Dowell J, Nicholson S, Cousans F, Cleland J. How effective are selection methods in medical education? A systematic review. Medical Education 2016; 50:1: 36 – 60.
2. https://www.ucat.edu.au/