Neville Chiavaroli¹
Clare Ozolins¹
1 Australian Council for Educational Research 

Background
The importance of validity in assessment is widely accepted, as is the notion of validity as an argument based on evidence (Cook et al., 2015). One of the key forms of evidence for validity arguments is item analysis data, especially for MCQ exams. While many forms of data are available, the most common statistics for written examinations are the item facility/difficulty, discrimination index and distractor analyses (Tavakol & Dennick, 2016). These data are an essential part of quality assurance for any high-stakes examinations and for making defensible decisions based on the results of the examination. While such data is increasingly available through online teaching and testing platforms, their interpretation and application are not always made clear or adequately supported. 

Workshop format and participants
This workshop will explore the nature of these statistics, how they can be calculated using simple software, and most importantly, how they can be interpreted and applied to participants’ own testing contexts to evaluate the quality of examination items and contribute to the validity of results and decisions. In addition, the workshop will also briefly demonstrate other data which can be calculated using more advanced approaches and software and which provide further value from a validity perspective such as mean ability, item characteristic curves, and differential item functioning (Hope et al, 2018). While focussed primarily on the use of item analysis for MCQs, the workshop will also demonstrate how to apply and interpret similar data for short answer questions. 

The workshop will include several practical activities guided by the facilitators, including calculating key statistics with sample data using basic spreadsheet software; group review of sample items with corresponding item analysis data for interpretation practice; and demonstration of further statistical and visual data available with more advanced item analysis software. Participants are encouraged to bring their own laptops and item data for practice purposes, but these are not necessary for attending the workshop. 

This workshop will be useful for health professional educators involved in constructing written examinations and who wish to have a sound understanding of how item analysis can be used to guide item selection and support decisions based on examination results. The workshop is primarily aimed at educators who have limited experience in using and/or generating item analyses. 

Workshop outcomes 

After attending this workshop, participants will be able to: 

References (maximum three)