ANUPONG KANTIWONG 

Background:
Rational drug use (RDU) is the process of appropriate prescribing, dispensing, and patient use of medications[1]. In pharmacology course, third-year medical students are encouraged to create media tool for promote health awareness about RDU concept. The purpose of this study was to determine the reliability of multiple-perspective assessment across different source of raters (instructors, representative students and audiences) by using G-theory. 

Summary of work:
This study enrolled 100 students, and divided into 10 groups to create a short-communication video about common irrational drug use such as ATBs smart use, drug over the counter, non-essential national drug lists, drug administration and cannabinoid abuse. The G-study with a fully crossed one-facet design (GroupXRater) quantifies the amount of variance and G-coefficient. The D-study provides information about the optimal number of raters in different sources needed to obtain reliable measurements. 

Results:
G-study shows the G-coefficients of assessing by single instructor, representative student or audience were 0.75,0.54 and 0.66, respectively and D-study suggested that dependability level can be achieved by using two instructors, five students or three audiences for a one-time assessment (Absolute coefficients = 0.75, 0.78 and 0.85, respectively). 

Discussion and conclusions:
This research demonstrated that all source of raters has high reliability score and suggested that greater improvements in dependability might be made by increasing the number of instructors rather than by increase representative student or audience numbers. Although instructor had the highest D-coefficient and increasing the number is efficient in increasing reliability score, but evaluation from multiple sources will contribute to the diverse perspectives in terms of utilization and feedback. 

Take-home Messages:
G-theory allows the estimation of the magnitude of multiple sources of variance and helps decision makers to determine the number of raters in multiple sources needed to obtain reliable measurements and should be used for limit number of occasions. 

References (maximum three) 

[1] World Health Organization. (2012). The pursuit of responsible use of medicines: sharing and learning from country experiences. World Health Organization.