Inter-rater reliability of the assessment of adverse drug reactions in the hospitalised elderly

BACKGROUND The identification and assessment of adverse drug reactions (ADRs) is very challenging especially among the older person. Inter observer reliability of an ADR classification system by different healthcare providers is vital to establish the validity of the reaction. OBJECTIVE To assess the inter-rater reliability of an ADR classification system in hospitalised elderly patients and to investigate the differences in reliability by different professions. METHODS From a cohort of 330 elderly patients, patients who experienced a suspected medication related incident (n=87) were selected. The data were analysed by four healthcare professionals (2 pharmacists and 2 physicians) who independently classified the events into event type, types of adverse drug reactions, severity and preventability after a standardised induction based on previously published criteria. Fleiss' kappa was used to assess the level of agreement between the four raters. The difference in level of agreement between the professions was assessed using the weighted least-squares approach for comparing correlated kappa of Barnhart et al. RESULTS Pharmacists and physicians showed high agreement in the identification and on the type and causality of ADRs. However there was lower (moderate) agreement for the severity (kappa = 0.61) and preventability of ADR (kappa = 0.48). Statistically significant differences were also noted between the professions; pharmacists have higher agreement in the classification of preventability (p=0.03) whereas the physicians pairs showed stronger agreement for classifying severity (p<0.001). CONCLUSION Despite the high agreement in the identification, type and causality of ADRs, physicians and pharmacists have difficulties in classifying preventability and severity in a reliable way. A multi-disciplinary approach would enable each profession to share their expert knowledge in order to facilitate better or safer patient care.