Automatically estimating the incidence of symptoms recorded in GP free text notes

Koeling, Rob, Tate, A Rosemary and Carroll, John A (2011) Automatically estimating the incidence of symptoms recorded in GP free text notes. In: Proceedings of the first international workshop / Managing interoperability and complexity in health systems (MIXHS 2011). Conference on Information and Knowledge Management . ACM, New York, NY, pp. 43-49. ISBN 9781450309547

Full text not available from this repository.


The UK General Practice Research Database (GPRD) is a valuable source of information for health services research. It contains coded data supplemented by free text (physicians' notes and letters). However, due to the difficulty of extracting useful information and the cost of anonymisation, this text is seldom utilised in epidemiological research. We annotated the records of 344 women in the year prior to a diagnosis of ovarian cancer and developed a method for automatically detecting mentions of symptoms in text. We estimated the incidence of five commonly presenting symptoms using: (1) coded symptoms, (2) codes augmented by symptoms automatically extracted from text, and (3) a 'gold standard' dataset of codes and text tagged by three clinically trained annotators. The estimates of incidence of each symptom increased by at least 40% when coded information was enhanced using the manually tagged free text. Our automatic method extracted a significant proportion of this extra information. Our straightforward approach should be extremely useful for medical researchers who wish to validate studies based on codes, or to accurately assess symptoms, using information that can be automatically extracted from unanonymised free text.

Item Type: Book Section
Keywords: Information extraction, primary care health records, clinical data, epidemiology
Schools and Departments: Brighton and Sussex Medical School > Primary Care and Public Health
Subjects: R Medicine > R Medicine (General) > R864 Medical records
Related URLs:
Depositing User: Rob Koeling
Date Deposited: 06 Feb 2012 19:43
Last Modified: 28 Feb 2014 12:07
📧 Request an update