Evaluation of a Concept Mapping Task Using Named Entity Recognition and Normalization in Unstructured Clinical Text
- PDF / 524,762 Bytes
- 16 Pages / 439.37 x 666.142 pts Page_size
- 72 Downloads / 177 Views
Open Access
Evaluation of a Concept Mapping Task Using Named Entity Recognition and Normalization in Unstructured Clinical Text Sapna Trivedi 1 & Roger Gildersleeve 2 Andrew S. Kanter 2 & Afzal Chaudhry 1
& Sandra
Franco 2
&
Received: 9 April 2020 / Revised: 7 September 2020 / Accepted: 1 October 2020 # The Author(s) 2020
Abstract In this pilot study, we explore the feasibility and accuracy of using a query in a commercial natural language processing engine in a named entity recognition and normalization task to extract a wide spectrum of clinical concepts from free text clinical letters. Editorial guidance developed by two independent clinicians was used to annotate sixty anonymized clinic letters to create the gold standard. Concepts were categorized by semantic type, and labels were applied to indicate contextual attributes such as negation. The natural language processing (NLP) engine was Linguamatics I2E version 5.3.1, equipped with an algorithm for contextualizing words and phrases and an ontology of terms from Intelligent Medical Objects to which those tokens were mapped. Performance of the engine was assessed on a training set of the documents using precision, recall, and the F1 score, with subset analysis for semantic type, accurate negation, exact versus partial conceptual matching, and discontinuous text. The engine underwent tuning, and the final performance was determined for a test set. The test set showed an F1 score of 0.81 and 0.84 using strict and relaxed criteria respectively when appropriate negation was not required and 0.75 and 0.77 when it was. F1 scores were higher when concepts were derived from continuous text only. This pilot study showed that a commercially available NLP engine delivered good overall results for identifying a wide spectrum of structured clinical concepts. Such a system holds promise for extracting concepts from free text to populate problem lists or for data mining projects. Keywords Natural language processing . Named entity recognition . Clinical letters . Gold
standard . Text mining . Annotation Electronic supplementary material The online version of this article (https://doi.org/10.1007/s41666-02000079-z) contains supplementary material, which is available to authorized users.
* Sapna Trivedi [email protected] 1
Cambridge Clinical Informatics, NIHR Cambridge Biomedical Research Centre, Cambridge University Hospitals NHS Foundation Trust, Hills Road, Cambridge, England, UK
2
Intelligent Medical Objects (IMO), Rosemont, IL, USA
Journal of Healthcare Informatics Research
1 Introduction The use of electronic health records (EHRs) has transformed patient care by facilitating easier access to organized healthcare data, making service delivery safer and more efficient [1, 2]. The retrieval and analysis of data stored within EHRs have the potential to drive further improvement in patient care. Data derived from structured fields (for example, coded data such as diagnoses, medications, allergies, and lab results) have successfully been used for r
Data Loading...
