A qualitative research framework for the design of user-centered displays of explanations for machine learning model pre
- PDF / 1,824,504 Bytes
- 16 Pages / 595.276 x 790.866 pts Page_size
- 61 Downloads / 127 Views
(2020) 20:257
RESEARCH ARTICLE
Open Access
A qualitative research framework for the design of user-centered displays of explanations for machine learning model predictions in healthcare Amie J. Barda1, Christopher M. Horvat2,3,4,5 and Harry Hochheiser1,6*
Abstract Background: There is an increasing interest in clinical prediction tools that can achieve high prediction accuracy and provide explanations of the factors leading to increased risk of adverse outcomes. However, approaches to explaining complex machine learning (ML) models are rarely informed by end-user needs and user evaluations of model interpretability are lacking in the healthcare domain. We used extended revisions of previously-published theoretical frameworks to propose a framework for the design of user-centered displays of explanations. This new framework served as the basis for qualitative inquiries and design review sessions with critical care nurses and physicians that informed the design of a user-centered explanation display for an ML-based prediction tool. Methods: We used our framework to propose explanation displays for predictions from a pediatric intensive care unit (PICU) in-hospital mortality risk model. Proposed displays were based on a model-agnostic, instance-level explanation approach based on feature influence, as determined by Shapley values. Focus group sessions solicited critical care provider feedback on the proposed displays, which were then revised accordingly. Results: The proposed displays were perceived as useful tools in assessing model predictions. However, specific explanation goals and information needs varied by clinical role and level of predictive modeling knowledge. Providers preferred explanation displays that required less information processing effort and could support the information needs of a variety of users. Providing supporting information to assist in interpretation was seen as critical for fostering provider understanding and acceptance of the predictions and explanations. The user-centered explanation display for the PICU in-hospital mortality risk model incorporated elements from the initial displays along with enhancements suggested by providers. (Continued on next page)
* Correspondence: [email protected] 1 Department of Biomedical Informatics, School of Medicine, University of Pittsburgh, 5607 Baum Boulevard, Pittsburgh, PA 15206, USA 6 Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA 15213, USA Full list of author information is available at the end of the article © The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwi
Data Loading...
