Concordance as evidence in the Watson for Oncology decision-support system
- PDF / 642,684 Bytes
- 8 Pages / 595.276 x 790.866 pts Page_size
- 100 Downloads / 177 Views
ORIGINAL ARTICLE
Concordance as evidence in the Watson for Oncology decision‑support system Aaro Tupasela1 · Ezio Di Nucci2 Received: 20 September 2019 / Accepted: 15 January 2020 © The Author(s) 2020
Abstract Machine learning platforms have emerged as a new promissory technology that some argue will revolutionize work practices across a broad range of professions, including medical care. During the past few years, IBM has been testing its Watson for Oncology platform at several oncology departments around the world. Published reports, news stories, as well as our own empirical research show that in some cases, the levels of concordance over recommended treatment protocols between the platform and human oncologists have been quite low. Other studies supported by IBM claim concordance rates as high as 96%. We use the Watson for Oncology case to examine the practice of using concordance levels between tumor boards and a machine learning decision-support system as a form of evidence. We address a challenge related to the epistemic authority between oncologists on tumor boards and the Watson Oncology platform by arguing that the use of concordance levels as a form of evidence of quality or trustworthiness is problematic. Although the platform provides links to the literature from which it draws its conclusion, it obfuscates the scoring criteria that it uses to value some studies over others. In other words, the platform “black boxes” the values that are coded into its scoring system. Keywords Artificial intelligence · Decision support · Machine learning · Oncology · Watson for Oncology · IBM · Clinical trials
1 Introduction: decision‑support systems in healthcare During the past several years, IBM has been developing, among others, the Watson for Oncology platform (WFO), which is an artificial intelligence cognitive computing system (see IBM 2018). Such systems are more generally called medical decision-support systems. These systems are designed to support doctors in making decisions on which treatment option is best suited for their patients based on the latest medical evidence that is available (He et al. 2019; Char et al. 2018). The system relies on natural language processing and machine learning to provide treatment recommendations.
* Aaro Tupasela [email protected] 1
Faculty of Social Science, University of Helsinki, Unioninkatu 35, 00014 Helsinki, Finland
Centre for Medical Science and Technology Studies, University of Copenhagen, Copenhagen, Denmark
2
Machine learning platforms are not new and have been operating for years within search engines, such as Google, as well as financial markets and many other everyday services (Carlson 2018; Mittelstadt 2016; Sharon 2016; Buchanan 2015). Algorithmic decision-making is becoming increasingly omnipresent in our everyday lives (Zarsky 2015). According to its proponents, one of the benefits of algorithmic decision-making is that decisions become more objective (cf. Lepri et al. 2018; D’Agostino and Durante 2018). Machine learning-based medical de
Data Loading...
