ROC with Cost Pareto Frontier Feature Selection Using Search Methods
- PDF / 1,571,706 Bytes
- 13 Pages / 595.224 x 790.955 pts Page_size
- 33 Downloads / 190 Views
(2020) 4:6
ORIGINAL ARTICLE
ROC with Cost Pareto Frontier Feature Selection Using Search Methods Ryan Meekins1 · Stephen Adams2
· Kevin Farinholt1 · Sherwood Polter3 · Peter A. Beling2
Received: 12 January 2020 / Revised: 28 July 2020 / Accepted: 25 August 2020 © The Author(s) 2020
Abstract Cyber-physical systems (CPS) are finding increasing application in many domains. CPS are composed of sensors, actuators, a central decision-making unit, and a network connecting all of these components. The design of CPS involves the selection of these hardware and software components, and this design process could be limited by a cost constraint. This study assumes that the central decision-making unit is a binary classifier, and casts the design problem as a feature selection problem for the binary classifier where each feature has an associated cost. Receiver operating characteristic (ROC) curves are a useful tool for comparing and selecting binary classifiers; however, ROC curves only consider the misclassification cost of the classifier and ignore other costs such as the cost of the features. The authors previously proposed a method called ROC Convex Hull with Cost (ROCCHC) that is used to select ROC optimal classifiers when cost is a factor. ROCCHC extends the widely used ROC Convex Hull (ROCCH) method by combining it with the Pareto analysis for cost optimization. This paper proposes using the ROCCHC analysis as the evaluation function for feature selection search methods without requiring an exhaustive search over the feature space. This analysis is performed on 6 real-world data sets, including a diagnostic cyber-physical system for hydraulic actuators. The ROCCHC analysis is demonstrated using sequential forward and backward search. The results are compared with the ROCCH selection method and a popular Pareto selection method that uses classification accuracy and feature cost. Keywords Feature selection · Receiver operating characteristics · Pareto optimality
This article belongs to the Topical Collection: Data-Enabled Discovery for Industrial Cyber-Physical Systems Guest Editor: Raju Gottumukkala Stephen Adams
[email protected] Ryan Meekins [email protected] Kevin Farinholt [email protected] Sherwood Polter [email protected] Peter A. Beling [email protected] 1
Luna Innovations, Charlottesville, VA, USA
2
Department of Engineering Systems and Environment, University of Virginia, Charlottesville, VA, USA
3
Naval Surface Warfare Center Philadelphia Division, Philadelphia, PA, USA
Introduction Cyber-physical systems (CPS) [42] have combined the physical world with advanced computation through sensing and significant advances in artificial intelligence (AI) and machine learning. While CPS are relatively new, the term was first coined in 2006 [36], the potential advantages of these systems have been seen in numerous fields, including manufacturing [19, 21, 36], medicine [27], and transportation [44, 47]. Industrial CPS is a general term applied to any CPS functioning in an industrial setting [
Data Loading...
