Reverse-engineering bar charts using neural networks
- PDF / 2,512,556 Bytes
- 17 Pages / 595.276 x 790.866 pts Page_size
- 37 Downloads / 223 Views
R E G UL A R P A P E R
Fangfang Zhou • Yong Zhao • Wenjiang Chen • Yijing Tan • Yaqi Xu • Yi Chen • Chao Liu • Ying Zhao
Reverse-engineering bar charts using neural networks
Received: 29 June 2020 / Revised: 12 August 2020 / Accepted: 19 August 2020 Ó The Visualization Society of Japan 2020
Abstract Reverse-engineering bar charts extract textual and numeric information from the visual representations of bar charts to support application scenarios that require the underlying information. In this paper, we propose a neural network-based method for reverse-engineering bar charts. We adopt a neural network-based object detection model to simultaneously localize and classify textual information. This approach improves the efficiency of textual information extraction. We design an encoder-decoder framework that integrates convolutional and recurrent neural networks to extract numeric information. We further introduce an attention mechanism into the framework to achieve high accuracy and robustness. Synthetic and real-world datasets are used to evaluate the effectiveness of the method. To the best of our knowledge, this work takes the lead in constructing a complete neural network-based method of reverseengineering bar charts. Keywords Information extraction Neural network Reverse engineering Bar chart
1 Introduction Bar charts, which are popular chart types on the Internet (Battle et al. 2018), are commonly used to visually present quantitative information. In most cases, only visual representations of the charts but not the underlying data are available. Extracting the underlying raw data is a common requirement in many application scenarios (Savva et al. 2011; Kong and Agrawala 2012). For example, journalists who are compiling news find some interesting statistics expressed in old-style bar charts and want to use them in their article. Without extracting tools, they must manually extract the raw data to perform chart redesign or further analysis (Chen et al. 2020). For another example, a team of software engineers intends to build a chart search engine (Chen et al. 2015; Siegel et al. 2016). They typically require automated tools to extract the raw data, such as the chart title and axis title, to build accurate indexes.
F. Zhou Y. Zhao W. Chen Y. Tan Y. Xu Y. Zhao (&) School of Computer Science and Engineer, Central South University, Changsha, China E-mail: [email protected] Y. Chen Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University, Beijing, China C. Liu Institute of Systems Engineering, Academy of Military Sciences, Beijing, China Y. Zhao Rail Data Research and Application Key Laboratory of Hunan Province, Changsha, China
F. Zhou et al.
Reverse-engineering bar charts are used to extract chart information. Information in bar charts has two main types: textual and numeric information. Thus, extracting chart information indicates the extraction of textual and numeric information. Textual information annotates the visualization to provide m
Data Loading...
