A system based on Hadoop for radar data analysis
- PDF / 3,766,676 Bytes
- 15 Pages / 595.276 x 790.866 pts Page_size
- 24 Downloads / 238 Views
ORIGINAL RESEARCH
A system based on Hadoop for radar data analysis Chi Yang1 · Xiaomin Yang1 · Feng Yang2 Received: 31 March 2018 / Accepted: 11 August 2018 © Springer-Verlag GmbH Germany, part of Springer Nature 2018
Abstract Nowadays, radar technique is widely used in many applications, such as electronic warfare, weather prediction, navigation, and self-driving car. Large amounts of radar data has been generated by the wide use of radar technique. Analyzing radar data has a quite important role in daily life, as well as in military. Finding the frequent sequences in radar data is significant for radar data analysis. However, traditional analysis systems using standalones cannot process big data due to the four features of big data, namely, volume, velocity, variety and value. Many distributed frameworks are promising for processing large scale data sets, such as Hadoop and Spark. Therefore, to deal with the problem of finding frequent sequences from large amounts of radar data, we built a system based on Hadoop and Spark. With the combination of Hadoop and Spark, we can store big data, as well as analyze big radar data more easily. In the proposed system, Hadoop distributed file system offers stable data storage, and Spark offers efficient in-memory calculation. In this paper, a three-node Hadoop–Spark cluster was built to perform the distributed data mining algorithm. Additionally, to make the analysis of radar data accurate, we proposed ideas of preprocessing radar data and post processing mining results. Experimental results show that the system we proposed can analyze the large amounts of radar data efficiently and accurately. Keywords Radar data · Hadoop · Spark · Data mining
1 Introduction Nowadays, radar technology plays an increasingly vital role in modern warfare, generating a large amount of radar data. There are some common parameters of radar signal, such as radio frequency (RF), pulse repletion frequency (PRF), direction of arrival (DOA), pulse width (PW), pulse frequency (PF), pulse repetition interval (PRI), time of arrival (TOA), pulse amplitude (PA) etc (Jiang et al. 2006). Usually, RF, DOA, PW, PA, TOA are called pulse description words (PDW). These five parameters are important for radar signal classification since they could basically represent the characteristics of radar signals. Common patterns exist in radar data of each parameter, finding frequent patterns helps classify radar signals. The problem of finding patterns in radar data is to discover the comlete frequent sequences. That is to * Xiaomin Yang [email protected] 1
College of Electronics and Information Engineering, Sichuan University, Chengdu 610064, Sichuan, China
Science and Technology on Electronic Information Control Laboratory, Chengdu 610036, China
2
say, large volumes of radar data need to be processed. However, radar data sets are becoming larger, leading it increasingly difficult for traditional systems using standalones to process. As the amount of data continues to explode, big data processing is appli
Data Loading...
