A visual big data system for the prediction of weather-related variables: Jordan-Spain case study
- PDF / 3,134,020 Bytes
- 37 Pages / 439.37 x 666.142 pts Page_size
- 66 Downloads / 178 Views
A visual big data system for the prediction of weather-related variables: Jordan-Spain case study Shadi Aljawarneh 1 & Juan A. Lara 2
& Muneer Bani Yassein
1
Received: 7 May 2020 / Revised: 30 July 2020 / Accepted: 9 September 2020 # Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract
The Meteorology is a field where huge amounts of data are generated, mainly collected by sensors at weather stations, where different variables can be measured. Those data have some particularities such as high volume and dimensionality, the frequent existence of missing values in some stations, and the high correlation between collected variables. In this regard, it is crucial to make use of Big Data and Data Mining techniques to deal with those data and extract useful knowledge from them that can be used, for instance, to predict weather phenomena. In this paper, we propose a visual big data system that is designed to deal with high amounts of weather-related data and lets the user analyze those data to perform predictive tasks over the considered variables (temperature and rainfall). The proposed system collects open data and loads them onto a local NoSQL database fusing them at different levels of temporal and spatial aggregation in order to perform a predictive analysis using univariate and multivariate approaches as well as forecasting based on training data from neighbor stations in cases with high rates of missing values. The system has been assessed in terms of usability and predictive performance, obtaining an overall normalized mean squared error value of 0.00013, and an overall directional symmetry value of nearly 0.84. Our system has been rated positively by a group of experts in the area (all aspects of the system except graphic desing were rated 3 or above in a 1–5 scale). The promising preliminary results obtained demonstrate the validity of our system and invite us to keep working on this area. Keywords Big data . Weather forecasting . Data mining . Information fusion . MongoDB
* Juan A. Lara [email protected]
1
JUST, Faculty of Computer and Information Technology, Jordan University of Science and Technology, P.O.Box 3030, Irbid 22110, Jordan
2
School of Computer Science, Madrid Open University, UDIMA, Carretera de La Coruña, KM. 38,500, Vía de Servicio, n° 15, 28400 Collado Villalba, Madrid, Spain
Multimedia Tools and Applications
1 Introduction Big Data is a term that refers to huge and complex amounts of data that needs the application of computing techniques to be managed. By extension, it also refers to a series of procedures and methods that are used to capture, store, visualize, and analyze those amounts of data in search of interesting and value patterns [28]. Most authors agree that data should meet a series of requirements so that they can be called big data. Those are the so-called three V’s (Volume, Variety, and Velocity). Other authors have more recently added other V’s such as Veracity or Variability, to name a few [13]. Big Data may come from a series of envi
Data Loading...
