Using independent resource allocation strategies to solve conflicts of Hadoop distributed architecture in virtualization
- PDF / 1,763,798 Bytes
- 21 Pages / 595.276 x 790.866 pts Page_size
- 99 Downloads / 209 Views
(0123456789().,-volV) (0123456789().,-volV)
Using independent resource allocation strategies to solve conflicts of Hadoop distributed architecture in virtualization Jin-Bang Hsu1 • Chi-Fang Lin1,2 • Yang-Cheng Chang1,2 • Ren-Hao Pan1,2,3,4,5 Received: 16 March 2019 / Revised: 31 October 2020 / Accepted: 3 November 2020 Ó Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract The strong computing power of Cloud computing enables rapid processing of mass data. Hadoop, the most extensively-distributed architecture among Cloud-computing platforms, uses the MapReduce programming model in a server cluster to separate the applications into many small parts in order to conduct arithmetic processing of big data. Virtualization technology renders the various computer physical resources into abstract virtual resources; thus, computer resources are used more flexibly to simplify management. In practice, Hadoop is frequently combined with virtualization technology to reduce handling costs; however, the virtualized Resource Pooling characteristic results in hardware access conflicts and performance degradation. This study revealed where virtualization degrades read and write capabilities by comparison of the substantialistic architecture and virtualized architecture, and strategies for solving conflicts in virtualization are presented. An optimized virtualized Hadoop architecture was strategically designed; conflicts were gradually solved, and the performance of the virtualized Hadoop was enhanced. The experimental results proved that the strategies presented solved the conflicts while maintaining the fault tolerance, processing performance, and expandability of Hadoop. Keywords Cloud computing Big data Hadoop Virtualization Resource allocation
1 Introduction In recent years, the strong computing power of the Cloud has enabled processing of mass data more quickly than older techniques with inherent bottlenecks [1, 2]. The rapid & Ren-Hao Pan [email protected] Jin-Bang Hsu [email protected] Chi-Fang Lin [email protected] Yang-Cheng Chang [email protected] 1
Department of Computer Science and Engineering, Yuan Ze University, Taoyuan City 32003, Taiwan
2
Innovation Center for Big Data and Digital Convergence, Yuan Ze University, Taoyuan City 32003, Taiwan
3
Department of Information Management, Tunghai University, Taichung 40704, Taiwan
4
Preventive Medicine Center, National Yang-Ming University, Taipei 11221, Taiwan
5
La Vida Tec Co., Ltd, Taichung 43347, Taiwan
development of Cloud computing technology and big data analysis has been of benefit in extensive areas to meet a wide variety of needs. Moreover, information obtained by big data analysis is more accurate, meaning that decisionmakers can create better implementation guidelines, which has numerous advantages [3–5]. For example, in the medical field, patients may receive more appropriate medical care following medical care data analysis, while driver behavior analysis can lead to safer driving habits [6–8]. Hadoop
Data Loading...
