Multiple query optimization approach based on hive+¹

Issue title: Special Section: Recent Advances in Machine Learning and Soft Computing

Guest editors: Srikanta Patnaik

Article type: Research Article

Authors: Han, Ying | Yu, Xiao Qiang^{; *} | Li, Xu | Yao, Chun Long | Zhao, Xin

Affiliations: School of Information Science and Engineering, Dalian Polytechnic University, Dalian, China

Correspondence: [*] Corresponding author. Xiao QiangYu, School of Information Science and Engineering, Dalian Polytechnic University, Dalian 116034, China. E-mail: [email protected].

Note: [1] Fund Program: Dalian Science and Technology Plan Projects (2015A11GX011).

Abstract: To improve the processing efficiency on batch query for MapReduce, a multiple query optimization approach based on Hive+ is proposed to reduce the number of MapReduce tasks on multiple query, decrease the start time of MapReduce task and the overhead of fault tolerance, improve the query efficiency. TPC-H benchmark test set is selected as the use cases to experiment on Hive-0.12. The experiment shows that the processing efficiency of batch query is effectively improved.

Keywords: Hive+, multiple query optimization, inter-query flow analysis, MapReduce

DOI: 10.3233/JIFS-169565

Journal: Journal of Intelligent & Fuzzy Systems, vol. 35, no. 1, pp. 47-53, 2018