Improved FTWeightedHashT Apriori Algorithm for Big Data using Hadoop-MapReduce Model
Sarem M. Ammar *
Departement of IT, Yemen Academic for Graduate Studies, Yemen.
Fadl M. Ba-Alwi
Faculty of Computer & IT, Sana’a University, Yemen.
*Author to whom correspondence should be addressed.
Abstract
The most significant problem of data mining is the frequent itemset mining on big datasets. The best-known basic algorithm for frequent mining itemset is Apriori. Due to the drawbacks of Apriori algorithm, many improvements have been done to make Apriori better, efficient and faster. We have reviewed over 100 papers related to this work that include enhancements be done to improve Apriori algorithm. Weighted based Apriori and Hash Tree based Apriori are the most significant improvements. One of the recent papers integrated the weight concept of weighted Apriori and Hash tree construction concept of Hash Tree Apriori to produce a hybrid Apriori algorithm named WeightedHashT. In this paper, we aim to propose a new approach to improve WeightedHashT Apriori algorithm on big data using Hadoop-MapReduce model by employing the transaction filtering technique. The experiment of this work using different datasets manifests that the proposed algorithm is efficient and effective regarding execution time.
Keywords: Big data, hadoop, mapreduce, apriori, frequent itemset mining.