Evaluation of Frequent Itemset Mining Platforms using Apriori and FP-Growth Algorithm

Abstract

With the overwhelming amount of complex and heterogeneous data pouring from any-where, any-time, and any-device, there is undeniably an era of Big Data. The emergence of the Big Data as a disruptive technology for next generation of intelligent systems, has brought many issues of how to extract and make use of the knowledge obtained from the data within short times, limited budget and under high rates of data generation. Companies are recognizing that big data can be used to make more accurate predictions, and can be used to enhance the business with the help of appropriate association rule mining algorithm. To help these organizations, with which software and algorithm is more appropriate for them depending on their dataset, we compared the most famous three MapReduce based software Hadoop, Spark, Flink on two widely used algorithms Apriori and Fp-Growth on different scales of dataset.