Design of iMacros-based Data Crawler and the Behavioral Analysis of Facebook Users

Abstract

Obtaining the desired dataset is still a prime challenge faced by researchers while analyzing Online Social Network (OSN) sites. Application Programming Interfaces (APIs) provided by OSN service providers for retrieving data impose several unavoidable restrictions which make it difficult to get a desirable dataset. In this paper, we present an iMacros technology-based data crawler called IMcrawler, capable of collecting every piece of information which is accessible through a browser from the Facebook website within the legal framework which permits access to publicly shared user content on OSNs. The proposed crawler addresses most of the challenges allied with web data extraction approaches and most of the APIs provided by OSN service providers. Two broad sections have been extracted from Facebook user profiles, namely, Personal Information and Wall Activities. The present work is the first attempt towards providing the detailed description of crawler design for the Facebook website.