Analysis of Big Data using Apache Spark

Abstract

Data analysis is concerned with the automatic extraction of data related information from variety of sources. Although most analytical model addresses commercial tasks, such as product reviews, there is increasing interest in the affective dimension of the social media websites and various other sources. The current analytical models are not ideally suited for real time analysis of data. This model will collect data from sources of structured and un-structured data; it will filter the relevant data from the raw data in real time or stored data and make it useful for analysis and process it. Hence, such model will be successful in the sense of performing real time analysis than the current ones.

Keywords: Big Data; Large Dataset; Hadoop; HDFS; Spark; SparkSQL; RDD; Python; Java