Uncovering Sentiments: A Big Data Analytic Framework for Twitter Data using Unsupervised Learning

Abstract

Now days, Micro blogging websites are producing enormous unstructured data due to web 2.0 technologies. This unstructured data can be used to extract feelings or interest of people. However, there is limitation to extract feelings or interest of people from web 2.0. Therefore, the aim of this paper is to describe fine grain sentiment clustering (into strong positive, positive, neutral, negative and strong negative) study on 1.6 million tweets by applying Big Data Analytic framework using unsupervised machine learning “Kmeans” algorithm. The framework can work both for big data or non-big data environment. The framework consists of two stages. First stage consists of phases to manage and process social media text data to establish a Machine Learning Model (MLM) and work for non-big data environment. While, second stage described Big Data (BD) architecture and data analysis phases that used MLM model of first stage to get results using Big Data Analytics as well as other BD techniques. Study provides percentage, polarities of each sentiment group with web interface of the model to find the sentiment. Results shows that 351965, 208367, 342536, 159075, 538057 tweets are Strong Positive, Negative, Positive, Strong Negative and Neutral respectively.