Accurate frequency-based lexicon generation for opinion mining

Article type: Research Article

Authors: Keshavarz, Hamidreza | Abadeh, Mohammad Saniee^{; *}

Affiliations: Faculty of Electrical and Computer Engineering, Tarbiat Modares University, Tehran, Iran

Correspondence: [*] Corresponding author. Mohammad Saniee Abadeh, Faculty of Electrical and Computer Engineering, Tarbiat Modares University, Tehran, Iran. Tel./Fax: +98 2182884349; E-mail: [email protected].

Abstract: Sentiment analysis deals with classifying the opinions in text. Twitter is the most popular microblogging platform in social media, with hundreds of millions of tweets posted every day. A considerable number of tweets contain opinions. The goal of this paper is to classify the polarity of the tweets into positive and negative classes using dynamic sentiment lexicons based on frequencies of words in positive and negative classes. We extract five meta-level features incorporating the generated sentiment lexicons and classify the text based on them. We also incorporate some previously known lexicon-based and corpus-based features. The proposed method is assessed on six datasets, and outperforms previous papers on accuracy on four datasets, and on f-measure on three datasets. This method generates sentiment lexicons dynamically. The changes of meanings of words can be captured by the generated lexicons. Our research produces very promising results in sentiment analysis in terms of accuracy and f-measure. The accuracy of our method on four datasets and the f-measure of our method on three datasets are higher than 85%.

Keywords: Sentiment analysis, opinion mining, sentiment lexicons, twitter

DOI: 10.3233/JIFS-16562

Journal: Journal of Intelligent & Fuzzy Systems, vol. 33, no. 4, pp. 2223-2234, 2017