Affiliations: [a] Department of Computer Engineering and Applications, GLA University, Mathura 281406, India | [b] Department of Computer Engineering, Aligarh Muslim University, Aligarh 202002, India
Correspondence:
[*]
Corresponding author: Shashi Shekhar, Department of Computer Engineering and Applications, GLA University, Mathura 281406, India. E-mail: [email protected].
Abstract: The language used by the users in social media nowadays is Code-mixed text, i.e., mixing of two or more languages. This paper describes the application of the code mixed index in Indian social media texts and comparing the complexity to identify language at word level using Bi-directional Long Short Term Memory model. Social media platforms are now widely used by people to express their opinion and interest. The major contribution of the work is to propose a technique for identifying the language of Hindi-English code-mixed data used in three social media platforms namely, Facebook, Twitter, and WhatsApp. We recommend a deep learning framework based on cBoW and Skip gram model that predicts the origin of the word from language perspective in the sequence based on the specific words that have come before it in the sequence. The context capture module of the system gives better accuracy for word embedding model as compared to character embedding.
Keywords: Language identification, transliteration, character embedding, word embedding, Natural Language Processing, cBoW, skip-gram
