Affiliations: Department of Computer Engineering and Artificial Intelligence, Military Technical College, Egypt
Correspondence:
[*]
Corresponding author. Hossam Elzayady, Department of Computer Engineering, Military Technical College Cairo, Egypt. E-mail: [email protected].
Abstract: The use of hateful language in public debates and forums is becoming more common. However, this might result in antagonism and conflicts among individuals, which is undesirable in an online environment. Countries, businesses, and educational institutions are exerting their greatest efforts to develop effective solutions to manage this issue. In addition, recognizing such content is difficult, particularly in Arabic, due to a variety of challenges and constraints. Long-tailed data distribution is often one of the most significant issues in actual Arabic hate speech datasets. Pre-trained models, such as bidirectional encoder representations from transformers (BERT) and generative pre-trained transformers (GPT), have become more popular in numerous natural language processing (NLP) applications in recent years. We conduct extensive experiments to address data imbalance issues by utilizing oversampling methods and a focal loss function in addition to traditional loss functions. Quasi-recurrent neural networks (QRNN) are employed to fine-tune the cutting-edge transformer-based models, MARBERTv2, MARBERTv1, and ARBERT. In this context, we suggest a new approach using ensemble learning that incorporates best-performing models for both original and oversampled datasets. Experiments proved that our proposed approach achieves superior performance compared to the most advanced methods described in the literature.
Keywords: Text classification, Arabic hate speech, oversampling method, transformers, ensemble learning
