Smooth it up!: Extractive summary coherence enhancement

Article type: Research Article

Authors: Akhmetova, Dilyara^{a; b} | Akhmetov, Iskander^{a; b; *} | Pak, Alexander^{a; b} | Gelbukh, Alexander^c

Affiliations: [a] Institute of Information and Computational Technologies, Almaty, Republic of Kazakhstan | [b] Kazakh-British Technical University, Almaty, Republic of Kazakhstan | [c] Instituto Politecnico Nacional, Mexico, Mexico

Correspondence: [*] Corresponding author. Iskander Akhmetov. E-mail: [email protected].

Abstract:  The paper focuses on the importance of coherence and preserving the breadth of content in summaries generated by the extractive text summarization method. The study utilized the dataset containing 16,772 pairs of extractive and corresponding abstractive summaries of scientific papers specifically tailored to increase text coherence. We smoothed the extractive summaries with a Large Language Model (LLM) fine-tuning approach and evaluated our results by applying the coefficient of variation approach. The statistical significance of the results was assessed using the Kolmogorov-Smirnov test and Z-test. We observed an increase in coherence in the predicted texts, highlighting the effectiveness of our proposed methods.

Keywords: Coherence, cohesion, extractive summary, abstractive summary, GPT2, summarization, seq2seq, random forest

DOI: 10.3233/JIFS-219353

Journal: Journal of Intelligent & Fuzzy Systems, vol. Pre-press, no. Pre-press, pp. 1-14, 2024