Image based agorithm for automatic generation of chinese couplets

Article type: Research Article

Authors: Zhu, Cunxin^{a; b; *} | Huang, Xuhong^{a; b; c} | Chen, Yanyi^{a; b} | Tang, Shengping^{a; b} | Zhao, Nan^{a; b} | Xiao, Weihao^{a; b}

Affiliations: [a] Fujian University of Technology, China Fujian Fuzhou | [b] Fujian Provincial Key Laboratory of Big Data Mining and Applications (Fujian University of Technology), China Fujian Fuzhou | [c] National Demonstration Center for Experimental Electronic Information and Electrical Technology Education (Fujian University of Technology), China Fujian Fuzhou

Correspondence: [*] Corresponding author. Cunxin Zhu, E-mail: [email protected].

Abstract: Chinese couplet is one of the important forms of expression in Chinese and even world literature, with its own unique charm and beauty. In order to meet the needs of users who only need one image to obtain corresponding couplets, realize the function of computer automatically writing couplets with images, and improve the literary expression ability of couplets to images, this paper proposes an image based intelligent generative model of couplets. The model first outputs corresponding descriptions based on image extraction features, and then extracts keywords through an improved hybrid algorithm according to the descriptions. Then, based on the keywords, the Chinese GPT-2 model automatically expands the first line of a couplet, and finally generates the second line of a couplet from the first line of a couplet through the encoding and decoding framework. Through experiments, it has been shown that the generated couplets of the model meet the requirements for image description, and the effectiveness of the model has been confirmed by manual evaluation results.

Keywords: Chinese couplet, image description, keyword extraction, Encoding and decoding framework

DOI: 10.3233/JIFS-231155

Journal: Journal of Intelligent & Fuzzy Systems, vol. 45, no. 3, pp. 5093-5105, 2023