Scene text spotting based on end-to-end

Article type: Research Article

Authors: Wei, Guangcun^{a; b; *} | Rong, Wansheng^a | Liang, Yongquan^a | Xiao, Xinguang^a | Liu, Xiang^a

Affiliations: [a] College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, Shandong, China | [b] College of Intelligent Equipment, Shandong University of Science and Technology, Taian, Shandong, China

Correspondence: [*] Corresponding author. Guangcun Wei, E-mail: [email protected].

Abstract: Aiming at the problem that the traditional OCR processing method ignores the inherent connection between the text detection task and the text recognition task, This paper propose a novel end-to-end text spotting framework. The framework includes three parts: shared convolutional feature network, text detector and text recognizer. By sharing convolutional feature network, the text detection network and the text recognition network can be jointly optimized at the same time. On the one hand, it can reduce the computational burden; on the other hand, it can effectively use the inherent connection between text detection and text recognition. This model add the TCM (Text Context Module) on the basis of Mask RCNN, which can effectively solve the negative sample problem in text detection tasks. This paper propose a text recognition model based on the SAM-BiLSTM (spatial attention mechanism with BiLSTM), which can more effectively extract the semantic information between characters. This model significantly surpasses state-of-the-art methods on a number of text detection and text spotting benchmarks, including ICDAR 2015, Total-Text.

Keywords: Scene text spotting, End-to-end, Joint optimization, TCM, SAM-BiLSTM

DOI: 10.3233/JIFS-200903

Journal: Journal of Intelligent & Fuzzy Systems, vol. 40, no. 5, pp. 8871-8881, 2021