Affiliations: School of Control and Computer Engineering, North China Electric Power University, Baoding 071003, Hebei, China
Correspondence:
[*]
Corresponding author: Youchan Zhu, School of Control and Computer Engineering, North China Electric Power University, Baoding 071003, Hebei, China. E-mail: [email protected].
Abstract: The image caption generation algorithm allows computer to understand the picture and generate sentences that comply with grammar rules and picture features. Under the Encoder-Decoder framework, the CNN (Convolutional Neural Networks) model is widely used as an encoder to extract image features and the RNN (Recurrent Neural Networks) model as a decoder to generate the description sentence to solve the problem of image caption generation. The most famous algorithm is the NIC, which used Inception-v3 as the encoder, and the LSTM (Long Short-term Memory) as the decoder. However, there are too many parameters in LSTM, and the quality of generated sentences is not high. In the field of visual features, deepening the network structure can improve the feature extraction ability, but the network will degenerate. Therefore, the NIC algorithm is improved. The Inception-ResNet-v2 network is used as the encoder, and the LSTMP network is introduced as the decoder. Taking BLUE-4, ROUGE, METEOR, and CIDEr as evaluation indicators, MSCOCO and Flickr30k are used as datasets to make comparative test between the NIC and the improved NIC. Experimental results show that the improved NIC algorithm outperforms the NIC algorithm in all four evaluation indicators.
