Affiliations: [a]
School of National Security, People’s Public Security University of China, Beijing, China
| [b]
Institute of Scientific and Technical Information of China, Beijing, China
Correspondence:
[*]
Corresponding author. Dawei Wang, Institute of Scientific and Technical Information of China, Beijing, 100038, China. E-mail: [email protected].
Note: [1] This work was supported by the Scientific Research Startup Fund Project of the PPSUC (No. 2022JKF438) and the Beijing Municipal Social Science Foundation (No. 22GLB225).
Abstract: Image semantic learning techniques are crucial for image understanding and classification. In social networks, image data is widely disseminated thanks to convenient acquisition and intuitive expression characteristics. However, due to the freedom of users to publish information, the image has apparent context dependence and semantic fuzziness, which brings difficulties to image representation learning. Fortunately, social attributes such as hashtags carry rich semantic relations, which can be conducive to understanding the meaning of images. Therefore, this paper proposes a new method named Social Heterogeneous Graph Networks (SHGN) for image semantic learning in social networks. First, a heterogeneous graph is built to expand image semantic relations by social attributes. Then the consistent semantic space is reconstructed through cross-media feature alignment. Finally, an image semantic extended learning network is designed to capture and integrate the social semantics and visual feature, which obtains a rich semantic representation of images from a social context. The experiments demonstrate that SHGN can achieve efficient image representation, and favorably against many baseline algorithms.
Keywords: Social networks image, representation learning, heterogeneous graph, social semantic aggregation
