Affiliations: [a] Faculty of Information Engineering, China University of Geosciences, Wuhan, Hubei, China | [b] School of Computer Science, Wuhan University, Wuhan, Hubei, China | [c] State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, Hubei, China | [d] School of Data Science, University of Science and Technology of China, Anhui, China
Correspondence:
[*]
Corresponding author: Dejun Zhang, Faculty of Information Engineering, China University of Geosciences, Wuhan, Hubei, China. E-mail: [email protected].
Abstract: The geometric and semantic information of 3D point clouds significantly influence the analysis of 3D point cloud structures. However, semantic learning of 3D point clouds based on deep learning is challenging due to the naturally unordered data structure. In this work, we strive to impart machines with the knowledge of 3D object shapes, thereby enabling them to infer the high-level semantic information from the 3D model. Inspired by the vector of locally aggregated descriptors, we propose indirectly describing the high-level semantic information by associating each point’s low-level geometric descriptor with a few visual words. Based on this approach, we design an end-to-end network for 3D shape analysis that combines pointwise low-level geometric and high-level semantic information. The network includes a spatial transform and a uniform operation that make it invariant to input rotation and translation, respectively. Our network also employs pointwise feature extraction and pooling operations to solve the unordered point cloud problem. In a series of experiments with popular 3D shape analysis benchmarks, our network exhibits competitive performance on many important tasks, such as 3D object classification, 3D object part segmentation, semantic segmentation in scenes, and commercial 3D CAD model retrieval.
Keywords: 3D point clouds, convolutional neural network, object classification, semantic segmentation, shape retrieval
