A similarity measure method fusing deep feature for mammogram retrieval

Article type: Research Article

Authors: Wang, Zhiqiong^{a; f; g} | Xin, Junchang^{b; *} | Huang, Yukun^c | Xu, Ling^a | Ren, Jie^a | Zhang, Hao^{d; *} | Qian, Wei^e | Zhang, Xia^f | Liu, Jiren^f

Affiliations: [a] College of Medicine and Biological Information Engineering, Northeastern University, China | [b] School of Computer Science and Engineering, Key Laboratory of Big Data Management and Analytics (Liaoning), Northeastern University, China | [c] College of Information Science and Engineering, Northeastern University, China | [d] Department of Breast Surgery, Liaoning Cancer Hospital and Institute, Cancer Hospital of China Medical University, China | [e] College of Engineering, University of Texas at El Paso, USA | [f] Neusoft Research of Intelligent Healthcare Technology, Co. Ltd., China | [g] Acoustics Science and Technology Laboratory, Harbin Engineering University, China

Correspondence: [*] Corresponding authors: Hao Zhang, Department of Breast Surgery, Liaoning Cancer Hospital and Institute, Cancer Hospital of China Medical University, China (E-mail: [email protected]) and Junchang Xin, School of Computer Science and Engineering, Key Laboratory of Big Data Management and Analytics (Liaoning), Northeastern University, China (E-mail: [email protected]).

Abstract: BACKGROUND:Breast cancer is one of the most important malignant tumors among women causing a serious impact on women’s lives and mammography is one the most important methods for breast examination. When diagnosing the breast disease, radiologists sometimes may consult some previous diagnosis cases as a reference. But there are many previous cases and it is important to find which cases are the similar cases, which is a big project costing lots of time. Medical image retrieval can provide objective reference information for doctors to diagnose disease. The method of fusing deep features can improve the retrieval accuracy, which solves the “semantic gap” problem caused by only using content features and location features. METHODS:A similarity measure method combining deep feature for mammogram retrieval is proposed in this paper. First, the images are pre-processed to extract the low-level features, including content features and location features. Before extracting location features, registration with the standard image is performed. Then, the Convolutional Neural Network, the Stacked Auto-encoder Network, and the Deep Belief Network are built to extract the deep features, which are regarded as high-level features. Next, content similarity and deep similarity are calculated separately using the Euclidean distance between the query image and the dataset images. The location similarity is obtained by calculating the ratio of intersection to union of the mass regions. Finally, content similarity, location similarity, and deep similarity are fused to form the image fusion similarity. According to the similarity, the specified number of the most similar images can be returned. RESULTS:In the experiment, 740 MLO mammograms are used, which are from women in Northeast China. The content similarity, location similarity, and deep similarity are fused by different weight coefficients. When only considering low-level features, the results are better with fusing 60% content feature similarity and 40% lesion location feature similarity. On this basis, CNN deep similarity, DBN deep similarity, and SAE deep similarity are fused separately. The experiments show that when fusing 60% DBN deep feature similarity and 40% low-level feature similarity, the results have obvious advantages. At this time, the precision is 0.745, recall is 0.850, comprehensive evaluation index is 0.794. CONCLUSIONS:We propose a similarity measure method fusing deep feature, content feature, and location feature. The retrieval results show that the precision and recall of this method have obvious advantage, compared with the content-based image retrieval and location-based image retrieval.

Keywords: Breast cancer, image retrieval, similarity measure, deep feature, mammograms

DOI: 10.3233/XST-190575

Journal: Journal of X-Ray Science and Technology, vol. 28, no. 1, pp. 17-33, 2020