Research on the classification of lymphoma pathological images based on deep residual neural network

1.Introduction

Malignant lymphoma is a disease in which lymphocytes undergo malignant lesions and form malignant tumors, accounting for about 50% of malignant blood diseases [1, 2]. The etiology of malignant lymphoma is complex and its pathological morphology is diverse [3]. Only in the classification standards of the World Health Organization (WHO), there are nearly 100 kinds of known malignant lymphomas [4], among which the most typical ones were chronic lymphocytic leukemia (CLL), follicular lymphoma (FL), and mantle cell lymphoma (MCL). The pathological images are shown in Fig. 1. In recent years, the morbidity and mortality of malignant lymphoma have been on the rise year by year. According to the analysis report on the prevalence of malignant tumor in China by the National Cancer Center in 2015, the mortality rate of malignant lymphoma in China was 2.39 per 100 000 population, accounting for the 10th death number of all malignant tumors, seriously threatening the physical and mental health and life safety of human beings [5]. It is important to determine the type of lymphoma for clinical treatment and prognosis [6, 7].

Figure 1.

Pathological image samples of three types of lymphoma.

The pathological analysis is the “gold standard” for the accurate diagnosis of lymphoma types [8]. Clinicians generally judge lymphoma types by observing Hematoxylin-Eosin (HE) staining pathological section images [9]. However, this method has three disadvantages. First, there is a shortage of professional doctors who read photos and their diagnostic level is uneven, which increases the working pressure of doctors with more images. Second, the long-term observation images easily cause visual fatigue caused by misdiagnosis and missed diagnosis; Third, the diagnostic conclusion is determined by the subjective consciousness of pathologists, and different doctors may have distinct opinions, leading to inconsistent diagnostic results. Therefore, an automatic classification technology is urgently needed to alleviate the above problems.

With the advent of deep learning technology, the method of using a deep neural network to automatically extract and classify lymphoma image features gradually becomes mature. In this paper, the BP neural network model and based on BP neural network and genetic algorithm (GA-BP) optimization were compared. A ResNet-50 based residual network model was used to classify CLL, FL, and MCL lymphoma types, hoping to provide a reference basis for pathologists to correctly determine lymphoma types.

2.Related work

Realized the automatic classification of lymphoma pathological images has important research value for clinical treatment. However, due to the lack of public data sets and the complex characteristics of pathological images, the research progress of automatic classification technology is very difficult. In recent years, researchers have put forward many solutions for the automatic classification of lymphoma pathological images. In the late 1990s, Ruschenburg et al. [10] used the classification tree method in decision trees to classify low-grade Non-Hodgkin lymphoma (NHL) and myoepithelial sialadenitis. In 2010, Orlov et al. [11] used the global features of pathological images to realize the classification of FL, CLL, MCL through the WND-CHARM classifier. In 2011, Zorman et al. [12] realized the classification of lymphoma pathological images with a symbol-based machine learning method. In 2015, Ting et al. [13] proposed an improved SVM classification algorithm for seven types of lymphoma pathological images, which effectively improved the classification accuracy.

In 2019, Guan et al. [14] proposed the application of Inception-V3 network architecture to classify NHL, squamous cell carcinoma, and other pathological images, the overall classification accuracy reached 89.62%. Brousset et al. [15] used different architectures in the convolution neural network (CNN) such as VGGNet and GoogLeNet to distinguish between FL and diffuse large B cell lymphoma (DLBCL), the results showed that a simple CNN model could be used to assist pathologists in the diagnosis of lymphoma. In 2020, Biaosheng et al. [16] proposed the Faster R-CNN method to obtain a higher accuracy of 96% for the recognition and classification of lymphoma pathological images.

To sum up, many scholars have been carried out a lot of research on lymphoma image classification, which laid a theoretical and practical foundation for the study of malignant lymphoma tissue pathological image classification in this paper. With the development of deep learning in the field of image classification [17, 18], researching on automatic classification technology based on deep learning to assist doctors in diagnosis has important application value.

3.Methods

The traditional image classification technology is to extract image features by human and segment image tissues by threshold method to achieve the purpose of assisting doctors in recognition. This method can improve the accuracy of judgment to a certain extent, but still has the defect of subjective judgment by human. With the development of machine learning, using machine learning techniques to extract the feature dimension reduction and classification by BP neural network output can classification results, giving the doctor diagnosed the opinion, but still exist manually to extract the features of the limitations of the method, by manually extracting image features can’t good screen out the most representative, so the classification accuracy cannot meet the clinical application value. The emergence of the CNN in deep learning solves the above problems. This method can automatically extract image features and input them to the output results of the classifier. Moreover, it has good generalization ability and robustness.

The study of CNN originated in the 1980s to 1990s, and the earliest CNN was the Lenet-5 network [19]. Due to the over-fitting phenomenon, network models such as the AlexNet network [20] and the VGG network [21] were gradually generated. Network-level of deepening enhances the accuracy of the model, but followed by the disappearance of the gradient and explosion model convergence.

In 2015, Hekaiming et al. [22] proposed a new network architecture model-Deep residual neural network model (ResNet), which solves the problem of gradient disappearance. Based on this, resnet-50 was selected as the infrastructure to realize the classification of CLL, FL, and MCL lymphoma. The ResNet-50 network model adopts the residual idea, which can not only optimize the number of network layers but also reduce the number of training parameters without affecting the accuracy, which is conducive to the optimization of the network model. The model have 50 layer includes an input layer, four residual blocks, a full connection layer, among them, the four residual blocks contains 48 convolution layer. To solve the problem of small data sets and prevent overfitting, in this paper, the data enhancement method of image transformation is used to process the image and then input to the ResNet-50 network for training and testing. The algorithm process diagram is shown in Fig. 2.

Figure 2.

Algorithm process diagram.

3.2BP neural network

BP neural network was proposed by a group of scientists led by Rumelhart and McCelland in 1986 [23]. The network consists of three layers of the input layer, hidden layer, and output layer. It is a multi-layer feedforward network trained by an error backpropagation algorithm. The network structure is illustrated in Fig. 3.

Figure 3.

BP neural network structure diagram.

3.4ResNet-50 network

Figure 4.

Residual block structure.

The basic structure of the ResNet-50 network is the residual block unit, and the structure of the residual block is shown in Fig. 4. A basic residual block includes the convolution layer, batch normalization, and ReLU activation function. Assuming that the initial input is xi, the output xi+1 of the unit can be calculated by Eq. (2). F⁢(x) is the residual function of the residual structure.

(2)

xi+1=F⁢(x)+x

In the formula, the calculation results are completed jointly by the quick connection and forward propagation, and the quick connection can be established by skipping one or more layers. If you don’t add a shortcut connection, F⁢(x) will be pushed to 0 with the deepening of the network, which will lead to the dispersion of gradient and the network degradation. However, if there is a quick connection, gradient dissipation can be suppressed to maintain the optimization of the network due to its input characteristics. At the same time, the quick connection can perform simple identity mapping, doesn’t increase additional parameters will not affect the complexity of the calculation, and the whole network can still be trained end-to-end through backpropagation.

3.5Network model design

In this paper, the resnet-50 network architecture is adopted to realize lymphoma pathological image classification. As the network architecture is based on residual blocks, the model can extract deeper features to improve accuracy. In the process of ResNet-50 convolution using the ReLu activation function and the cross-entropy function is used to calculate loss value, between each convolution layer and activation function contains a BN layer, applications of BN layers to speed up network training and convergence speed. The network model structure is shown in Fig. 5, and the structure of each module is shown in Table 1. The value of each network parameter is shown in Table 2.

Table 1

Stage structure

Name	Layers
Stage 1	[1×1,643×3,641×1,256]×3
Stage 2	[1×1,1283×3,1281×1,512]×4
Stage 3	[1×1,2563×3,2561×1,1024]×6
Stage 4*	[1×1,5123×3,5121×1,1048]×3

*The first stage contains 3 three-layer convolutions, the second stage contains 4 three-layer convolutions, the third stage contains 6 three-layer convolutions, and the fourth stage contains 3 three-layer convolutions, each stage contains the same convolution kernel size, and the number of channels is different.

Table 2

Training parameter values

Type	Value/type
Epoch	100
Lr	0.0001
Batch_size	32
Num_classes	3
Loss function	Cross Entropy

*Epoch is the number of training rounds of the data set; Lr is the learning rate; Batch_size is the number of samples selected for one training.

Figure 5.

ResNet-50 network architecture.

4.Experimental results

5.Discussion

CLL, FL, and MCL are common types of malignant lymphoma with highly heterogeneous pathological morphology. It is of great significance to correctly identify the three types of lymphoma in terms of clinical treatment and prognosis. CLL [24] is a clonal proliferation disease of B lymphocytes, which is composed of small lymphocytes similar to normal lymphocytes. FL [25] is derived from B cells in the follicular germinal center, which is characterized by follicular growth of small and large lymphocytes. MCL [26] is a kind of tumor formed by the translocation of the landmark chromosome t(11:14) leading to the overexpression of cyclin that promotes cell proliferation, the tumor cell is composed of small and medium-sized B lymphocytes which are similar to central cells in morphology, and the nucleus usually presents irregular deformation. It can be seen that the histopathological images of the three types of malignant lymphoma show differences in morphological, textural and other characteristics, the deep learning method can be used to realize the classification of lymphoma.

In recent years, deep learning technology has made a lot of progress in lymphoma classification, but there are still have some difficulties hindering the further development of the research. Firstly, lymphoma has few public data sets, and most studies are conducted on private data sets, making it difficult to compare its technologies. Secondly, the magnitude of the data set is small. Compared with traditional classification methods, deep learning needs enough data support. Only when the data set is large enough can it show the technical advantages of deep learning. Thirdly, the pathological morphology of lymphoma is complex and diverse, and there are multiple diseases crossing phenomenon in the actual clinical diagnosis, which increased the difficulty of lymphoma classification. In view of the above problems, it is necessary to improve on the basis of pursuing technology in the future.

With the development of artificial intelligence, more and more powerful deep learning algorithms will be proposed, which can provide strong support for further research on the automatic classification of lymphoma histopathological images.

6.Conclusion

In this paper, image transformation technology was used to preprocess the data set, and deep learning technology was used to realize the classification of lymphoma pathological images under the ResNet-50 network model. Compared with BP neural network and GA-BP neural network, the classification results were significantly improved, which can provide references basis for doctors to determine the type of lymphoma in clinical practice. While this article using ResNet-50 network can achieve better classification accuracy, but it is only on CLL, FL, MCL classification of three types of lymphoma, and it is still difficult to classify the histopathological images of some lymphoma with similar morphology but different treatment regiments and prognosis. Later still need to add other types of lymphoma pathological image classification recognition, to increase the persuasive