Benchmark for Hyperspectral Unmixing Algorithm Evaluation

2.2.1Sparse Regression

A regression problem is learning a function or model capable of estimating the dependent variables from given observations or features. Sparsity refers to the input and output data being incomplete and not fully populated. In machine learning, sparsity indicates data that includes many zeros or other non-significant values. In turn, sparse regression is a subcategory of regression machine learning algorithms designed to handle non-densely packed data. The same regression algorithms can be used for sparse regression (linear, lasso, ridge, and others), but an additional step is often required to determine the subset of predictors. The problem of regression is learning a model that allows estimating a certain quantity of the dependent variable from several observed variables, known as independent variables. Table 1 shows an overview of algorithm results. More detailed explanations of each algorithm featured in Table 1 are provided below in paragraphs of this subsection.

SUnSAL and Total Variation (SUnSAL-TV) (Iordache et al., 2012) is a variation of the SUnSAL algorithm with an added total variation regularization which spatial information for better spectral unmixing results. The created total variation regularizer accounts for spatial homogeneity because it is very likely that neighbouring pixels will have quite similar abundance fractions of the same endmembers. The total variation regularizer is most commonly used as a denoising algorithm in a bigger processing pipeline. A similar algorithm using total variation minimization to unmix and increase the hyperspectral image’s spectral resolution was suggested in Guo et al. (2009). Still, it uses the N-FINDR algorithm (Zhang et al., 2009) to infer endmembers. The authors provided the signal reconstruction error (SRE) results for the algorithm performance evaluation with values: 12.67dB for USGS (Kokaly et al., 2017) dataset and 14.64dB for the ASTER dataset (NASA, 2004). The given results are lower than some of the other review algorithms, but it is hard to compare because the synthetic data is created differently by different authors.

The Sparse Unmixing by variable Splitting and Augmented Lagrangian (SUnSAL) and Constrained SUnSAL (C-SUnSAL) algorithms (Bioucas-Dias and Figueiredo, 2010) are based on the alternating direction method of multipliers (ADMM) (Gabay and Mercier, 1976). The ADMM algorithm splits a difficult problem into an array of simpler problems. The results provided by the authors are in dB values of the reconstruction signal-to-noise ratio (RSNR) metric, and both algorithms were tested using 50dB of artificial noise. SUnSAL algorithm got RSNR values of 48dB and 23dB for Gaussian and USGS (Kokaly et al., 2017) datasets, while SUnSAL-TV got 47dB and 14.5dB, respectively.

Collaborative Sparse Unmixing by variable Splitting and Augmented Lagrangian (CLSUnSAL) (Iordache et al., 2014) is an elaboration of a previous algorithm SUnSAL introduced in Bioucas-Dias and Figueiredo (2010). The difference between the SUnSAL and collaborative SUnSAL algorithms is that the non-constrained algorithm performs regression on each pixel independently, while the constrained algorithm calculates sparsity for all pixels. Algorithm performance results provided by the authors were in the SRE metric with dB as the unit and an artificial noise level of 40dB SNR. The results were 21dB for 2 endmembers, 14dB for 4 endmembers, and 8.7dB for 6 endmembers.

Spectral–Spatial Weighted Sparse Unmixing (S²WSU) (Zhang et al., 2018) is a hyperspectral unmixing framework that tries to get a sparse solution that is constrained by both spectral and spatial domains at the same time. It implements ADMM for parameter and coefficient optimization purposes. A dataset generated from USGS spectral library (Kokaly et al., 2017) is used to determine the algorithm’s performance and compare it to other popular solutions. Synthetic cubes were used to test the algorithm with SRE as the given metric with results: 20.5dB for the first cube and 19.6dB for the second cube with given SNR of 30dB.

Superpixel-based Reweighted Low-Rank and Total Variation (SUSRLR-TV) (Li et al., 2021) is a sparse unmixing algorithm based on simple linear iterative clustering (SLIC) algorithm to segment the hyperspectral images into homogeneous regions and combining total variation and ADMM algorithm to calculate abundance maps. The algorithm was tested using three synthetic data cubes created with different abundances and endmembers gathered from USGS spectral library and using the Cuprite dataset (NASA, 2015). A set of different datasets were used to determine the algorithm performance. The synthetic datasets were used to get accurate metrics, and the Cuprite dataset was used to inspect the algorithm’s performance visually.

Spectral-Spatial-Weighted Multiview Collaborative Sparse Unmixing (MCSU) (Qi et al., 2020) is a sparse regression hyperspectral unmixing algorithm based on spatial and spectral correlation. The main algorithm idea is to use the existing correlations between adjacent spectral bands and neighbouring pixels that are assumed to have highly correlated information. A hyperspectral camera may have captured the same mixture of materials in multiple groups of pixels or a transitioning mixture of materials which will have a very strong correlation to its neighbouring pixels. The authors provide the SRE, and root mean squared error (RMSE) metrics for algorithms performance evaluation. Three different datasets were used with the following results: SRE=33dB for the simulated dataset, RMSE=0.057 for Cuprite dataset (NASA, 2015), and SRE=13.55dB for Jasper Ridge (Zhu et al., 2014b) dataset.

Spectral Variability Augmented Sparse Unmixing model (SVASU) (Zhang et al., 2022) is a model that takes into account the spectral variability of the same endmember spectra. Principal component analysis (PCA) decomposition calculates a spectral variability library. The algorithm adopts a two-stage decomposition of hyperspectral images: first is the decomposition into endmember and abundance matrices, and the second is the reconstruction error of the pixels from the first stage, taking into account the spectral variability library. Algorithm testing by the authors was conducted on a synthetic dataset generated from USGS spectral library (Kokaly et al., 2017), Jasper Ridge (Zhu et al., 2014b) and Samson datasets (Zhu, 2017).

Superpixel-Based Weighted Collaborative sparse regression and Reweighted Low-Rank Representation Unmixing (SBWCRLRU) (Su et al., 2022) is a hyperspectral unmixing algorithm that utilizes spatial-spectral data and incorporates superpixel segmentation methods. The segmentation algorithm divides the hyperspectral image into homogeneous regions with similar properties. For the superpixel segmentation, the authors adapt the SLIC algorithm, which enables its use for hyperspectral data and not on RGB images. To test the algorithm, authors created a synthetic dataset using USGS library (Kokaly et al., 2017) data and a combination of three different real-world hyperspectral data cubes: Samson (Zhu, 2017), Jasper Ridge (Zhu et al., 2014b), and Cuprite (NASA, 2015) datasets.

2.2.2Conclusions from Sparse Regression Related Works Review

A few key takeaways and conclusions were made from the review of semi-supervised hyperspectral unmixing algorithms and the results provided by the authors of these papers:

2.3.1Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is an algorithm group that, as the name states, factorizes a matrix into two separate matrices with an additional assumption that all matrices have no negative elements. Because hyperspectral data cannot have negative values and, in turn, the endmember and abundance matrices are also not negative, these algorithms are widely used in hyperspectral unmixing. Table 2 summarizes the datasets and metrics used in testing the algorithms reviewed below and the results achieved by the authors of corresponding papers.

The spectra at each hyperspectral image pixel are assumed to be a linear mixture of several endmembers. Therefore, the image Z, which represents the whole hyperspectral image cube consisting of three dimensions, two spatial and one spectral, can be formulated as:

Coupled Nonnegative Matrix Factorization (CNMF) (Yokoya et al., 2012) is an algorithm used to unmix a high spatial resolution multispectral data and a high spectral resolution hyperspectral data together to achieve a hyperspectral and multispectral data fusion. Multispectral data usually has a much smaller amount of separate spectral bands that it gathers while having a higher spatial resolution than hyperspectral data. And in turn, the fusion between both data types is used to increase the spatial resolution of hyperspectral data. The algorithms use a vertex component analysis (VCA) algorithm to calculate the initial endmember matrix from the spectral data and a user-set number of endmembers to find. A peak SNR (PSNR) and spectral angle error (SAE) metrics were used by the authors to determine the performance of the unmixing algorithm. Spectral angle error is used to determine the accuracy of reconstructed spectra by calculating the angle of estimated spectra in λ-dimensional space and comparing it to actual spectra. A smaller angle indicates a more accurate spectral reconstruction. A value of 40dB for the PSNR metric is given for the algorithm performance.

Graph regularized L_1/2-NMF (GLNMF) (Lu et al., 2013) is a hyperspectral unmixing algorithm that takes into consideration the local geometrical structures of hyperspectral image data. To detect the geometrical structure, graph regularization and sparsity constraints are used. A synthetic dataset was created using the endmembers from USGS Spectral library (Kokaly et al., 2017) with the number of endmembers differing from 5 to 10. AVIRIS Cuprite (NASA, 2015) dataset was used to test the accuracy of abundance estimation of different minerals. In total, six experiments were conducted by the authors to test different performance metrics. Spectral angle distance (SAD) metric (Yuhas et al., 1992) results are given by the authors: 0.019 for the synthetic dataset, and 0.155 for Jasper Ridge (Zhu et al., 2014b) dataset with noisy bands, 0.135 without noisy bands.

LIDAR-aided total variation regularized Non-negative Tensor Factorization for hyperspectral unmixing (LIDAR-NTF) (Kaya et al., 2021) proposes using a Digital Surface Model (DSM) that is created using LIDAR data to provide accurate elevation information about the observed scene. The provided DSM data is used in total variation regularization as a spatial constraint, increasing the tensor decomposition accuracy, especially in areas of the hyperspectral image with a significant height difference between neighbouring pixels. A tensor is a multidimensional array equal to a matrix if the tensor dimension is two. The same methods can be used for matrix and tensor decomposition if the dimension is two, while higher-dimension tensors require different algorithms. Five randomly selected materials from USGS library (Kokaly et al., 2017) were selected to create a synthetic dataset. An additional Gaussian noise was added to corrupt the data. An RMSE value was calculated for synthetic images, and the results were: 0.1197 with 20dB noise and 0.1185 with 50dB noise.

Total Variation Regularized Reweighted Sparse NMF (TV-RSNMF) (He et al., 2017) is a blind hyperspectral unmixing algorithm based on nonnegative matrix factorization and is implemented using reweighted sparse regularizer to promote abundance sparsity and a TV regularizer to enhance the spatial information because the nearby pixel is likely to be highly correlated due to similar chemical composition. RMSE values of 0.049 and 0.051 are given for 10dB SNR and 40dB SNR with the synthetic dataset.

Robust Collaborative Nonnegative Matrix Factorization (R-CoNMF) (Li et al., 2016) is an unmixing algorithm that performs three steps of a hyperspectral unmixing chain. The three steps denoted by the authors are as follows:

Spatial Group Sparsity regularized NMF (SGSNMF) (Wang et al., 2017) is a blind unmixing method that incorporates a spatial groups sparsity regularizer constraint that takes into account the pixel location (spatial data) and the fact that abundance matrices are sparse. A simulated dataset was created from USGS library (Kokaly et al., 2017), and a real dataset was used to test the created algorithm. RMSE of 0.02 for 3 endmembers and 0.06 for 15 endmembers are given for the synthetic dataset.

Endmember Constraint Non-negative Tensor Factorization via Total Variation for hyperspectral unmixing (EC-NTF-TV) (Wang et al., 2021) is an algorithm that uses a proposed endmember constraint to mitigate the high correlation between spectral signatures for estimating endmembers and a total variation regularization for exploiting the spatial correlation in calculating the abundance maps. The authors also use an augmented multiplicative algorithm to solve their abundance map objective function. To test the algorithm’s performance, SAD and RMSE metrics were used with synthetically generated data and the Jasper Ridge dataset (Zhu et al., 2014b). For the Jasper Ridge dataset, a mead SAD score was calculated from SAD values for each data class. The mean SAD for Jasper Ridge was 0.1248.

Subspace Clustering constrained sparse NMF (SC-NMF) (Lu et al., 2020) is a spectral unmixing framework that uses subspace clustering with NMF to improve the precision of the unmixing. A coefficient matrix derived from the mentioned subspace clustering algorithm instead of a simple Euclidean distance is used to create a similarity graph. A synthetic hyperspectral image was created from the USGS Spectral library (Kokaly et al., 2017) to test the algorithm. SAD values for algorithm performance are 0.09 for the Indiana dataset and 0.089 for the Cuprite dataset (NASA, 2015).

Correntropy-based Spatial-spectral Robust Unmixing Model (CSsRS) (Li X. et al., 2021) is an unmixing model that uses correntropy-based nonnegative matrix factorization, loss function, and a sparsity penalty. The algorithm is tested using a synthetic dataset created from USGS spectral library (Kokaly et al., 2017) and real datasets: Jasper Ridge (Zhu et al., 2014b), and Urban (Zhu et al., 2014a). The authors provide values of SAD to evaluate the algorithm performance: 0.05 for a synthetic dataset with 3 endmembers, 1.4 for a synthetic dataset with 8 endmembers, 0.084 for the Jasper Ridge dataset, and 0.17 for the Urban dataset.

General Loss-based NMF (GLNMF) (Peng et al., 2022) is a hyperspectral unmixing algorithm that uses a general robust loss function in place of the least-squares loss function. The algorithm is tested using a synthetic dataset created from USGS spectral library (Kokaly et al., 2017), and Jasper Ridge (Zhu et al., 2014b) dataset. Values of RMSE are given, which are: 0.06 for 5 endmembers and 0.06 for 10 endmembers.

Correntropy-based Autoencoder-like Nonnegative Matrix Factorization with Total Variation (CANMF-TV) (Feng et al., 2022) is an unmixing algorithm that adds correntropy-induced metric to create the unmixing model, and total variation regularizer is added to preserve spatial information. Synthetic datasets created from USGS spectral library (Kokaly et al., 2017) and Cuprite (NASA, 2015) datasets were used to test the algorithm. The authors provide values of SAD: 0.13 for a synthetic dataset with 10dB SNR, 0.05 for a synthetic dataset with 40dB SNR and 0.095 for the Cuprite dataset.

Spectral–Spatial Weighted sparse NMF (SSWNMF) (Zhang S. et al., 2022) is a hyperspectral unmixing algorithm that introduces weighting factors in the L1-NMF unmixing model. In addition, spatial and spectral data weighting factors can be included in L1-NMF to enhance the sparsity of the abundance matrix. To test the algorithm, the authors used a combination of synthetic and real data. The synthetic dataset was created using USGS spectral library (Kokaly et al., 2017) data with different amounts of added Gaussian noise, and the real data used was Urban (Zhu et al., 2014a) dataset and Cuprite (NASA, 2015) data.

2.3.2Conclusions from Nonnegative Matrix Factorization Related Works Review

From the conducted review of algorithms using nonnegative matrix factorization for hyperspectral unmixing, a few conclusions were gathered:

2.3.3Autoencoder Networks

Autoencoders are a type of unsupervised learning-based neural network architecture. An artificial neuron bottleneck is created to create an autoencoder network that forces the input data to be compressed into a small number of features, extracting additional nonlinear information from the data. A few different types of encoder networks exist and are used for different purposes:

Convolutional Neural Network AutoEncoder Unmixing (CNNAEU) (Palsson et al., 2021) is a hyperspectral unmixing model based on both autoencoder neural network architecture and usage of convolutional layers. It is based on exploiting the spatial structures of hyperspectral images (HSI) and their spectral information. It is achieved by using the convolutional neural network (CNN) to extract spatial features from the structure of HSI. The authors give values of mean MSE for different datasets used: 0.078 for Samson dataset (Zhu, 2017), 0.056 for Urban (Zhu et al., 2014a) dataset, 0.13 for Houston dataset of Science and Technology, 0.10 for Apex dataset (Schaepman et al., 2015).

Deep Generative Unmixing algorithm (DeepGUn) (Borsoi et al., 2020) is a spectral unmixing algorithm based on Generative models such as generative adversarial networks (GANs) and variational autoencoders (VAEs). According to the authors, their proposed strategy leads to more accurate abundance estimation at a small cost of computation resources. Their proposed autoencoder architecture consists of 3 hidden encoder layers with the rectified linear unit (ReLU) activation functions, 3 hidden decoder layers with ReLU activation functions, and an input and output layer with a sigmoid activation function. The experiment was conducted with 4 synthetically created data cubes from USGS Spectral Library (Kokaly et al., 2017) data, and hyperspectral images called Houston dataset of Science and Technology, Samson (Zhu, 2017) and Jasper Ridge (Zhu et al., 2014b) datasets. The authors provide RMSE values for these datasets: 0.045 for the synthetic dataset, 0.236 for the Houston dataset of Science and Technology, 0.086 for the Samson dataset (Zhu, 2017), 0.11 for the Jasper dataset (Zhu et al., 2014b).

Deep autoencoders with Multitask learning for Bilinear hyperspectral Unmixing (DMBU) (Su et al., 2021) is an unmixing algorithm created using deep autoencoder networks and a multitask learning framework. In the proposed method, authors train two instances of autoencoder networks together by minimizing the errors between encoder reconstructed data and original hyperspectral images. Using multitask learning frameworks, authors create a model to obtain endmembers and abundances and a second model to estimate the bilinear components of hyperspectral data. As a result, a bilinear mixture model is created that can more accurately predict the nonlinear interaction of light scattering. The authors used a variety of synthetic and real datasets to test the algorithm’s accuracy and computation time. For the Jasper Ridge dataset (Zhu et al., 2014b), an RMSE of 0.247 was achieved, and a Mean SAD over all of the different classes was 0.150.

Deep Half-Siamese Network (Deep HSNet) (Dong et al., 2020) is a hyperspectral unmixing algorithm that consists of two different networks: endmember guided network and reconstruction network. The first network maps extracted endmembers to the abundances, while the reconstruction network is an autoencoder architecture network that recreates hyperspectral pixels. Two different parameter networks were used in the experimentation with a synthetic dataset created using USGS spectral library (Kokaly et al., 2017) and Urban dataset (Zhu et al., 2014a). Values of reconstruction RMSE are given: 0.12 for a synthetic dataset with 40dB SNR and 0.087 for the Urban dataset.

LSTM-DNN based autoencoder network for nonlinear hyperspectral image unmixing (LSTM-DNN) (Zhao et al., 2021a) is a proposed hyperspectral unmixing algorithm that uses long short-term memory (LSMT) based deep learning network. The authors propose an architecture of a recurrent neural network (RNN) architecture, specifically LSTM layers, and an autoencoder structure to calculate hyperspectral endmembers and abundances using the encoder and reconstruct a hyperspectral data cube using the decoder network. Authors created synthetic datasets using USGS spectral library data using a laboratory-created mixture data, Urban (Zhu et al., 2014a) dataset and other scenes to test the algorithm performance. Multiple metrics were calculated: average spectral angle distance (aSAD), average spectral information divergence (aSID), RMSE, and a few others that were not used in the experiment conducted for the Urban dataset. Results for Urban dataset were: aSAD – 9.2±2.9 , aSID (∗10−3)−115.7±84.7 , RMSE (∗10−3)−13.4±3.4 .

Autoencoder network with Adaptive Abundance Smoothing (AAS) (Hua et al., 2021) is a hyperspectral unmixing algorithm based on an autoencoder network with an adaptive spatial smoothing algorithm to improve the unmixing performance. The synthetic dataset created from USGS spectral library (Kokaly et al., 2017) and Samson (Zhu, 2017), and Jasper Ridge (Zhu et al., 2014b) datasets were used to carry out the algorithm benchmark experiments. For Samson and Jasper datasets, endmember SAD values are given: 0.11 and 0.16, respectively.

Guided encoder-decoder Architecture for hyperspectral Unmixing with Spatial Smoothness (GAUSS) (Ranasinghe et al., 2022) is a three-network hyperspectral unmixing architecture. It consists of the: approximation network, unmixing network, and mixing network, the first two of which are the encoder part of the network, and the last one is the decoder. The authors also propose the pseudo-ground truth mechanism to generate better abundance in the algorithm’s decoder network and other parts. Algorithm testing and experimentation were conducted using USGS spectral library (Kokaly et al., 2017) data and three real hyperspectral datasets: Samson (Zhu, 2017), Jasper Ridge (Zhu et al., 2014b), and Urban (Zhu et al., 2014a).

Sparsity Constrained Convolutional AutoEncoder network for hyperspectral image unmixing (SC-CAE) (Zhao et al., 2021b) is a convolution-based autoencoder network algorithm for hyperspectral unmixing with constrained sparsity. Authors propose an algorithm that uses PCA on hyperspectral data that is then fed into a convolutional autoencoder deep learning network that can find abundance maps and spectral endmembers and reconstruct original hyperspectral data given enough training data and time. A combination of synthetic data generated using spectral information from USGS library (Kokaly et al., 2017) and Jasper Ridge dataset (Zhu et al., 2014b) was used to test the performance proposed algorithm. Mean spectral angle distance (mSAD) and mean abundance angle distance metrics (mAAD) were used for comparing algorithms. For the synthetic dataset, mSAD values were 0.0135 with an SNR of 20dB, 0.0051 with an SNR of 50dB, and mAAD values were 0.0671 with an SNR of 20dB, 0.0306 with an SNR of 50dB.

2.3.4Conclusions from Autoencoder Networks Related Works Review

A few conclusions were derived from the review of algorithms using autoencoder networks to solve the hyperspectral unmixing problems:

2.3.5Linear Mixture Models

Linear mixture models (LMM) are regression model that simultaneously considers the variation of the dependent and the independent variables. The variations of both types of variables are often called fixed and random effects, and because the model uses both of these effects, it is called a mixed model. The linear mixture model is represented in equation (3). In the equation, y is the outcome variable or mixture, X is the predictor multiplied by β regression coefficients, and Z is the design matrix of random effects of mixed data groups. The ε is residuals like noise. An overview of algorithm-acquired results, metrics, and datasets used in experimentation are shown in Table 4.

Augmented Linear Mixing Model (ALMM) (Hong et al., 2019) is a changed linear mixture model that uses an endmember dictionary to determine the scaling factors and an additional dictionary to help model the rest of spectral variabilities. The proposed algorithm also implements an ADMM-based optimization to solve multi-block optimization problems (Xu et al., 2012). In the experiment proposed by the authors, a combination of synthetic data generated from the USGS spectral library (Kokaly et al., 2017) and an AVIRIS gathered a hyperspectral image called Cuprite (NASA, 2015). The results of the reconstruction RMSE given by the authors are 0.0003 for the Cuprite dataset.