Preprocessing Strategies for Sparse Infrared Spectroscopy: A Case Study on Cartilage Diagnostics

The aim of the study was to optimize preprocessing of sparse infrared spectral data. The sparse data were obtained by reducing broadband Fourier transform infrared attenuated total reflectance spectra of bovine and human cartilage, as well as of simulated spectral data, comprising several thousand spectral variables into datasets comprising only seven spectral variables. Different preprocessing approaches were compared, including simple baseline correction and normalization procedures, and model-based preprocessing, such as multiplicative signal correction (MSC). The optimal preprocessing was selected based on the quality of classification models established by partial least squares discriminant analysis for discriminating healthy and damaged cartilage samples. The best results for the sparse data were obtained by preprocessing using a baseline offset correction at 1800 cm−1, followed by peak normalization at 850 cm−1 and preprocessing by MSC.publishedVersio

En optimert algoritme for separasjon av spredning og kjemisk absorpsjon i biomedisinsk infrarød spektroskopi og avbildning

Infrarødspektroskopi av biologiske prøver har blitt utviklet til et lovende verktøy for ikke-destruktiv biokjemisk analyse gjennom de siste tiårene. Infrarøde absorbansspektre representerer molekylære fingeravtrykk. Enkeltceller og vev forårsaker imidlertid komplekse Mie-spredningsegenskaper i infrarøde absorbansspektre som forurenser de rene kjemiske signaturene. Flere prosesseringsteknikker har blitt foreslått for å håndtere spredning i infrarødspektroskopi. Mie-korreksjon [24, 5, 28, 26] basert på extended multiplicative signal correction (EMSC) [32, 18, 21, 35, 34] betraktes for tiden som det kraftigste verktøyet for å separere Mie-spredning og biokjemisk absorpsjon i infrarøde spektra av celler og vev.Over the past decades, infrared spectroscopy of biological samples has been developed to a promising tool for non-destructive biochemical analysis. Infrared absorbance spectra provide molecular fingerprints. However, single cells and tissues cause complex Mie scattering features in infrared absorbance spectra contaminating the pure chemical signatures. Several preprocessing methods have been proposed to handle scattering in infrared spectroscopy. The Mie correction [24, 5, 28, 26] based on extended multiplicative signal correction (EMSC) [32, 18, 21, 35, 34] is currently considered as the most powerful tool for separating Mie scattering and biochemical absorption in infrared spectra of cells and tissues. Kohler et al. [24] developed an algorithm based on EMSC that could successfully predict Mie scattering features and remove them from infrared absorbance spectra. Bassan et al. developedtheMieEMSCmodelfurthertohandlethesocalleddispersioneffect. The model was implemented in an iterative algorithm, and a compiled program for Mie correction was published [5]. This program is currently the mostly used pre-processing tool for infrared spectra of cells and tissues in the diagnosis of cancer by infrared imaging. However, the algorithm is observed to be strongly biased, since corrected spectra adapt features of the reference spectrum. During recent years, Konevskikh et al. improved the Mie EMSC model further, however a user-friendly program based on the improved algorithm is not yet available [28, 26]. The main aim of this thesis is to further develop the Mie correction algorithm, such that a user-friendly program for Mie correction can be published. This is achieved by proposing a number of improvements to the Mie correction algorithm related to stabilization and optimization. In addition, there is a need for establishing a simulated data set with known pure absorbance spectra and scatter features that mimic measured apparent absorbance spectra, in order to validate different features of the algorithm. The improvements of the Mie EMSC correction algorithm include a number of aspects. The algorithm presented in this thesis sets the number of principal components in the Mie EMSC model automatically by the program, based on a desired level of explained variance in the Mie extinction curves. A flexible stop criterion, based on the convergence of a forward Mie EMSC model is implemented. Further, the initialization parameters are standardized by controlling the scaling of the reference spectrum. Additional stability is gained by weighting the reference spectrum and by setting negative parts of the reference spectrum to zero. A simple quality test for evaluating the correction based on the error of the forward model is implemented, which is used to optimize the initialization parameters. In order to validate the algorithm, a set of absorbance spectra mimicking measured apparent absorbance spectra was simulated. In the simulations, the underlying pure absorbance is known, and scattering features were based on measured spectra. The simulated spectra were used for validation, and to assess critical features of the algorithm. We demonstrate that the correction is not biased by the initial reference spectrum and that a more reliable amide I peak position is retrieved. Sensitivity towards the initialization parameters is further reviewed. It is further demonstrated that the estimated scatter parameters from the EMSC model are meaningful and can be used for clustering of samples with respect to morphological characteristics. The advantage of pre-processing for a subsequent multivariate analysis by chemometrics and machine learning is discussed and suggestions are made how the algorithm can be employed on big spectral data from FTIR imaging. As a result of the proposed improvements, a user-friendly code for correcting highly Mie scatter-distorted absorbance spectra is published at https://bitbucket.org/biospecnorway/mie-emsc-code.M-M

An automated approach for fringe frequency estimation and removal in infrared spectroscopy and hyperspectral imaging of biological samples

In infrared spectroscopy of thin film samples, interference introduces distortions in spectra, commonly referred to as fringes. Fringes may alter absorbance peak ratios, which hampers the spectral analysis. We have previously introduced extended multiplicative signal correction (EMSC) for fringes correction. In the current article, we provide a robust open-source algorithm for fringe correction in infrared spectroscopy and propose several improvements to the Fringe EMSC model. The suggested algorithm achieves a more precise fringe frequency estimation by mean centering of the measured spectrum and applying a window function prior to the Fourier transform. It selects two frequencies from a user defined number of maxima in the Fourier domain. The improved Fringe EMSC algorithm is validated on two experimental datasets, one of them being a hyperspectral image. Techniques for separating sample spectra from background spectra in hyperspectral images, and techniques to identify spectra affected by fringes are also provided.publishedVersio

Deep convolutional neural network recovers pure absorbance spectra from highly scatter‐distorted spectra of cells

Infrared spectroscopy of cells and tissues is prone to Mie scattering distortions, which grossly obscure the relevant chemical signals. The state‐of‐the‐art Mie extinction extended multiplicative signal correction (ME‐EMSC) algorithm is a powerful tool for the recovery of pure absorbance spectra from highly scatter‐distorted spectra. However, the algorithm is computationally expensive and the correction of large infrared imaging datasets requires weeks of computations. In this paper, we present a deep convolutional descattering autoencoder (DSAE) which was trained on a set of ME‐EMSC corrected infrared spectra and which can massively reduce the computation time for scatter correction. Since the raw spectra showed large variability in chemical features, different reference spectra matching the chemical signals of the spectra were used to initialize the ME‐EMSC algorithm, which is beneficial for the quality of the correction and the speed of the algorithm. One DSAE was trained on the spectra, which were corrected with different reference spectra and validated on independent test data. The DSAE outperformed the ME‐EMSC correction in terms of speed, robustness, and noise levels. We confirm that the same chemical information is contained in the DSAE corrected spectra as in the spectra corrected with ME‐EMSC.imag

Preclassification of Broadband and Sparse Infrared Data by Multiplicative Signal Correction Approach

Preclassification of raw infrared spectra has often been neglected in scientific literature. Separating spectra of low spectral quality, due to low signal-to-noise ratio, presence of artifacts, and low analyte presence, is crucial for accurate model development. Furthermore, it is very important for sparse data, where it becomes challenging to visually inspect spectra of different natures. Hence, a preclassification approach to separate infrared spectra for sparse data is needed. In this study, we propose a preclassification approach based on Multiplicative Signal Correction (MSC). The MSC approach was applied on human and the bovine knee cartilage broadband Fourier Transform Infrared (FTIR) spectra and on a sparse data subset comprising of only seven wavelengths. The goal of the preclassification was to separate spectra with analyte-rich signals (i.e., cartilage) from spectra with analyte-poor (and high-matrix) signals (i.e., water). The human datasets 1 and 2 contained 814 and 815 spectra, while the bovine dataset contained 396 spectra. A pure water spectrum was used as a reference spectrum in the MSC approach. A threshold for the root mean square error (RMSE) was used to separate cartilage from water spectra for broadband and the sparse spectral data. Additionally, standard noise-to-ratio and principle component analysis were applied on broadband spectra. The fully automated MSC preclassification approach, using water as reference spectrum, performed as well as the manual visual inspection. Moreover, it enabled not only separation of cartilage from water spectra in broadband spectral datasets, but also in sparse datasets where manual visual inspection cannot be applied

Preclassification of broadband and sparse infrared data by multiplicative signal correction approach

Abstract Preclassification of raw infrared spectra has often been neglected in scientific literature. Separating spectra of low spectral quality, due to low signal-to-noise ratio, presence of artifacts, and low analyte presence, is crucial for accurate model development. Furthermore, it is very important for sparse data, where it becomes challenging to visually inspect spectra of different natures. Hence, a preclassification approach to separate infrared spectra for sparse data is needed. In this study, we propose a preclassification approach based on Multiplicative Signal Correction (MSC). The MSC approach was applied on human and the bovine knee cartilage broadband Fourier Transform Infrared (FTIR) spectra and on a sparse data subset comprising of only seven wavelengths. The goal of the preclassification was to separate spectra with analyte-rich signals (i.e., cartilage) from spectra with analyte-poor (and high-matrix) signals (i.e., water). The human datasets 1 and 2 contained 814 and 815 spectra, while the bovine dataset contained 396 spectra. A pure water spectrum was used as a reference spectrum in the MSC approach. A threshold for the root mean square error (RMSE) was used to separate cartilage from water spectra for broadband and the sparse spectral data. Additionally, standard noise-to-ratio and principle component analysis were applied on broadband spectra. The fully automated MSC preclassification approach, using water as reference spectrum, performed as well as the manual visual inspection. Moreover, it enabled not only separation of cartilage from water spectra in broadband spectral datasets, but also in sparse datasets where manual visual inspection cannot be applied

Preprocessing strategies for sparse infrared spectroscopy:a case study on cartilage diagnostics

Abstract The aim of the study was to optimize preprocessing of sparse infrared spectral data. The sparse data were obtained by reducing broadband Fourier transform infrared attenuated total reflectance spectra of bovine and human cartilage, as well as of simulated spectral data, comprising several thousand spectral variables into datasets comprising only seven spectral variables. Different preprocessing approaches were compared, including simple baseline correction and normalization procedures, and model-based preprocessing, such as multiplicative signal correction (MSC). The optimal preprocessing was selected based on the quality of classification models established by partial least squares discriminant analysis for discriminating healthy and damaged cartilage samples. The best results for the sparse data were obtained by preprocessing using a baseline offset correction at 1800 cm⁻¹, followed by peak normalization at 850 cm⁻¹ and preprocessing by MSC

Deep convolutional neural network recovers pure absorbance spectra from highly scatter‐distorted spectra of cells

Infrared spectroscopy of cells and tissues is prone to Mie scattering distortions, which grossly obscure the relevant chemical signals. The state‐of‐the‐art Mie extinction extended multiplicative signal correction (ME‐EMSC) algorithm is a powerful tool for the recovery of pure absorbance spectra from highly scatter‐distorted spectra. However, the algorithm is computationally expensive and the correction of large infrared imaging datasets requires weeks of computations. In this paper, we present a deep convolutional descattering autoencoder (DSAE) which was trained on a set of ME‐EMSC corrected infrared spectra and which can massively reduce the computation time for scatter correction. Since the raw spectra showed large variability in chemical features, different reference spectra matching the chemical signals of the spectra were used to initialize the ME‐EMSC algorithm, which is beneficial for the quality of the correction and the speed of the algorithm. One DSAE was trained on the spectra, which were corrected with different reference spectra and validated on independent test data. The DSAE outperformed the ME‐EMSC correction in terms of speed, robustness, and noise levels. We confirm that the same chemical information is contained in the DSAE corrected spectra as in the spectra corrected with ME‐EMSC.imag