Measurement-Based Drift Correction in Spectroscopic Calibration Models

Correct prediction of analyte concentrations from a new spectrum without drift is possible provided the spectrum lies in the row space spanned by the calibration spectra (space-inclusion condition). However, this condition may be violated as on-line spectrometers are compromised by instrumental, process and operational drifts that are not seen during calibration. A space- inclusion condition, which new spectra possibly corrupted with drift should fulfill, is proposed for drift-correction methods. These methods are characterized as either explicit or implicit based on whether or not drift is estimated using on-line reference measurements. A property of the kernel used in explicit methods is proposed based on the space-inclusion condition. The results are illustrated with a simulation study that uses mathematical models for different drift types

Drift Correction in Multivariate Calibration Models Using On-line Reference Measurements

On-line measurements from first-order instruments such as spectrometers may be compromised by instrumental, process and operational drifts that are not seen during off-line calibration. This can render the calibration model unsuitable for prediction of key components such as analyte concentrations. In this work, infrequently available on-line reference measurements of the analytes of interest are used for drift correction. The drift-correction methods that include drift in the calibration set are referred to as implicit correction methods (ICM), while explicit correction methods (ECM) model the drift based on the reference measurements and make the calibration model orthogonal or invariant to the space spanned by the drift. Under some working assumptions such as linearity between the concentrations and the spectra, necessary and sufficient conditions for correct prediction using ICM and ECM are proposed. These so-called space-inclusion conditions can be checked on-line by monitoring the Q-statistic. Hence, violation of these conditions implies the violation of one or more of the working assumptions, which can be used e.g. to infer the need for new reference measurements. These conditions are also valid for rank-deficient calibration data, i.e. when the concentrations of the various species are linearly dependent. A constraint on the kernel used in ECM follows from the space-inclusion condition. This kernel does not estimate the drift itself but leads to an unbiased estimate of the drift space. In a noise-free environment, it is shown that ICM and ECM are equivalent. However, in the presence of noise, a Monte Carlo simulation shows that ECM performs slightly better than ICM. A paired t-test indicates that this difference is statistically significant. When applied to experimental fermentation data, ICM and ECM lead to a significant reduction in prediction error for the concentrations of five metabolites predicted from infrared spectra

Partial least-squares regression with unlabeled data

It is well known that the prediction errors from principal component regression (PCR) and partial least-squares regression (PLSR) can be reduced by using both labeled and unlabeled data for stabilizing the latent subspaces in the calibration step. An approach using Kalman Filtering has been proposed to optimally use unlabeled data with PLSR. In this work, a sequential version of this optimized PLSR as well as two new PLSR models with unlabeled data, namely PCA-based PLSR (PLSR applied to PCA-preprocessed data) and imputation PLSR (iterative procedure to impute the missing labels), are proposed. It is shown analytically and verified with both simulated and real data that the sequential version of the optimized PLSR is equivalent to PCA-based PLSR

On-Line Recalibration of Spectral Measurements Using Metabolite Injections and Dynamic Orthogonal Projection

Spectrometers are enjoying increasing popularity in bioprocess monitoring due to their non-invasiveness and in-situ sterilizability. Their on-line applicability and high measurement frequency create an interesting opportunity for process control and optimization tasks. However, building and maintaining a robust calibration model for the on-line estimation of key variables of interest (e.g., concentrations of selected metabolites) is time-consuming and costly. One of the main drawbacks of using IR spectrometers on-line is that IR spectra are compromised by both long-term drifts and short-term sudden shifts due to instrumental effects or process shifts which might be unseen during calibration. The effect of instrumental drifts can normally be reduced by referencing the measurements against a background solution, but this option is difficult to implement for single-beam instruments due to sterility issues. In this work, in order to maintain the robustness of calibration models for single-beam IR and to increase resistance to process and instrumental drifts/offsets, planned spikes of small amounts of analytes were injected periodically into the monitored medium. The corresponding measured difference spectra were scaled-up and used as reference measurements for updating the calibration model in real-time based on Dynamic Orthogonal Projection (DOP). Applying this technique lead to a noticeable decrease in the standard error of prediction of metabolite concentrations monitored during an anaerobic fermentation of the yeast Saccharomyces cerevisiae

Framework for explicit drift correction in multivariate calibration models

Latent-variable calibrations using principal component regression and partial least-squares regression are often compromised by drift such as systematic disturbances and offsets. This paper presents a two-step framework that facilitates the evaluation and comparison of explicit drift-correction methods. In the first step, the drift subspace is estimated using different types of correction data in a master/slave setting. The correction data are measured for the slave with drift and computed for the master with no drift. In the second step, the original calibration data are corrected for the estimated drift subspace using shrinkage or orthogonal projection. The two cases of no correction and drift correction by orthogonal projection can be seen as special cases of shrinkage. The two-step framework is illustrated with four different experimental data sets. The first three examples study drift correction on one instrument (temperature effects, spectral differences between samples obtained from different plants, instrumental drift), while the fourth example studies calibration transfer between two instruments

Metrics of calibration for probabilistic predictions

Predictions are often probabilities; e.g., a prediction could be for precipitation tomorrow, but with only a 30% chance. Given such probabilistic predictions together with the actual outcomes, "reliability diagrams" help detect and diagnose statistically significant discrepancies -- so-called "miscalibration" -- between the predictions and the outcomes. The canonical reliability diagrams histogram the observed and expected values of the predictions; replacing the hard histogram binning with soft kernel density estimation is another common practice. But, which widths of bins or kernels are best? Plots of the cumulative differences between the observed and expected values largely avoid this question, by displaying miscalibration directly as the slopes of secant lines for the graphs. Slope is easy to perceive with quantitative precision, even when the constant offsets of the secant lines are irrelevant; there is no need to bin or perform kernel density estimation. The existing standard metrics of miscalibration each summarize a reliability diagram as a single scalar statistic. The cumulative plots naturally lead to scalar metrics for the deviation of the graph of cumulative differences away from zero; good calibration corresponds to a horizontal, flat graph which deviates little from zero. The cumulative approach is currently unconventional, yet offers many favorable statistical properties, guaranteed via mathematical theory backed by rigorous proofs and illustrative numerical examples. In particular, metrics based on binning or kernel density estimation unavoidably must trade-off statistical confidence for the ability to resolve variations as a function of the predicted probability or vice versa. Widening the bins or kernels averages away random noise while giving up some resolving power. Narrowing the bins or kernels enhances resolving power while not averaging away as much noise.Comment: 50 pages, 36 figure

Classification of magnetic resonance images from renal perfusion studies of rabbits

Dynamic Magnetic Resonance Imaging (MRI) with contrast media injection is an important tool to study renal perfusion in humans and animals. The goal of this study is to build classifiers for the automatic classification of a kidney as healthy or pathological. A new algorithm is developed that segments out the cortex from the rest of the kidney including the medulla, the renal pelvic, and the background. The performance of two classifier-types (Soft Independent Method of Class Analogy, SIMCA; Partial Least Squares Discriminant Analysis, PLS-DA) is compared for various types of data pre-processing including segmentation, feature extraction, baseline correction, centering, and standard normal variate (SNV)