Remote detection and identification of chemicals in a scene is a challenging problem. We introduce an approach that uses some of the image's pixels to establish the background characteristics while other pixels represent the target for which we seek to identify all chemical species present. This leads to a generalized least squares problem in which we focus on 'subset selection' to identify the chemicals thought to be present. Bayesian model selection allows us to approximate the posterior probability that each chemical in the library is present by adding the posterior probabilities of all the subsets which include the chemical. We present results using realistic simulated data for the case with 1 to 5 chemicals present in each target and compare performance to a hybrid of forward and backward stepwise selection procedure using the F statistic

Burr, Tom

Fry, H. A. (Herbert A.)

McVey, B. D. (Brian D.)

Sander, E. (Eric)

UNT Digital Library

LA-U R- Od - 7m I Approved for public release; distribution is unlimited. Title: Author(s): Submitted to: Chemical Identification Using Bayesian Model Selection Tom Burr Herb Fry Brian McVey Eric Sander Spring Research Conference on Statistics in Industry and Technology University of Michigan, Ann Arbor Michigan May 2002 1 I A 0 LosAlamos N A T I O N A L  L A B O R A T O R Y  Los Alamos National Laboratory, an affirmative actionlequal opportunity employer, is operated by the University of California for the US. Department of Energy under contract W-7405-ENG-36. By acceptance of this article, the publisher recognizes that the U.S. Government retains a nonexclusive, royalty-free license to publish or reproduce the published form of this contribution, or to allow others to do so, for US. Government purposes. Los Alamos National Laboratory requests that the publisher identify this article as work performed under the auspices of the US. Department of Energy. Los Alamos National Laboratory strongly supports academic freedom and a researcher's right to publish; as an institution, however, the Laboratory does not endorse the viewpoint of a publication or guarantee its technical correctness. Form 836 (8/00) Chemical Identification using Bayesian Model Selection Tom Burr, Herb Fry, Brian McVey Los Alamos National Laboratory Eric Sander National Nuclear Security Administration Abstract Remote detection and identification of chemicals in a scene is a challenging problem. We introduce an approach that uses some of the image’s pixels to establish the background characteristics while other pixels represent the target for which we seek to identify all chemical species present. This leads to a generalized least squares problem in which we focus on “subset selection” to identify the chemicals thought to be present. Bayesian model selection allows us to approximate the posterior probability that each chemical in the library is present by adding the posterior probabilities of all the subsets which include the chemical. We present results using realistic simulated data for the case with 1 to 5 chemicals present in each target and compare performance to a hybrid of forward and backward stepwise selection procedure using the F statistic. Introduction We consider infrared hyperspectral image data in which we first locate chemical plumes and then characterize the chemical components of the plume (McVey et. al., 2002). Our focus is chemical identification from a library of tens to hundreds of chemicals using a passive infrared (IR) detector several kilometers above the ground. A typical scene is 1000 x 128 pixels, where the detected signal at each pixel depends on the ground radiance, atmospheric transmission, instrument noise, and whether a chemical plume lies between the ground and the detector. A prior analysis identifies the plume pixels (pixels that have a plume influencing the signal) from among the background pixels (pixels that do not have a plume influencing the signal). Simplifying assumptions lead to a general least squares (GLS) problem for which we choose which subset (up to 3 chemicals from approximately 100 chemicals for example) of chemicals is most likely to be present in the plume. The next section presents a physical model and the assumptions to convert this to a GLS problem. Following sections present model selection approaches including Bayesian model averaging (BMA) and “pick the winner” (PW) using penalized likelihood (the F test is a special case). We present results with simulated data, and include cases where the errors do not have a Gaussian distribution and the predictors have nonnegligible measurement error. We conclude that the false positive and negative rates for BMA are similar to those for PW provided the appropriate penalty is used with PW. Background A hyperspectral detector detects photons emitted in the IR region from the ground. The signal from background pixel i is (11, si b =&i (v j )Lf (v j ) z ( v j )+Ni (v j )  where q ( v j )  is the emissivity at wavelength (or frequency) vj , Lf ( v j )  is the Planck function at ground temperature, z ( v j )  is the atmospheric transmission, and Ni(vj) is the noise. The signal from plume pixel i is si” =a,(vj)[L;(”j)-&i(”j)Lf(vj)]t(“j)+s~ (2), where a,,(vj) is the plume absorption andL;(vi) is the Planck function at plume temperature. The plume has two effects: it emits in the IR region, but it also absorbs the radiation emitted from itself and from the ground. The ci (v j )  terms (emissitivites) depend on the properties of the background. Concrete, asphalt, buildings, grass, dirt, water, and other common background features each have their characteristic emissitivity. Our synthetic background scenes are generated from various mixtures of approximately 100 typical background emissitivites. Our synthetic plumes are generated from a library of approximately 100 chemical species of interest. Typically there are 10,000 to 16,000 pixels in a scene with 1 to a few plumes. The number of pixels in a plume is 10’s to 100’s. For each wavelength vi,  we subtract the mean response 7 (over all pixels), from the responses so that the centered response y has mean 0. We then assume that y - N(O,$), where i is the n- by-n sample covariance matrix, where n is the number of wavelengths, and all of the pixels. Alternatively, i is estimated using only the background (nonplume) pixels, but in practice there is little difference between using all or just the background pixels to estimate E .  Next, the effect of the plume is assumed to be linear and additive in the chemicals present (assuming Beer’s law, and a weak plume so that 1 -e-’ z x ). Therefore, the model for the plume pixels is is estimated using y = X/3 + e,  with e - N(0,E) (3). Equation (3) is effectively solved using GLS in which we multiply both sides of (3) by i-1’2 to convert to ordinary least squares with transformed data. To simplify notation, we will continue to use y as the transformed response and X as the transformed predictor matrix. We have a library of 100’s of chemicals so we cannot usually fit the full model. Instead, we assume that the true model has 1 to 3 chemicals present, so the problem reduces to model selection (actually, to subset selection, which is a special case of model selection). Challenging issues include: (a) the error distribution is not necessarily well approximated by a Gaussian; (b) there are errors in X due to imperfect corrections for atmospheric transmission, and (c) a more realistic model is a mixture of Gaussians, each with different mean to represent a heterogeneous scene with concrete, grass, dirt, and other patches. We will consider issues (a) and (b) in this paper. Model Selection There are M = ( ~ ) + ( ~ ) + ( ~ ) + ( ~ )  models if we allow the null model, all single-chemical models, all paired-chemical models, and all triple-chemical models. We will report results for two cases: (1) P L i b  is too large for an exhaustive search (2)  P L i b  is small enough that an exhaustive over all possible models; and search is possible ’ If exhaustive search is not possible, several stochastic searches could be considered. We report results only for a strategy involving the leaps algorithm and Occam’s window (Raftery et. al. 1997), as implemented in Splus6 (2002). BMA description. Following Raftery et. al. (1997), and Neath and Cavanaugh (1 997), we approximate the Bayesian information criterion (BIC) using where r j  is the number of chemicals in model i, RSSi = 2 ( y j  - x P ) ~  is the the residual sum of squares from model i, and y is the prior probability for rj chemicals. Then the probability of model i is approximated using P ( M i )  oc e chemical j is present is j=1 The probability that M i=l P(Cj )  = I (C j  E M i ) P ( M i )  , where Z(9 equals 1 if its argument is true. We select a tunable threshold, such as T = 0.90 and predict that chemical j is present if P( C j )  exceeds T. Although it is tempting to conclude that the false positive probability is 1-T, we show elsewhere that the false positive probability must be evaluated by considering the distribution of P ( C j )  givenX, p, and C. PW description We use RSSi from model i and the estimated residual variance using the best fitting model in the criterion L, where L . = L  RSS’ + Kq for model i. (George, 2000, - 2  Obest except we use model having smallest Li is the chosen model. The value K = 2 corresponds to the Aikike information criterion (AIC) or Mallow’s C, and K = ln(n) corresponds to the BIC (George, 2000). The usual F test is a special case of a PW method, in which the model having the largest F test (compared to the null model) is the selected model. It is often implemented using a stepwise 2 rather than &Full 1. The 120 100 80 60 40 20 0 0.05 [ 0.5 TOTALS search, but the F statistic can be evaluated for all models if M is not too large. 238,O 86,O 3 17,O 3 16,O Results on simulated scenes. Example l a  Stochastic search using leaps/Occam with n = 300 wavelengths, p = 1 16 chemicals for BMA and stepwise forward regression as implemented in HIP(McVey et al, 2002) for PW. The true model was chemicals (64,69,82, and 83). For 4 different signal-to-noise ratios (SNR), BMA and PW results using F as special case of the PW method. We assume there is no error in X. Figure 1 is a typical example of the results of a matched filter ( bi ) over all frequencies for chemical k on a synthetic image using the HIP software. In this case, the signal strength is defined empirically as the noise equivalent concentration length (NECL), where NECLk = {- average over all pixels of the estimate of the coefficient for chemical k andp is the number of plume pixels. Results are in Table 1 as applied to the “superpixel” which is the average over all plume pixels. 1 p  P i=l (bi - p)2}-”2 where p i s  the I OOOeMDO 0 OOOeMOO 0 20 40 60 80 100 120 Figure 1. Synthetic image with the background emissitivitiess varying randomly across the scene. A plume is visible in the lower portion. BMA PW NECLl I 64,69,82,83 I 64,69,82,83 NECL2 I 64,69,82 I 64,69,82 NECL3 1 64,82 I 64,82 NECL4 I 21.80 I 82 Table 1. The found chemicals for each of 4 NECLs. The true model is chemicals (64,69,82,83). The BMA threshold was 0.90 and the PW threshold was 0.001. Note that BMA and PW give the same results except for NECL4 (worst case). For NECL4, BMA has 4 FNs and 2 FPs, and PW as 3 FNs. Example l b  Same as Example l a  using NECLl, but we scaledy and the Xcolumns to have a variance equal to 1, and added additional error toy and introduced error in X. Results are in Table 2. Table 2. False positive (FP) and false negative (FN) counts for each of 5 ox and cry values for BMA and PW. Overall, the BMA and PW results are very similar. The BMA threshold was 0.90 and the PW threshold was 0.001. Example IC. True model is randomly selected subset of 0 to 4 of the 1 16 chemicals. Results are in Table 3. The BMA threshold was T = 0.99 QX cry BMA PW FP. FN FP. F N  0.02 51,O 0.03 0.3 80,O 0.04 0.4 79.0 86.0 I TOTAL ERRORS I317 I316 Table 3. False positive (FP) and false negative (FN) counts for each of 5 ox and cry values for BMA and PW. Overall, the BMA and PW results are very similar. Note from Table 3 that for the smaller values of ox and o,,, the FP rate is higher for PW. We have not yet thoroughly explored the operating characteristic curves that evaluate FP and FN rates as a function of the decision thresholds. Also, in this case, there is a threshold that the F test (a special case of the PW method) must exceed in order for a chemical to be included in the chosen model. This absolute threshold can be chosen empirically just as the threshold T for BMA. Example 2. Exhaustive search, p~ib=13, n =300. Here we restricted the library to only 13 chemicals so that exhaustive search would be feasible in our Splus code. Elsewhere (McVey et. al. 2002) we report on exhaustive search results using larger libraries with the search executed in C++. We randomly generated artificial X matrices (artificial chemicals) having unit variance, zero mean, and 0.9 or 0.5 as the correlation (all off- diagonal entries ofXTX/n were either 0.9 or 0.5). Results are in Tables 4a and 4b for 2 BMA methods and 2 PW methods. The first BMA method uses only the first term in Eq. 4, which is the most common approximation to the Bayes factor. The second BMA method uses all terms in Eq. 4, which are the same as Neath and Cavaneaugh, 1997 plus the r, ln(27r) term that appears to have been dropped in Neath and Caveneaug by mistake. The threshold T = 0.9 was used to decide whether a chemical was present. The first PW method using K = ln(n) and the second uses K = 1.5 ln(n), where the factor of 1.5 was chosen to reduce the FP rate. Although the results are not shown here, the value K = 2 (corresponds to the AIC) was too small, leading to a large FP rate of approximately 6 times the FP rate of the K = ln(n) = 5.7. It is known that unless the true model size increases with n, the AIC selects models that are too large (George, 2002). negative (FN) counts for each of 5 ox and oy values for BMA and PW for XTWn= 0.5 on the off-diagonal. Overall, there is little difference between BMA using only the first term in Eq. (4) and BMA using all terms in Eq. (4). This contrasts with the results in Neath and Cavanaugh which suggested that the extra terms in Eq. (4) would improve performance. Comparing BMA to PW, because BMA has a lower FP rate, it also had a higher FN rate. However, because the total error rate was much lower for the PW methods, we have little reason to suspect that the BMA performance can be better than the PW performance in any general sense. Also note that the only change between Tables 4a and 4b is the value of the off-diagonal entry in XTX. Because the erformance was much worse in Table 4b with X Xln - 0.5, this shows that the FP and FN rates of BMA cannot be extracted from the threshold value of T = 0.9. Also, because of the very high false negative rate in the X T X h  = 0.5 case, we expect that if we lowered the threshold we could reduce the error rate. However, the false positive rate is already as high or nearly as high as the X T X h  = 0.9 case, so we expect to still do worse in the XTXln = 0.5 case. This is a surprising result that warrants further study. Model Departure Results We simulated data having Gaussian, Uniform, and Gaussian mixture (GM, to mimic the effect of outliers) distributions, each with the same variance. We used a bimodal GM to mimic having a main distributionfi from whch most (more than 90%) of the errors arise and an outlying distributionfi from whch the rest of the Y -  Table 4a. False positive (FP) and false negative (FN) counts for each of 5 ox and cy values for BMA and PW for XTWn= 0.9 on the off-diagonal. errors arise. The GM arises from a combining the two distributions resulting in a probability density off= alfi + (1- a&, where where chose al= 0.9. We used PLib=13, n =300, and the true model had chemicals (1,2,3) present. First we evaluated P(C,), P(C,), . . ., P(C,,) for each of the 13 chemicals and compared these 13 probabilities in the Uniform errors case and in the GM case to the Gaussian errors case. Qualitatively, the conclusion is that there is mild disagreement in Uniform vs Gaussian, and moderate disagreement in GM vs Gaussian. More quantitatively, 65 t-scores (estimated probabilities for 13 chemicals, each with 5 error variances) with many degrees of freedom that should have been approximately N(0,l) distributed had several values exceeding 3 and a few exceeding 4 for Uniform vs G, and many values exceeding 4 for GM vs G. The most relevant effect of model departure is the potential for the FP and FN rates to be impacted. Using the same three error models (Gaussian, Uniform, and GM), and repeating the analysis from Example 1 c, we find that the FP and FN rates are not noticeably impacted with Uniform or GM errors compared to the Gaussian error case. This is tentative good news that suggests performance can be reasonably well evaluated assuming Gaussian errors even if the errors are rather badly non-Gaussian. Summary Overall, the BMA performance (FP and FN rates) is similar to the PW performance. Also, both methods exhibit similar robustness to model departure of the types evaluated. For both BMA and PW, it is important to select good thresholds, which is always possible using simulation. It would be useful to have analytical approximations to complement simulation results. The main source of errors in the predictor matrix Xis a transmission-through-the atmosphere correction step. Current work is aimed at quantifying how errors in transmission corrections propagate into errors in X. This work evaluated the impact of errors in X in a generic way, and simultaneously increased errors in X and y. A more systematic exploration of the impact of error sources is under way. Also, we are addressing issue (c) mentioned in the Introduction regarding whether the most appropriate data model is a mixture of Gaussians, each having a different mean vector. This would model the fact that each type of non- plume pixel falls into a category such as concrete, grass, water, etc. Similarly, each plume pixel has its own background effect. Therefore, model (3) does not physically represent any given plume pixel (unless there is only one type of background pixel, such as concrete) because the covariance matrix C is estimated by pooling all non-plume pixels. A more realistic model is a Gaussian mixture model on a per-plume-pixel basis. However, unless we could estimate the true background of the plume pixels, this approach would be infeasible. A compromise approach would be to cluster the non-plume pixels into categories such as concrete and grass, assume the plume is entirely over concrete and apply model (3), using 2 as estimated using the concrete-background pixels, then repeat by assuming the plume is entirely over grass, water, etc. Such a strategy would extend our current BMA or PW approaches regarding assessing confidence in the chosen solution. References Hoeting, J., Madigan, D., Raftery, A,, and George, E. (2000), “The Variable Selection Problem,” University of Texas-Austin Technical report available at http://bevo2.bus.utexas.edu/GeorgeE/ McVey, B., Burr, T., and Fry, H., “Distribution of Chemical False Positives for Hyperspectral Image Data,” Official Use Only Los Alamos Report, 2002. Neath, A., and Cavanaugh, J (1997), “Regression and Time Series Model Selection Using Variants of the Schwarz Information Criterion,” Communications in Statistics- Theory and Models, 26, 559-580. Raftery, A., Madigan, D., and Hoeting, J. (1997), “Bayesian Model Averaging for Linear Regression Models,” Journal of the American Statistical Association, 92, 179- 19 1. Splus 6 for Linux, Insightful Corp., 2002. Volinsky, C. (1999), “Bayesian Model Averaging: A Tutorial,” Statistical Science 14(4), 382-417. 

Chemical identification using Bayesian model selection

https://digital.library.unt.edu/ark:/67531/metadc926076/m2/1/high_res_d/976456.pdf

Chemical identification using Bayesian model selection

Abstract

Similar works

Full text

Available Versions

UNT Digital Library