Article thumbnail
Location of Repository

Statistical aspects of elastic scattering spectroscopy with applications to cancer diagnosis

By Y. Zhu


Elastic scattering spectroscopy (ESS), is a non-invasive and real-time in vivo optical diagnosis technique sensitive to changes in the physical properties of human tissue, and thus able to detect early cancer and precancerous changes. This thesis focuses on the statistical issue on how to eliminate irrelevant variations in the high-dimensional ESS spectra and extract the most useful information to enable the classification of tissue as normal or abnormal. Multivariate statistical methods have been used to tackle the problems, among which principal component discriminant analysis and partial least squares discriminant analysis are the most explored throughout the thesis as general tools for supervised dimension reduction and classification. Customized multivariate methods are proposed in the specific context of ESS. When ESS spectra are measured in vivo by a hand-held optical probe, differences in the angle and pressure of the probe are a major source of variability between the spectra from replicate measurements. A customized spectral pre-treatment called error removal by orthogonal subtraction (EROS) is designed to ameliorate the effect of this variability. This pre-treatment reduces the complexity and increases both the accuracy and interpretability of the subsequent classification models when applied to early detection of cancer risk in Barrett’s oesophagus. For the application of ESS to diagnosis of sentinel lymph node metastases in breast cancer, an automated ESS scanner was developed to take measurements from a larger area of tissue to produce ESS images for cancer diagnosis. Problems arise due to the existence of background area in the image with considerable between-node variation and no training data available. A partially supervised Bayesian multivariate finite mixture classification model with a Markov random field spatial prior in a reduced dimensional space is proposed to recognise the background area automatically at the same time as distinguishing normal from metastatic tissue

Publisher: UCL (University College London)
Year: 2009
OAI identifier:
Provided by: UCL Discovery

Suggested articles


  1. (1993). (Eds.)
  2. (2004). A Bayesian analysis of mixture modeling using the multivariate t distribution.
  3. (1992). A classification EM algorithm for clustering and two stochastic versions.
  4. (2008). A finite mixture model for image segmentation.
  5. (2003). A randomized comparison of sentinel-node biopsy with routine axillary dissection in breast cancer.
  6. (2002). A User-friendly Guide to Multivariate Calibration and Classification. NIR publications,
  7. (1993). An endoscopic biopsy protocol can differentiate high-grade dysplasia from early adenocarcinoma in Barrett's esophagus.
  8. (1998). An evaluation of orthogonal signal correction applied to the calibration transfer of near-infrared spectra.
  9. (1989). An iterative Gibbsian technique for reconstruction of m-ary images.
  10. (1997). Analysis of Incomplete Multivariate Data.
  11. (2000). Assessing the conditions for in vivo electrical virtual biopsies in Barrett’s oesophagus.
  12. (2007). Balloon-based, circumferential, endoscopic199 radiofrequency ablation of Barrett's esophagus: 1-year follow-up of 100 patients.
  13. (1991). Barrett's esophagus. Prevalence and incidence of adenocarcinoma.
  14. (1995). Bayesian Data Analysis.
  15. (1997). Bayesian Method for Mixtures of Mormal Distributions.
  16. (2007). Bayesian regularization for normal mixture estimation and model-based clustering.
  17. (1994). Bayesian Theory.
  18. (2004). CancerStats Monograph
  19. (1998). Clinical decision making in Barrett's oesophagus can be supported by computerized immunoquantitation and morphometry of features associated with proliferation and differentiation.
  20. (2002). Comparison of discrimination methods for the classification of tumors using expression data.
  21. (2002). Comparison of side effects between sentinel lymph node and axillary lymph node dissection for breast cancer.
  22. (1974). Cross-validatory choice and assessment of statistical predictions,
  23. (2005). Data Mining: Practical Machine Learning Tools and Techniques.
  24. (2000). Detection of preinvasive cancer cells.
  25. (1988). Determinants of tumor blood flow: a review. Cancer Res.
  26. (1999). Diffuse reflectance spectroscopy of human adenomatous colon polyps in vivo.
  27. (1992). Discriminant Analysis and Statistical Pattern Recognition.
  28. (2006). Elastic scattering spectroscopy accurately detects high-grade dysplasia and cancer in Barrett’s oesophagus.
  29. (2004). Elastic scattering spectroscopy for detection of dysplasia in Barrett's esophagus.
  30. (2004). Elastic scattering spectroscopy for intra-operative determination of sentinel lymph node status in the breast.
  31. (2006). Elastic scattering spectroscopy for the diagnosis of colon lesions: initial results of a novel optical biopsy technique.
  32. (2003). EM procedures using mean field-like approximations for Markov model-based image segmentation.
  33. (2000). Endoscopic detection of dysplasia in patients with Barrett's esophagus using light-scattering spectroscopy.
  34. (1999). Endoscopic fluorescence detection of dysplasia in patients with Barrett's esophagus, ulcerative colitis, or adenomatous polyps after 5-aminolevulinic acid-induced protoporphyrin IX sensitization.
  35. (2003). EPO-PLS external parameter orthogonalisation of PLS application to temperature-independent measurement of sugar content of intact fruits.
  36. (2008). Error removal by orthogonal subtraction (EROS): a customised pretreatment for spectroscopic data.
  37. (1991). Estimation of parameters in hidden Markov models
  38. (1982). Evaluation of Diagnostic Systems: Methods from Signal Detection Theory.
  39. (1998). Evidence of intrinsic differences in the light scattering properties of tumorigenic and nontumorigenic cells.
  40. (1990). Finding groups in data.
  41. (2000). Finite Mixture Models.
  42. (2001). Fluorescence, reflectance, and light-scattering spectroscopy for evaluating dysplasia in patients with Barrett's esophagus.
  43. (1997). Histopathologic validation of the sentinel lymph node hypothesis for breast carcinoma.
  44. (1996). Hypothesis testing and model selection via posterior simulation. In:
  45. (2001). Imaging human epithelial properties with polarized light-scattering spectroscopy.
  46. (1998). Impact of endoscopic biopsy surveillance of Barrett's oesophagus on pathological stage and clinical outcome of Barrett's carcinoma.
  47. (2004). Is cross-validation valid for smallsample microarray classification?
  48. (1999). Light scattering from cells: finite-difference time-domain simulations and goniometric measurements.
  49. (2000). Light scattering from cells: the contribution of the nucleus and the effects of proliferative status.
  50. (1985). Linearisation and scatter correction for near infrared reflectance spectra of meat.
  51. (1971). Markov field on finite graphs and lattices.
  52. (2001). Markov Random Field Modeling in Image Analysis.
  53. (1977). Maximum likelihood form incomplete data via EM algorithm.
  54. (1988). Measuring the accuracy of diagnostics systems.
  55. (1988). Mixture Models: Inference and Applications to Clustering.
  56. (2002). Model-based clustering, discriminant analysis, and density estimation.
  57. (1998). Multivariate Calibration.
  58. (1997). Near infrared light absorption in the human eye media.
  59. (1977). Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters.
  60. (1997). On Bayesian analysis of mixtures with an unknown number of components.
  61. (2000). On orthogonal signal correction.
  62. (1986). On the statistical analysis of dirty pictures,
  63. (1973). Optical constants of water in 200-nm to 200-μm wavelength region.
  64. (2002). Orthogonal projections to latent structures (O-PLS).
  65. (1998). Orthogonal signal correction of near-infrared spectra.
  66. (2003). Partial least squares for discrimination.
  67. (1996). Pattern Recognition and Neural Networks.
  68. (2001). Pre-processing method minimizing the need for reference analyses.
  69. (2005). Prediction error estimation: A comparison of resampling methods.
  70. (1997). Predictions and measurements of scattering and absorption over broad wavelength ranges in tissue phantoms.
  71. (2001). Predictors of progression in Barrett's esophagus III: baseline flow cytometric variables.
  72. (2000). Robust mixture modelling using the t distribution,
  73. (1989). Robust statistical modeling using the t distribution.
  74. (2004). ROC graphs: Notes and practical considerations for data mining researchers. Tech report HPL-2003-4. HP Laboratories,
  75. (1998). Second order calibration: Bilinear least squares regression and a simple alternative.
  76. (2001). Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm.
  77. (1977). SIMCA: A method for analyzing chemical data in terms of similarity and analogy,
  78. Smilde (2001). Direct orthogonal signal correction.
  79. (1964). Smoothing and differentiation of data by simplified least squares procedures.
  80. (1974). Spatial Interaction and the Statistical Analysis of Lattice Systems.
  81. (1989). Standard normal variate tansformation and detrending of near infrared diffuse reflectance.
  82. (1985). Statistical Analysis of Finite Mixture Distributions.
  83. (1997). Statistical analysis of NIR data: data pre-treatment.
  84. (1975). Statistical analysis of non-lattice data.
  85. (1987). Statistical Analysis with Missing Data.
  86. (1984). Stochastic relaxation, Gibbs distribution and the Bayesian restoration of images.
  87. (1997). The EM Algorithm and Extensions.
  88. (1982). The meaning and use of the area under a196 receiver operating characteristic (ROC) curve.
  89. (1997). The use of area under the ROC curve in the evaluation of machine learning algorithms.
  90. (2004). Transfer by orthogonal projection: making nearinfrared calibrations robust to between-instrument variation.
  91. (2003). Trends in the subsite and morphology of oesophageal and gastric cancer in England and Wales 1971-1998. Aliment Pharmacol.
  92. (1997). Ultraviolet and visible spectroscopies for tissue diagnostics: Fluorescence spectroscopy and elastic scattering spectroscopy.
  93. (2006). Using unlabelled data to update classification rules with applications in food authenticity studies.
  94. (1989). van den Tweel
  95. (2000). Visible and Near Infrared Absorption Spectra of Human and Animal Haemoglobin: Determination and Application.

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.