mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral tradeoff. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from an RGB image. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep-learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral tradeoff, offering simple hardware requirements and potential applications of various machine-learning techniques.Comment: This paper will appear in PNAS Nexu

Clinical applications of non‐invasive multi and hyperspectral imaging of cell and tissue autofluorescence beyond oncology

Hyperspectral and multispectral imaging of cell and tissue autofluorescence employs fluorescence imaging, without exogenous fluorophores, across multiple excitation/ emission combinations (spectral channels). This produces an image stack where each pixel (matched by location) contains unique information about the sample's spectral properties. Analysis of this data enables access to a rich, molecularly specific data set from a broad range of cell-native fluorophores (autofluorophores) directly reflective of biochemical status, without use of fixation or stains. This non-invasive, non-destructive technology has great potential to spare the collection of biopsies from sensitive regions. As both staining and biopsy may be impossible, or undesirable, depending on the context, this technology great diagnostic potential for clinical decision making. The main research focus has been on the identification of neoplastic tissues. However, advances have been made in diverse applications—including ophthalmology, cardiovascular health, neurology, infection, assisted reproduction technology and organ transplantation.Jared M. Campbell, Saabah B. Mahbub, Abbas Habibalahi, Adnan Agha, Shannon Handley, Ayad G. Anwer, Ewa M. Goldy

Going Deep in Medical Image Analysis: Concepts, Methods, Challenges and Future Directions

Medical Image Analysis is currently experiencing a paradigm shift due to Deep Learning. This technology has recently attracted so much interest of the Medical Imaging community that it led to a specialized conference in `Medical Imaging with Deep Learning' in the year 2018. This article surveys the recent developments in this direction, and provides a critical review of the related major aspects. We organize the reviewed literature according to the underlying Pattern Recognition tasks, and further sub-categorize it following a taxonomy based on human anatomy. This article does not assume prior knowledge of Deep Learning and makes a significant contribution in explaining the core Deep Learning concepts to the non-experts in the Medical community. Unique to this study is the Computer Vision/Machine Learning perspective taken on the advances of Deep Learning in Medical Imaging. This enables us to single out `lack of appropriately annotated large-scale datasets' as the core challenge (among other challenges) in this research direction. We draw on the insights from the sister research fields of Computer Vision, Pattern Recognition and Machine Learning etc.; where the techniques of dealing with such challenges have already matured, to provide promising directions for the Medical Imaging community to fully harness Deep Learning in the future

Development and applications of high speed and hyperspectral nonlinear microscopy

Nonlinear microscopy refers to a range of laser scanning microscopy techniques that are based on nonlinear optical processes such as two-photon excited fluorescence and second harmonic generation. Nonlinear microscopy techniques are powerful because they enable the visualization of highly scattering biological samples with subcellular resolution. This capability is especially valuable for in vivo and live tissue imaging since it can provide both structural and functional information about tissues in their native environment. With the use of a range of exogenous dyes and intrinsic contrast, in vivo nonlinear microscopy can be used to characterize and measure dynamic processes of tissues in their normal environment. These advances have been particularly relevant in neuroscience, where truly understanding the function of the brain requires that its neural and vascular networks be observed while undisturbed. Despite these advantages, in vivo nonlinear microscopy still faces several major challenges. First, observing dynamics that occur in large areas over short time scales, such as neuronal signaling and blood flow, is challenging because nonlinear microscopy generally requires scanning to create an image. This limits the study of dynamic behavior to either a single plane or to a small subset of regions within a volume. Second, applications that rely on the use of exogenous dyes can be limited by the need to stain tissues before imaging, the availability of dyes, and specificity that can be achieved. Usually considered a nuisance, endogenous tissue contrast from autofluorescence or structures exhibiting second harmonic generation can produce stunning images for visualizing subcellular morphology. Imaging endogenous contrast can also provide valuable information about the chemical makeup and metabolic state of the tissue. Few methods have been developed to carefully and quantitatively examine endogenous fluorescence in living tissues. In this thesis, these two challenges in nonlinear microscopy are addressed. The development of a novel hyperspectral two-photon microscopy method to acquire spectroscopic data from tissues and increase the information available from endogenous contrast is presented. This system was applied to visualize and identify sources of endogenous contrast in gastrointestinal tissues, providing robust references for the assessment of normal and diseased tissues. Secondly, three methods for high speed volumetric imaging using laser scanning nonlinear microscopy were developed to address the need for improved high-speed imaging in living tissues. A spectrally-encoded high-speed imaging method that can provide simultaneous imaging of multiple regions of the living brain in parallel is presented and used to study spontaneous changes in vascular tone in the brain. This technique is then extended for use with second harmonic generation microscopy, which has the potential to greatly increase the degree of multiplexing. Finally, a complete system design capable of volumetric scan rates >1Hz is shown, offering improved performance and versatility to image brain activity

Machine learning methods for the characterization and classification of complex data

This thesis work presents novel methods for the analysis and classification of medical images and, more generally, complex data. First, an unsupervised machine learning method is proposed to order anterior chamber OCT (Optical Coherence Tomography) images according to a patient's risk of developing angle-closure glaucoma. In a second study, two outlier finding techniques are proposed to improve the results of above mentioned machine learning algorithm, we also show that they are applicable to a wide variety of data, including fraud detection in credit card transactions. In a third study, the topology of the vascular network of the retina, considering it a complex tree-like network is analyzed and we show that structural differences reveal the presence of glaucoma and diabetic retinopathy. In a fourth study we use a model of a laser with optical injection that presents extreme events in its intensity time-series to evaluate machine learning methods to forecast such extreme events.El presente trabajo de tesis desarrolla nuevos métodos para el análisis y clasificación de imágenes médicas y datos complejos en general. Primero, proponemos un método de aprendizaje automático sin supervisión que ordena imágenes OCT (tomografía de coherencia óptica) de la cámara anterior del ojo en función del grado de riesgo del paciente de padecer glaucoma de ángulo cerrado. Luego, desarrollamos dos métodos de detección automática de anomalías que utilizamos para mejorar los resultados del algoritmo anterior, pero que su aplicabilidad va mucho más allá, siendo útil, incluso, para la detección automática de fraudes en transacciones de tarjetas de crédito. Mostramos también, cómo al analizar la topología de la red vascular de la retina considerándola una red compleja, podemos detectar la presencia de glaucoma y de retinopatía diabética a través de diferencias estructurales. Estudiamos también un modelo de un láser con inyección óptica que presenta eventos extremos en la serie temporal de intensidad para evaluar diferentes métodos de aprendizaje automático para predecir dichos eventos extremos.Aquesta tesi desenvolupa nous mètodes per a l’anàlisi i la classificació d’imatges mèdiques i dades complexes. Hem proposat, primer, un mètode d’aprenentatge automàtic sense supervisió que ordena imatges OCT (tomografia de coherència òptica) de la cambra anterior de l’ull en funció del grau de risc del pacient de patir glaucoma d’angle tancat. Després, hem desenvolupat dos mètodes de detecció automàtica d’anomalies que hem utilitzat per millorar els resultats de l’algoritme anterior, però que la seva aplicabilitat va molt més enllà, sent útil, fins i tot, per a la detecció automàtica de fraus en transaccions de targetes de crèdit. Mostrem també, com en analitzar la topologia de la xarxa vascular de la retina considerant-la una xarxa complexa, podem detectar la presència de glaucoma i de retinopatia diabètica a través de diferències estructurals. Finalment, hem estudiat un làser amb injecció òptica, el qual presenta esdeveniments extrems en la sèrie temporal d’intensitat. Hem avaluat diferents mètodes per tal de predir-los.Postprint (published version