74 research outputs found
Enhancing Informative Frame Filtering by Water and Bubble Detection in Colonoscopy Videos
Colonoscopy has contributed to a marked decline in the number of colorectal cancer related deaths. However, recent data suggest that there is a significant (4-12%) miss-rate for the detection of even large polyps and cancers. To address this, we have been investigating an ‘automated feedback system’ which informs the endoscopist of possible sub-optimal inspection during colonoscopy. A fundamental step of this system is to distinguish non-informative frames from informative ones. Existing methods for this cannot classify water/bubble frames as non-informative even though they do not carry any useful visual information of the colon mucosa. In this paper, we propose a novel texture feature based on accumulation of pixel differences, which can detect water and bubble frames with very high accuracy with significantly less processing time. The experimental results show the proposed feature can achieve more than 93% overall accuracy in almost half of the processing time the existing methods take
Recommended from our members
Effective and Accelerated Informative Frame Filtering in Colonoscopy Videos Using Graphic Processing Units
Colonoscopy is an endoscopic technique that allows a physician to inspect the mucosa of the human colon. Previous methods and software solutions to detect informative frames in a colonoscopy video (a process called informative frame filtering or IFF) have been hugely ineffective in (1) covering the proper definition of an informative frame in the broadest sense and (2) striking an optimal balance between accuracy and speed of classification in both real-time and non real-time medical procedures. In my thesis, I propose a more effective method and faster software solutions for IFF which is more effective due to the introduction of a heuristic algorithm (derived from experimental analysis of typical colon features) for classification. It contributed to a 5-10% boost in various performance metrics for IFF. The software modules are faster due to the incorporation of sophisticated parallel-processing oriented coding techniques on modern microprocessors. Two IFF modules were created, one for post-procedure and the other for real-time. Code optimizations through NVIDIA CUDA for GPU processing and/or CPU multi-threading concepts embedded in two significant microprocessor design philosophies (multi-core design and many-core design) resulted a 5-fold acceleration for the post-procedure module and a 40-fold acceleration for the real-time module. Some innovative software modules, which are still in testing phase, have been recently created to exploit the power of multiple GPUs together
Recommended from our members
Detection of Ulcerative Colitis Severity and Enhancement of Informative Frame Filtering Using Texture Analysis in Colonoscopy Videos
There are several types of disorders that affect our colon’s ability to function properly such as colorectal cancer, ulcerative colitis, diverticulitis, irritable bowel syndrome and colonic polyps. Automatic detection of these diseases would inform the endoscopist of possible sub-optimal inspection during the colonoscopy procedure as well as save time during post-procedure evaluation. But existing systems only detects few of those disorders like colonic polyps. In this dissertation, we address the automatic detection of another important disorder called ulcerative colitis. We propose a novel texture feature extraction technique to detect the severity of ulcerative colitis in block, image, and video levels. We also enhance the current informative frame filtering methods by detecting water and bubble frames using our proposed technique. Our feature extraction algorithm based on accumulation of pixel value difference provides better accuracy at faster speed than the existing methods making it highly suitable for real-time systems. We also propose a hybrid approach in which our feature method is combined with existing feature method(s) to provide even better accuracy. We extend the block and image level detection method to video level severity score calculation and shot segmentation. Also, the proposed novel feature extraction method can detect water and bubble frames in colonoscopy videos with very high accuracy in significantly less processing time even when clustering is used to reduce the training size by 10 times
New Techniques in Gastrointestinal Endoscopy
As result of progress, endoscopy has became more complex, using more sophisticated devices and has claimed a special form. In this moment, the gastroenterologist performing endoscopy has to be an expert in macroscopic view of the lesions in the gut, with good skills for using standard endoscopes, with good experience in ultrasound (for performing endoscopic ultrasound), with pathology experience for confocal examination. It is compulsory to get experience and to have patience and attention for the follow-up of thousands of images transmitted during capsule endoscopy or to have knowledge in physics necessary for autofluorescence imaging endoscopy. Therefore, the idea of an endoscopist has changed. Examinations mentioned need a special formation, a superior level of instruction, accessible to those who have already gained enough experience in basic diagnostic endoscopy. This is the reason for what these new issues of endoscopy are presented in this book of New techniques in Gastrointestinal Endoscopy
Visual Odometer on Videos of Endoscopic Capsules (VOVEC)
Desde a sua introdução em 2001, as cápsulas endoscópicas tornaram-se o principal método para obter imagens do intestino - uma região de difícil acesso através de métodos de endoscopia tradicionais - revolucionando a maneira como os diagnósticos no campo das doenças do intestino delgado são feitos. Estas cápsulas com dimensões comparáveis a um comprimido vitamínico tiram partido de uma câmera wireless para criar vídeos de 8 a 10 horas do trato digestivo dos pacientes. Devido à longa duração dos vídeos produzidos, o diagnóstico humano é moroso, entediante e propício a erros. Para além disto, depois de encontrada uma lesão, a informação da sua localização é escassa e dependente de hardware externo, levando a que uma solução baseada apensa em software com precisão melhorada seja bastante desejada. Este trabalho advém desta necessidade e, tendo-a em mente, propomos a implementação de dois métodos baseados em deep-learning, visando melhorar em relação às limitações dos sistemas atuais de localização de cápsulas endoscópicas. Para treinar e testar as nossas redes neuronais, um dataset que contém 111 vídeos da cápsula PillCam SB3 e 338 da cápsula PillCam SB2 foi utilizado, cortesia do Centro Hospitalar do Porto (CHP).O primeiro método consiste numa simples estimação do deslocamento da cápsula ao longo do intestino delgado utilizando uma HomographyNet, uma abordagem de deep-learning supervisionado usada para o cálculo de homografias entre imagens.Já no segundo método uma posição relativa 3D da cápsula é fornecida ao longo do intestino delgado, recorrendo a um método não-supervisionado de deep-learning denominado SfMLearner. Este método combina uma DepthNet e uma PoseNet para aprender a profundidade da imagem e a posição da cápsula em simultâneo.Since its introduction in 2001, capsule endoscopy has become the leading screening method for the small bowel - a region not easily accessible with traditional endoscopy techniques - revolutionizing the way diagnostics work in the field of small bowel diseases. These capsules are vitamin-sized and leverage from a small wireless camera to create 8 to 10 hour videos of the patients digestive tract. Due to the long duration of the videos produced, the human-based diagnosis is elongated, tedious and error-prone. Moreover, once a lesion is found, the localization information is scarce and hardware dependent, entailing desirability for a software-only endoscopic capsule localization system with added precision. This work stems from this need and, bearing this in mind, we propose the implementation of two deep-learning based methods to improve upon the limitations of the techniques used so far for the capsule position estimation. To train and test our networks, a dataset of 111 PillCam SB3 and 338 PillCam SB2 videos were used, courtesy of Centro Hospitalar do Porto (CHP).The first method consists in a simple capsule displacement estimation throughout the small bowel utilizing HomographyNet, a deep learning supervised approach that is used for homography computation between images. (DeTone et al. (2016))Differently, the second proposed method is intended to provide a 3D position along the small intestine, utilizing a deep learning unsupervised approach labeled SfMLearner, which takes advantage of a combination between a DepthNet and a PoseNet to learn depth and ego-motion from video simultaneously. (Zhou et al. (2017)
ADVANCED INTRAVASCULAR MAGNETIC RESONANCE IMAGING WITH INTERACTION
Intravascular (IV) Magnetic Resonance Imaging (MRI) is a specialized class of interventional MRI (iMRI) techniques that acquire MRI images through blood vessels to guide, identify and/or treat pathologies inside the human body which are otherwise difficult to locate and treat precisely. Here, interactions based on real-time computations and feedback are explored to improve the accuracy and efficiency of IVMRI procedures.
First, an IV MRI-guided high-intensity focused ultrasound (HIFU) ablation method is developed for targeting perivascular pathology with minimal injury to the vessel wall. To take advantage of real-time feedback, a software interface is developed for monitoring thermal dose with real-time MRI thermometry, and an MRI-guided ablation protocol developed and tested on muscle and liver tissue ex vivo. It is shown that, with cumulative thermal dose monitored with MRI thermometry, lesion location and dimensions can be estimated consistently, and desirable thermal lesions can be achieved in animals in vivo.
Second, to achieve fully interactive IV MRI, high-resolution real-time 10 frames-per-second (fps) MRI endoscopy is developed as an advance over prior methods of MRI endoscopy. Intravascular transmit-receive MRI endoscopes are fabricated for highly under-sampled radial-projection MRI in a clinical 3Tesla MRI scanner. Iterative nonlinear reconstruction is accelerated using graphics processor units (GPU) to achieve true real-time endoscopy visualization at the scanner. The results of high-speed MRI endoscopy at 6-10 fps are consistent with fully-sampled MRI endoscopy and histology, with feasibility demonstrated in vivo in a large animal model.
Last, a general framework for automatic imaging contrast tuning over MRI protocol parameters is explored. The framework reveals typical signal patterns over different protocol parameters from calibration imaging data and applies this knowledge to design efficient acquisition strategies and predicts contrasts under unacquired protocols. An external computer in real-time communication with the MRI console is utilized for online processing and controlling MRI acquisitions. This workflow enables machine learning for optimizing acquisition strategies in general, and provides a foundation for efficiently tuning MRI protocol parameters to perform interventional MRI in the highly varying and interactive environments commonly in play. This work is loosely inspired by prior research on extremely accelerated MRI relaxometry using the minimal-acquisition linear algebraic modeling (SLAM) method
Pattern Recognition
Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition
Multispectral image analysis in laparoscopy – A machine learning approach to live perfusion monitoring
Modern visceral surgery is often performed through small incisions. Compared to open surgery, these minimally invasive interventions result in smaller scars, fewer complications and a quicker recovery. While to the patients benefit, it has the drawback of limiting the physician’s perception largely to that of visual feedback through a camera mounted on a rod lens: the laparoscope. Conventional laparoscopes are limited by “imitating” the human eye. Multispectral cameras remove this arbitrary restriction of recording only red, green and blue colors. Instead, they capture many specific bands of light. Although these could help characterize important indications such as ischemia and early stage adenoma, the lack of powerful digital image processing prevents realizing the technique’s full potential.
The primary objective of this thesis was to pioneer fluent functional multispectral imaging (MSI) in laparoscopy. The main technical obstacles were: (1) The lack of image analysis concepts that provide both high accuracy and speed. (2) Multispectral image recording is slow, typically ranging from seconds to minutes. (3) Obtaining a quantitative ground truth for the measurements is hard or even impossible.
To overcome these hurdles and enable functional laparoscopy, for the first time in this field physical models are combined with powerful machine learning techniques. The physical model is employed to create highly accurate simulations, which in turn teach the algorithm to rapidly relate multispectral pixels to underlying functional changes. To reduce the domain shift introduced by learning from simulations, a novel transfer learning approach automatically adapts generic simulations to match almost arbitrary
recordings of visceral tissue. In combination with the only available video-rate capable multispectral sensor, the method pioneers fluent perfusion monitoring with MSI. This system was carefully tested in a multistage process, involving in silico quantitative evaluations, tissue phantoms and a porcine study. Clinical applicability was ensured through in-patient recordings in the context of partial nephrectomy; in these, the novel system characterized ischemia live during the intervention. Verified against a fluorescence reference, the results indicate that fluent, non-invasive ischemia detection and monitoring is now possible.
In conclusion, this thesis presents the first multispectral laparoscope capable of videorate functional analysis. The system was successfully evaluated in in-patient trials, and future work should be directed towards evaluation of the system in a larger study. Due to the broad applicability and the large potential clinical benefit of the presented functional estimation approach, I am confident the descendants of this system are an integral part
of the next generation OR
- …