Tracking and Mapping in Medical Computer Vision: A Review

As computer vision algorithms are becoming more capable, their applications in clinical systems will become more pervasive. These applications include diagnostics such as colonoscopy and bronchoscopy, guiding biopsies and minimally invasive interventions and surgery, automating instrument motion and providing image guidance using pre-operative scans. Many of these applications depend on the specific visual nature of medical scenes and require designing and applying algorithms to perform in this environment. In this review, we provide an update to the field of camera-based tracking and scene mapping in surgery and diagnostics in medical computer vision. We begin with describing our review process, which results in a final list of 515 papers that we cover. We then give a high-level summary of the state of the art and provide relevant background for those who need tracking and mapping for their clinical applications. We then review datasets provided in the field and the clinical needs therein. Then, we delve in depth into the algorithmic side, and summarize recent developments, which should be especially useful for algorithm designers and to those looking to understand the capability of off-the-shelf methods. We focus on algorithms for deformable environments while also reviewing the essential building blocks in rigid tracking and mapping since there is a large amount of crossover in methods. Finally, we discuss the current state of the tracking and mapping methods along with needs for future algorithms, needs for quantification, and the viability of clinical applications in the field. We conclude that new methods need to be designed or combined to support clinical applications in deformable environments, and more focus needs to be put into collecting datasets for training and evaluation.Comment: 31 pages, 17 figure

25th International Congress of the European Association for Endoscopic Surgery (EAES) Frankfurt, Germany, 14-17 June 2017 : Oral Presentations

Introduction: Ouyang has recently proposed hiatal surface area (HSA) calculation by multiplanar multislice computer tomography (MDCT) scan as a useful tool for planning treatment of hiatus defects with hiatal hernia (HH), with or without gastroesophageal reflux (MRGE). Preoperative upper endoscopy or barium swallow cannot predict the HSA and pillars conditions. Aim to asses the efficacy of MDCT’s calculation of HSA for planning the best approach for the hiatal defects treatment. Methods: We retrospectively analyzed 25 patients, candidates to laparoscopic antireflux surgery as primary surgery or hiatus repair concomitant with or after bariatric surgery. Patients were analyzed preoperatively and after one-year follow-up by MDCT scan measurement of esophageal hiatus surface. Five normal patients were enrolled as control group. The HSA’s intraoperative calculation was performed after complete dissection of the area considered a triangle. Postoperative CT-scan was done after 12 months or any time reflux symptoms appeared. Results: (1) Mean HSA in control patients with no HH, no MRGE was cm2 and similar in non-complicated patients with previous LSG and cruroplasty. (2) Mean HSA in patients candidates to cruroplasty was 7.40 cm2. (3) Mean HSA in patients candidates to redo cruroplasty for recurrence was 10.11 cm2. Discussion. MDCT scan offer the possibility to obtain an objective measurement of the HSA and the correlation with endoscopic findings and symptoms. The preoperative information allow to discuss with patients the proper technique when a HSA[5 cm2 is detected. During the follow-up a correlation between symptoms and failure of cruroplasty can be assessed. Conclusions: MDCT scan seems to be an effective non-invasive method to plan hiatal defect treatment and to check during the follow-up the potential recurrence. Future research should correlate in larger series imaging data with intraoperative findings

Development Of A High Performance Mosaicing And Super-Resolution Algorithm

In this dissertation, a high-performance mosaicing and super-resolution algorithm is described. The scale invariant feature transform (SIFT)-based mosaicing algorithm builds an initial mosaic which is iteratively updated by the robust super resolution algorithm to achieve the final high-resolution mosaic. Two different types of datasets are used for testing: high altitude balloon data and unmanned aerial vehicle data. To evaluate our algorithm, five performance metrics are employed: mean square error, peak signal to noise ratio, singular value decomposition, slope of reciprocal singular value curve, and cumulative probability of blur detection. Extensive testing shows that the proposed algorithm is effective in improving the captured aerial data and the performance metrics are accurate in quantifying the evaluation of the algorithm

Rethinking residue, an investigation of pharyngeal residue on flexible endoscopic evaluation of swallowing: the past, present, and future directions

This dissertation investigated measures of pharyngeal residue as seen on flexible endoscopic evaluation of swallowing (FEES). Research in this area of deglutology has been stalled due to measurement problems. The particular aims of this project were to compare visual analog scale ratings to categorical ratings of residue on FEES, and to investigate various measurement aspects. METHODS: Speech language pathologists were asked to rate residue from 81 swallows on FEES that demonstrated a wide range of residue severity for thin liquid, applesauce, and cracker boluses. A total of 33 clinicians rated the amount of residue at the time point after the first swallow, twice in a randomized fashion: the first time on a visual analog scale (VAS) and the second time categorically on a five point Likert scale. The results were analyzed for (1) inter/intra-rater agreement, (2) correlations between ratings and residue severity for each rating method, and (3) clusters of ratings to better define the scales and their clinical significance. A total of 2,673 VAS ratings and 2,673 categorical ratings were collected. RESULTS: (1) Both inter- and intra-rater reliability met acceptable levels of agreement, although intra-rater reliability on VAS ratings were slightly higher (r=0.8–0.9) than categorical ratings (k=0.7–0.8). Expert ratings were not significantly different from other clinicians’ ratings for any severity of any of the 3 boluses. (2) Residue ratings fit best on a curvilinear model; a quadratic fit of the data significantly improved the r2 values for each bolus type. (3) An increased residue amount, rated on either the VAS or categorical scale, was significantly associated with worse penetration-aspiration scale scores, but no significant relationship was found between the two methods of residue ratings and measures of quality of life or diet. Novel computerized methods are proposed for future measurement pursuits. CONCLUSION: The results of this dissertation suggest that residue is best measured on a scale with unequal intervals, and clinicians can be reliable in rating overall amount of residue on FEES after the first swallow. Novel computerized measurement approaches are useful building blocks for future research. It is hoped that with better measurement will come better understanding of residue, its risks, and consequences

Rethinking residue, an investigation of pharyngeal residue on flexible endoscopic evaluation of swallowing: the past, present, and future directions

This dissertation investigated measures of pharyngeal residue as seen on flexible endoscopic evaluation of swallowing (FEES). Research in this area of deglutology has been stalled due to measurement problems. The particular aims of this project were to compare visual analog scale ratings to categorical ratings of residue on FEES, and to investigate various measurement aspects. METHODS: Speech language pathologists were asked to rate residue from 81 swallows on FEES that demonstrated a wide range of residue severity for thin liquid, applesauce, and cracker boluses. A total of 33 clinicians rated the amount of residue at the time point after the first swallow, twice in a randomized fashion: the first time on a visual analog scale (VAS) and the second time categorically on a five point Likert scale. The results were analyzed for (1) inter/intra-rater agreement, (2) correlations between ratings and residue severity for each rating method, and (3) clusters of ratings to better define the scales and their clinical significance. A total of 2,673 VAS ratings and 2,673 categorical ratings were collected. RESULTS: (1) Both inter- and intra-rater reliability met acceptable levels of agreement, although intra-rater reliability on VAS ratings were slightly higher (r=0.8–0.9) than categorical ratings (k=0.7–0.8). Expert ratings were not significantly different from other clinicians’ ratings for any severity of any of the 3 boluses. (2) Residue ratings fit best on a curvilinear model; a quadratic fit of the data significantly improved the r2 values for each bolus type. (3) An increased residue amount, rated on either the VAS or categorical scale, was significantly associated with worse penetration-aspiration scale scores, but no significant relationship was found between the two methods of residue ratings and measures of quality of life or diet. Novel computerized methods are proposed for future measurement pursuits. CONCLUSION: The results of this dissertation suggest that residue is best measured on a scale with unequal intervals, and clinicians can be reliable in rating overall amount of residue on FEES after the first swallow. Novel computerized measurement approaches are useful building blocks for future research. It is hoped that with better measurement will come better understanding of residue, its risks, and consequences

Visual and Camera Sensors

This book includes 13 papers published in Special Issue ("Visual and Camera Sensors") of the journal Sensors. The goal of this Special Issue was to invite high-quality, state-of-the-art research papers dealing with challenging issues in visual and camera sensors