Laparoscopic image analysis for automatic tracking of surgical tools

Laparoscopy is a surgical technique nowadays embedded in the clinical routine. Recent researches have been focused on analysing video information captured by the endoscope for extracting cues useful for surgeons, such as depth information. In particular, the 3D pose estimation of the surgical tools presents three important added values: (1) to extract objective parameters for the surgical training stage, (2) to develop an image-guided surgery based on the knowledge of the surgery tools localization, (3) to design new roboticsystems for an automatic laparoscope positioning, according to the visual feedback. Tool’s shape and orientation in the image is the key to get its 3D position. This work presents an image analysis for automatic laparoscopic tool’s detection along the recorded video without extra tool markers, using an edges detection strategy. Also, this analysis includes a previous stage of barrel distortion correction for videoendoscopic image

Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids

Real-life control tasks involve matters of various substances---rigid or soft bodies, liquid, gas---each with distinct physical behaviors. This poses challenges to traditional rigid-body physics engines. Particle-based simulators have been developed to model the dynamics of these complex scenes; however, relying on approximation techniques, their simulation often deviates from real-world physics, especially in the long term. In this paper, we propose to learn a particle-based simulator for complex control tasks. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. This enables the model to quickly adapt to new environments of unknown dynamics within a few observations. We demonstrate robots achieving complex manipulation tasks using the learned simulator, such as manipulating fluids and deformable foam, with experiments both in simulation and in the real world. Our study helps lay the foundation for robot learning of dynamic scenes with particle-based representations.Comment: Accepted to ICLR 2019. Project Page: http://dpi.csail.mit.edu Video: https://www.youtube.com/watch?v=FrPpP7aW3L

Segmentation and 3D reconstruction approaches for the design of laparoscopic augmented reality environments

A trend in abdominal surgery is the transition from minimally invasive surgery to surgeries where augmented reality is used. Endoscopic video images are proposed to be employed for extracting useful information to help surgeons performing the operating techniques. This work introduces an illumination model into the design of automatic segmentation algorithms and 3D reconstruction methods. Results obtained from the implementation of our methods to real images are supposed to be an initial step useful for designing new methodologies that will help surgeons operating MIS techniques

Out-of-Distribution Generalization in Algorithmic Reasoning Through Curriculum Learning

Out-of-distribution generalization (OODG) is a longstanding challenge for neural networks, and is quite apparent in tasks with well-defined variables and rules, where explicit use of the rules can solve problems independently of the particular values of the variables. Large transformer-based language models have pushed the boundaries on how well neural networks can generalize to novel inputs, but their complexity obfuscates they achieve such robustness. As a step toward understanding how transformer-based systems generalize, we explore the question of OODG in smaller scale transformers. Using a reasoning task based on the puzzle Sudoku, we show that OODG can occur on complex problems if the training set includes examples sampled from the whole distribution of simpler component tasks

Winds of Change: Perspectives on the World\u27s Search for Stable Democracy

Not available