Distortion and instability compensation with deep learning for rotational scanning endoscopic optical coherence tomography

Optical Coherence Tomography (OCT) is increasingly used in endoluminal procedures since it provides high-speed and high resolution imaging. Distortion and instability of images obtained with a proximal scanning endoscopic OCT system are significant due to the motor rotation irregularity, the friction between the rotating probe and outer sheath and synchronization issues. On-line compensation of artefacts is essential to ensure image quality suitable for real-time assistance during diagnosis or minimally invasive treatment. In this paper, we propose a new online correction method to tackle both B-scan distortion, video stream shaking and drift problem of endoscopic OCT linked to A-line level image shifting. The proposed computational approach for OCT scanning video correction integrates a Convolutional Neural Network (CNN) to improve the estimation of azimuthal shifting of each A-line. To suppress the accumulative error of integral estimation we also introduce another CNN branch to estimate a dynamic overall orientation angle. We train the network with semi-synthetic OCT videos by intentionally adding rotational distortion into real OCT scanning images. The results show that networks trained on this semi-synthetic data generalize to stabilize real OCT videos, and the algorithm efficacy is demonstrated on both ex vivo and in vivo data, where strong scanning artifacts are successfully corrected. (c) 2022 The Authors. Published by Elsevier B.V

A Novel Telemanipulated Robotic Assistant for Surgical Endoscopy: Preclinical Application to ESD

International audienceObjective: Minimally invasive surgical interventions in the gastrointestinal tract, such as Endoscopic Submucosal Dissection (ESD), are very difficult for surgeons when performed with standard flexible endoscopes. Robotic flexible systems have been identified as a solution to improve manipulation. However, only a few such systems have been brought to preclinical trials as of now. As a result, novel robotic tools are required.Methods: We developed a telemanipulated robotic device, called STRAS, which aims to assist surgeons during intraluminal surgical endoscopy. This is a modular system, based on a flexible endoscope and flexible instruments, which provides 10 degrees of freedom (DoFs). The modularity allows to easily set up the robot and to navigate towards the operating area. The robot can then be teleoperated using master interfaces specifically designed to intuitively control all available DoFs. STRAS capabilities have been tested in laboratory conditions and during preclinical experiments. Results: We report twelve colorectal ESDs performed in pigs, in which large lesions were successfully removed. Dissection speeds are compared with those obtained in similar conditions with the manual Anubiscope TM platform from Karl Storz. We show significant improvements (p = 0.01).Conclusion: These experiments show that STRAS (v2) provides sufficient DoFs, workspace and force to perform ESD, that it allows a single surgeon to perform all the surgical tasks and that performances are improved with respect to manual systems. Significance: The concepts developed for STRAS are validated and could bring new tools for surgeons to improve comfort, ease and performances for intraluminal surgical endoscopy

The global abundance of tree palms

Aim Palms are an iconic, diverse and often abundant component of tropical ecosystems that provide many ecosystem services. Being monocots, tree palms are evolutionarily, morphologically and physiologically distinct from other trees, and these differences have important consequences for ecosystem services (e.g., carbon sequestration and storage) and in terms of responses to climate change. We quantified global patterns of tree palm relative abundance to help improve understanding of tropical forests and reduce uncertainty about these ecosystems under climate change. Location Tropical and subtropical moist forests. Time period Current. Major taxa studied Palms (Arecaceae). Methods We assembled a pantropical dataset of 2,548 forest plots (covering 1,191 ha) and quantified tree palm (i.e., ≥10 cm diameter at breast height) abundance relative to co‐occurring non‐palm trees. We compared the relative abundance of tree palms across biogeographical realms and tested for associations with palaeoclimate stability, current climate, edaphic conditions and metrics of forest structure. Results On average, the relative abundance of tree palms was more than five times larger between Neotropical locations and other biogeographical realms. Tree palms were absent in most locations outside the Neotropics but present in >80% of Neotropical locations. The relative abundance of tree palms was more strongly associated with local conditions (e.g., higher mean annual precipitation, lower soil fertility, shallower water table and lower plot mean wood density) than metrics of long‐term climate stability. Life‐form diversity also influenced the patterns; palm assemblages outside the Neotropics comprise many non‐tree (e.g., climbing) palms. Finally, we show that tree palms can influence estimates of above‐ground biomass, but the magnitude and direction of the effect require additional work. Conclusions Tree palms are not only quintessentially tropical, but they are also overwhelmingly Neotropical. Future work to understand the contributions of tree palms to biomass estimates and carbon cycling will be particularly crucial in Neotropical forests

Contributions à l'asservissement visuel robuste

STRASBOURG-Sc. et Techniques (674822102) / SudocSudocFranceF

Towards in situ Backlash Estimation of Continuum Robots using an Endoscopic Camera

International audienceAccurate control of continuum robots requires handling non-linear behaviors between actuators and distal effectors. In this paper, we develop a method for estimating the non-linearities of tendon-driven degrees of freedom of flexible endoscopic systems by using a distal endoscopic camera and encoders at the proximal side. The proposed approach separates the non-linearities in two parts, namely a pure non uniform backlash and a non-linear function. The backlash is estimated without relying on any model, while the non-linear function is obtained by a pose estimation process. Experiments realized on a robotic flexible endoscopy platform (STRAS) show the validity of the approach for estimating in situ the quasi-static behavior of the robot and for compensating the non-linearities of the motion transmission

An adaptive and fully automatic method for estimating the 3D position of bendable instruments using endoscopic images

International audienceBackground. Flexible bendable instruments are key tools for performing surgical endoscopy. Being able to measure the 3D position of such instruments can be useful for various tasks, such as controlling automatically robotized instruments and analyzing motions. Methods. We propose an automatic method to infer the 3D pose of a single bending section instrument, using only the images provided by a monocular camera embedded at the tip of the endoscope. The proposed method relies on colored markers attached onto the bending section. The image of the instrument is segmented using a graph-based method and the corners of the markers are extracted by detecting the color transition along Bézier curves fitted on edge points. These features are accurately located and then used to estimate the 3D pose of the instrument using an adaptive model that allows to take into account the mechanical play between the instrument and its housing channel. Results. The feature extraction method provides good localization of markers corners with images of in vivo environment despite sensor saturation due to strong lighting. The RMS error on the estimation of the tip position of the instrument for laboratory experiments was 2.1, 1.96, 3.18 mm in the x, y and z directions respectively. Qualitative analysis in the case of in vivo images shows the ability to correctly estimate the 3D position of the instrument tip during real motions. Conclusions. The proposed method provides an automatic and accurate estimation of the 3D position of the tip of a bendable instrument in realistic conditions, where standard approaches fail

Sensors for Expert Grip Force Profiling: Towards Benchmarking Manual Control of a Robotic Device for Surgical Tool Movements

International audienc

Correlating Grip Force Signals from Multiple Sensors Highlights Prehensile Control Strategies in a Complex Task-User System

International audienceWearable sensor systems with transmitting capabilities are currently employed for the biometric screening of exercise activities and other performance data. Such technology is generally wireless and enables the noninvasive monitoring of signals to track and trace user behaviors in real time. Examples include signals relative to hand and finger movements or force control reflected by individual grip force data. As will be shown here, these signals directly translate into task, skill, and hand specific, dominant versus non dominant hand, grip force profiles for different measurement loci in the fingers and palm of the hand. The present study draws from thousands of such sensor data recorded from multiple spatial locations. The individual grip force profiles of a highly proficient left handed exper, a right handed dominant hand trained user, and a right handed novice performing an image guided, robot assisted precision task with the dominant or the non dominant hand are analyzed. The step by step statistical approach follows Tukeys detective work principle, guided by explicit functional assumptions relating to somatosensory receptive field organization in the human brain. Correlation analyses in terms of Person Product Moments reveal skill specific differences in covariation patterns in the individual grip force profiles. These can be functionally mapped to from global to local coding principles in the brain networks that govern grip force control and its optimization with a specific task expertise. Implications for the real time monitoring of grip forces and performance training in complex task user systems are brought forward

Sensors for Expert Grip Force Profiling : Towards Benchmarking Manual Control of a Robotic Device for Surgical Tool Movements

International audienc