Contribution à l'aide aux gestes pour la chirurgie cardiaque à coeur battant. Guidage échographique par asservissement prédictif non linéaire.

This work is concerned with robotic assisted beating heart surgery. First, a new image-based visual servoing scheme based on Nonlinear Model Predictive Control is proposed. With this controler, contraints of the system can naturally be taken into account by simultaneously handle the trajectory planification and the control. Experiments on a robotic platform with a classical camera validate the proposed approach. This scheme has also been adapted for echographic vision. It allows to control tool tip motion while ensuring intersection constraint with the echographic plane. Experimental validations highlight the efficientcy of the control scheme. This second part of the work takes place in a project of robotic assisted endovascular surgery, particularly for valve surgery. In this context, the real time localisation of the mitral valve is realized through a new method based on the tracking of the junction of the leaflet and the heart wall and a 2 dof model of the valve. The porposed algorithm has been valided with in vivo sequence. In the last part, in a first evaluation for heart motion compensation, we focus on 3D heart motion estimation using in vivo endoscopic images.Ces travaux de thèse s'inscrivent dans le cadre de l'assistance robotisée pour la chirurgie cardiaque à coeur battant. Une première contribution est la synthèse d'une nouvelle architecture d'asservissement visuel dans l'image en utilisant une commande prédictive non linéaire. Cette structure permet de respecter les contraintes du système en gérant simultanément la planification de trajectoire et le contrôle. La loi de commande a été validée sur une plate-forme robotique en vision standard. La structure a également été adaptée pour la vision échographique. Elle permet ainsi de contrôler les déplacements d'un outil en imposant que celui-ci respecte la contrainte d'intersection avec le plan échographique. Des expérimentations ont également permis de montrer l'efficacité de cette loi de commande. Cette seconde contribution s'intègre dans le cadre de l'assistance à la chirurgie cardiaque endovasculaire, et plus particulièrement la réparation de valve mitrale. Dans ce contexte, la localisation de la valve mitrale en temps réel est obtenue en utilisant une méthode qui repose sur la détection de la jonction entre la paroi et la valve et également d'un modèle à 2 ddl de la valve. L'algorithme proposé a été validé sur une séquence in vivo. Enfin, une estimation in vivo des déplacements 3D du coeur à partir des images endoscopiques a été proposée et représente une première étape d'un projet de compensation automatique des mouvements du coeur pour la chirurgie coronarienne

Downloaded from

LIRMM (Laboratoire d’informatique, de Robotiqu

Towards Nonlinear Model Predictive Control for Image Based Visual Servoing

International audienc

Image Based Visual Servoing through Nonlinear Model Predictive Control

International audienceImage based visual servoing (IBVS) is a vision sensor based control architecture. In classical approach, an image Jacobian matrix maps image space errors into errors in Cartesian space. Then a simple proportional control law can be applied guaranteeing local convergence to a desired set point. One of the main advantage of IBVS is its robustness w.r.t camera and robot calibration errors and image measurement errors. Nevertheless, this scheme can not deal with nonlinear constraint such as joint limits and actuator saturation. Visibility constraint is not ensured with classical IBVS. A new IBVS scheme based on Nonlinear Model Predictive Control (NMPC) is proposed considering the direct dynamic model of the robot, its joint and torque limits, the camera projection model and the visibility constraint. Simulations exhibit the efciency and the robustness of the proposed solution to control a 6 degrees of freedom mechanical system

Three-dimensional heart motion estimation using endoscopic monocular vision system: From artificial landmarks to texture analysis

International audienceIn robot-assisted beating heart surgery, motion of the heart surface might be virtually stabilized to let the surgeon work as in on-pump cardiac surgery. Virtual stabilization means to compensate physically the relative motion between the instrument tool tip and the region of interest on the heart surface, and to offer surgeon a stable visual display of the scene. To this end, motion of the heart must be estimated. This article focusses on motion estimation of the heart surface. Two approaches are considered in the paper. The first one is based on landmark tracking allowing 3D pose estimation. The second is based on texture tracking. Classical computer vision methods, as well as a new texture-based tracking scheme has been applied to track the heart motion, and, when possible, reconstruct 3D distance to the heart surface. Experimental results obtained on in vivo images show the estimated motion of heart surface points

3D Heart Motion Estimation Using Endoscopic Monocular Vision System

International audienceN/

Detection of Micro-Cracks on Highly Specular Reflectors: Dimensioning a Vision Machine Based on Optical Properties

International audienceThis paper describes the complete dimensioning of a method and apparatus dedicated to the automatic detection of defects on optical mirror-like components called optical solar reflector (OSR), which are used to ensure the thermal control of satellites. First, the requirements of the method are defined according to the properties of the OSR (multilayer and semitransparent) and the defects to be detected. The defect dimensions are then measured and exhibit very dispersed micrometric values. Second, the optical characterization of the OSR and its sublayers is conducted, enabling the dimensioning of the machine with regard to the localization of the defect (surface, interface, or volume). Third, a semi-industrial prototype is developed and evaluated through two kinds of experiments. The first experiment illustrates the production capacity of the prototype and the second the measurement capability. Results showed that the prototype exceeded the initial requirements and could be used to replace the currently used control with confidence