Allan, M

Hailes, S

Pachtrachai, K

Pawar, V

Stoyanov, D

UCL Discovery

Hand-eye Calibration Using Instrument CAD Models in Robotic Assisted Minimally Invasive Surgery  Krittin Pachtrachai, Max Allan, Vijay Pawar, Stephen Hailes and Danail Stoyanov  Centre for Medical Image Computing (CMIC), University College London  krittin.pachtrachai.13@ucl.ac.uk, maximilian.allan.11@ucl.ac.uk, v.pawar@ucl.ac.uk,  danail.stoyanov@ucl.ac.uk   INTRODUCTION Robot-assisted minimally invasive surgery (RMIS) enables the remote control of surgical instruments with a high degree of safety and accuracy. RMIS is performed by using articulated surgical instruments, including a laparoscopic stereo camera inserted through keyhole access ports [1]. Accurate and real-time localisation of the surgical instruments in a camera reference is an important step towards developing new applications in robotic surgery such as automatic instrument motion control with visual servo and assisted instrument guidance with augmented reality. For such applications to work, the robot kinematics needs to be used to add robustness to visual tracking and this requires accurate hand-eye calibration in order to correctly overlay the information from camera to robot frame.   Hand-eye calibration is the estimation of the rigid transformation linking a camera reference frame and an end-effector frame of a robot. The conventional setup for the calibration is illustrated in Figure 1. The world coordinate framegridF is fixed at the calibration grid, while the robot coordinate framebaseF is at the base of the robot. Camera calibration and forward kinematics are applied to identify the rigid transformationscamgridT andbaserobotT , respectively. In order to perform camera calibration, most existing hand-eye calibration methods involve the use of a known-dimensional object as a calibration target. Mostly used calibration objects are checkerboards or uniform grids with circle dots and their physical dimensions are priorly known. The principle behind this estimation is to identify a homography from which the pose of camera with respect to a calibration target known as extrinsic parameters can be extracted [3]. Although these calibration targets usually provide accurate data for hand-eye calibration, the use of these markers is time consuming and inconvenient in surgical applications. To handle this problem, structure-from-motion (SFM) approaches could be used  to calibrate the hand-eye matrix without using any calibration target [4], however, anatomical features in surgery can deform which makes the problem ill-posed and in addition, the movement of surgical camera is confined by remote centre of motion which prevents the collection of sufficient viewpoints to provide accurate calibration.   Figure 1. The conventional set-up for hand-eye calibration with a da Vinci Surgical Robot. Hand-eye calibration identifies the relative pose between camera frame and end-effector frame camrobotT  [2]. Alternatively, surgical instruments can be used as calibration targets for hand-eye calibration. Their physical dimensions are priorly known and they also have greater range of motion than the camera which give an advantage over checkerboards and uniform grids. Many existing methods are proposed for instrument tracking application, using differents features such as color or local gradient from the target to align 3D model of the instrument with the image [5]. This paper introduces a new approach for hand-eye calibration which uses a surgical instrument as a calibration target. The instrument is tracked while it moves within a field of view of the camera. We use a 3D instrument tracking method in [5] and demonstrate through experiments that hand-eye calibration using surgical tool tracking achieves higher accuracy in rotation than using a conventional calibration object.   METHODS  Hand-eye calibration is to solve for (3)SEX in the following mathematical equation.  AX XB                                   ( 1 ) In a conventional case, A and B are relative motion of camera and robot, but in our case where we use a surgical tool as a calibration target, we modify the relative transformations to be  arm arm 1base base( )( ( ))  A T T                  ( 2 )tool tool 1cam cam( )( ( ))  B T T                   ( 3 ) Figure 2. The schematic for a hand-eye calibration incorporating a tool tracking algorithm as mathematically represented in Eq. 2-3.   where  and   are discrete parameters indicating that the two transformations with different poses as shown in Figure 2. During the capturing, the surgical tool is tracked using the method from [5] which tracks the instrument by minimising the joint cost between aligning 3D model of the instrument with color-based segmentation and a local optical flow point tracking. The optimiser uses gradient descent for the stereo camera to create stereo constraints and Kalman filter for temporal consistency in frame-to-frame tracking. Da Vinci kinematics data is used to create A in Eq. 2 while tracking data is used in Eq. 3. Since da Vinci kinematics are noisy, we introduce an additional constraint derived from the property of the adjoint transformation to the problem to compensate the inaccuracy and this allows the alternate optimisation between the estimations of rotation (Eq. 4) and translation components (Eq. 5).   0 00 0 3( )[ ] ( )00 ( [ ] )[ ] [ [ ] ]TTA A BA A B A A Ba b a ba b a b a bv t vv t v v t v                     Iq    ( 4 ) [ ]A B At v v   R                            ( 5 ) where q , 0[ , ]a a and 0[ , ]b b  are quaternion representations of rotation components of arm toolT , A and B , respectively, ,A Av  and ,B Bv  are Lie algebra of A and B and t is the translation component of arm toolT . The algorithm solves these two equations alternately until the solution converges. The solution arm toolT allows us to finally compute the hand-eye matrix X . RESULTS  We evaluate the performance of the calibration by using the prediction method originally used in [6]. The experiment is performed by collecting 20 poses of a surgical tool from tracking data as well as the kinematic data, but only N poses are included into the calibration (N is run from 3 to 14, i.e. 2 to 13 motions).  Figure 3. Comparison of calibration performance between using a standard grid and a surgical instrument as a calibration target.   This process is repeated 100 times to generate meaningful results. According to Figure 3, using a surgical tool as a calibration target has a clear improvement in rotation estimation. Although the conventional method still outperforms tool tracking method in translation estimation, the result from using tool tracking has a comparable error.  CONCLUSION   In this paper, we propose a hand-eye calibration method using an unconventional calibration target, i.e. a robotic surgical instrument [7], [8]. In RMIS using the da Vinci Surgical Robot where camera motion is confined to a small volume, capturing an image of checkerboard does not provide sufficient viewpoints for the calibration. On the other hand, a surgical instrument has wider range of motion and thus can provide wider range of poses for the calibration which allows the calibration to achieve higher accuracy. The result shows a clear improvement in rotation estimation and a comparable error in translation estimation, after several data are included. Moreover, apart from the improved calibration accuracy itself, the use of a surgical tool as a calibration target potentially allows online and real-time calibration. The approach is also more practical than using a conventional calibration target and introduces the possibility of automatic calibration in computer assisted interventions which will offer a simpler workflow for calibrations during surgical procedures [9], [10].  ACKNOWLEDGEMENT  We would like to thank Simon Di Maio and Mahdi Azizian at Intuitive Surgical, CA for providing us with CAD models of the instrument and the DVRK community for their support. The work was part funded by the EPSRC (EP/N013220/1, EP/N022750/1, EP/N027078/1, NS/A000027/1), The Wellcome Trust (WT101957, 201080/Z/16/Z) and the EU-Horizon 2020 project EndoVESPA (H2020-ICT-2015-688592).  REFERENCES  [1] L. R. Kavoussi et al., “Comparison of robotic versus human laparoscopic camera control”, in J Urol. 1995; 158(4):  2134-2136. [2]  Intuitive Surgical, Inc. ISI API User Guide, 2014. [3]  S. J. D. Prince, Computer Vision Models, Learning and Inference, 1st ed. UK: Cambridge University Press, 2012. [4]   J. Heller et al., “Structure-from-motion based hand-eye calibration using L minimization”, in IEEE CVPR, 2011, pp. 3497-3503. [5]  M. Allan et al., “Image based surgical instrument pose estimation with multi-class labelling and optical flow”, in MICCAI, 2015, pp. 331-338. [6] K. Daniilidis, “Hand-eye calibration using dual quaternions”, in IJRR 1999; 18(3): 286-298. [7]   F. Vasconcelos et al., “Spatial calibration of a 2D/3D ultrasound using a tracked needle”, in IJCARS, 2016; 11(6): 1091-1099. [8] F. Vasconcelos et al., “Similarity registration problems for 2D/3D ultrasound calibration”, in ECCV, 2016 [Accepted]. Available: http://arxiv.org/abs/1608.00247. [9]  D. Stoyanov et al., “Laparoscope self-calibration for robotic assisted minimally invasive surgery”, in MICCAI, 2005, pp. 114-121.  [10] S. Thompson et al., “Hand-eye calibration for rigid laparoscopes using an invariant point”, in IJCARS, 2016; 11(6): 1071-1080.  

Hand-eye Calibration Using Instrument CAD Models in Robotic Assisted Minimally Invasive Surgery

http://discovery.ucl.ac.uk/1516221/1/Pachtrachai%20et%20al%202016%20Hand-eye%20Calibration%20Using%20Instrument%20CAD%20Models.pdf

Hand-eye Calibration Using Instrument CAD Models in Robotic Assisted Minimally Invasive Surgery

Abstract

Similar works

Full text

Available Versions

UCL Discovery