Hand-eye calibration with a remote centre of motion

In the eye-in-hand robot configuration, hand-eye calibration plays a vital role in completing the link between the robot and camera coordinate systems. Calibration algorithms are mature and provide accurate transformation estimations for an effective camera-robot link but rely on a sufficiently wide range of calibration data to avoid errors and degenerate configurations. This can be difficult in the context of keyhole surgical robots because they are mechanically constrained to move around a remote centre of motion (RCM) which is located at the trocar port. The trocar limits the range of feasible calibration poses that can be obtained and results in ill-conditioned hand-eye constraints. In this letter, we propose a new approach to deal with this problem by incorporating the RCM constraints into the hand-eye formulation. We show that this not only avoids ill-conditioned constraints but is also more accurate than classic hand-eye calibration with a free 6DoF motion, due to solving simpler equations that take advantage of the reduced DoF. We validate our method using simulation to test numerical stability and a physical implementation on an RCM constrained KUKA LBR iiwa 14 R820 equipped with a NanEye stereo camera

Solving the nearest rotation matrix problem in three and four dimensions with applications in robotics

Aplicat embargament des de la data de defensa fins ei 31/5/2022Since the map from quaternions to rotation matrices is a 2-to-1 covering map, this map cannot be smoothly inverted. As a consequence, it is sometimes erroneously assumed that all inversions should necessarily contain singularities that arise in the form of quotients where the divisor can be arbitrarily small. This misconception was clarified when we found a new division-free conversion method. This result triggered the research work presented in this thesis. At first glance, the matrix to quaternion conversion does not seem to be a relevant problem. Actually, most researchers consider it as a well-solved problem whose revision is not likely to provide any new insight in any area of practical interest. Nevertheless, we show in this thesis how solving the nearest rotation matrix problem in Frobenius norm can be reduced to a matrix to quaternion conversion. Many problems, such as hand-eye calibration, camera pose estimation, location recognition, image stitching etc. require finding the nearest proper orthogonal matrix to a given matrix. Thus, the matrix to quaternion conversion becomes of paramount importance. While a rotation in 3D can be represented using a quaternion, a rotation in 4D can be represented using a double quaternion. As a consequence, the computation of the nearest rotation matrix in 4D, using our approach, essentially follow the same steps as in the 3D case. Although the 4D case might seem of theoretical interest only, we show in this thesis its practical relevance thanks to a little known mapping between 3D displacements and 4D rotations. In this thesis we focus our attention in obtaining closed-form solutions, in particular those that only require the four basic arithmetic operations because they can easily be implemented on microcomputers with limited computational resources. Moreover, closed-form methods are preferable for at least two reasons: they provide the most meaningful answer because they permit analyzing the influence of each variable on the result; and their computational cost, in terms of arithmetic operations, is fixed and assessable beforehand. We have actually derived closed-form methods specifically tailored for solving the hand-eye calibration and the pointcloud registration problems which outperform all previous approaches.Dado que la función que aplica a cada cuaternión su matrix de rotación correspondiente es 2 a 1, la inversa de esta función no es diferenciable en todo su dominio. Por consiguiente, a veces se asume erróneamente que todas las inversiones deben contener necesariamente singularidades que surgen en forma de cocientes donde el divisor puede ser arbitrariamente pequeño. Esta idea errónea se aclaró cuando encontramos un nuevo método de conversión sin división. Este resultado desencadenó el trabajo de investigación presentado en esta tesis. A primera vista, la conversión de matriz a cuaternión no parece un problema relevante. En realidad, la mayoría de los investigadores lo consideran un problema bien resuelto cuya revisión no es probable que proporcione nuevos resultados en ningún área de interés práctico. Sin embargo, mostramos en esta tesis cómo la resolución del problema de la matriz de rotación más cercana según la norma de Frobenius se puede reducir a una conversión de matriz a cuaternión. Muchos problemas, como el de la calibración mano-cámara, el de la estimación de la pose de una cámara, el de la identificación de una ubicación, el del solapamiento de imágenes, etc. requieren encontrar la matriz de rotación más cercana a una matriz dada. Por lo tanto, la conversión de matriz a cuaternión se vuelve de suma importancia. Mientras que una rotación en 3D se puede representar mediante un cuaternión, una rotación en 4D se puede representar mediante un cuaternión doble. Como consecuencia, el cálculo de la matriz de rotación más cercana en 4D, utilizando nuestro enfoque, sigue esencialmente los mismos pasos que en el caso 3D. Aunque el caso 4D pueda parecer de interés teórico únicamente, mostramos en esta tesis su relevancia práctica gracias a una función poco conocida que relaciona desplazamientos en 3D con rotaciones en 4D. En esta tesis nos centramos en la obtención de soluciones de forma cerrada, en particular aquellas que solo requieren las cuatro operaciones aritméticas básicas porque se pueden implementar fácilmente en microcomputadores con recursos computacionales limitados. Además, los métodos de forma cerrada son preferibles por al menos dos razones: proporcionan la respuesta más significativa porque permiten analizar la influencia de cada variable en el resultado; y su costo computacional, en términos de operaciones aritméticas, es fijo y evaluable de antemano. De hecho, hemos derivado nuevos métodos de forma cerrada diseñados específicamente para resolver el problema de la calibración mano-cámara y el del registro de nubes de puntos cuya eficiencia supera la de todos los métodos anteriores.Postprint (published version

Calibration of spatial relationships between multiple robots and sensors

Classic hand-eye calibration methods have been limited to single robots and sensors. Recently a new calibration formulation for multiple robots has been proposed that solves for the extrinsic calibration parameters for each robot simultaneously instead of sequentially. The existing solutions for this new problem required data to have correspondence, but Ma, Goh and Chirikjian (MGC) proposed a probabilistic method to solve this problem which eliminated the need for correspondence. In this thesis, the literature of the various robot-sensor calibration problems and solutions are surveyed, and the MGC method is reviewed in detail. Lastly comparison with other methods using numerical simulations were carried out to draw some conclusions

Hand-eye calibration, constraints and source synchronisation for robotic-assisted minimally invasive surgery

In robotic-assisted minimally invasive surgery (RMIS), the robotic system allows surgeons to remotely control articulated instruments to perform surgical interventions and introduces a potential to implement computer-assisted interventions (CAI). However, the information in the camera must be correctly transformed into the robot coordinate as its movement is controlled by the robot kinematic. Therefore, determining the rigid transformation connecting the coordinates is necessary. Such process is called hand-eye calibration. One of the challenges in solving the hand-eye problem in the RMIS setup is data asynchronicity, which occurs when tracking equipments are integrated into a robotic system and create temporal misalignment. For the calibration itself, noise in the robot and camera motions can be propagated to the calibrated result and as a result of a limited motion range, the error cannot be fully suppressed. Finally, the calibration procedure must be adaptive and simple so a disruption in a surgical workflow is minimal since any change in the setup may require another calibration procedure. We propose solutions to deal with the asynchronicity, noise sensitivity, and a limited motion range. We also propose a potential to use a surgical instrument as the calibration target to reduce the complexity in the calibration procedure. The proposed algorithms are validated through extensive experiments with synthetic and real data from the da Vinci Research Kit and the KUKA robot arms. The calibration performance is compared with existing hand-eye algorithms and it shows promising results. Although the calibration using a surgical instrument as the calibration target still requires a further development, results indicate that the proposed methods increase the calibration performance, and contribute to finding an optimal solution to the hand-eye problem in robotic surgery

Quantization, Calibration and Planning for Euclidean Motions in Robotic Systems

The properties of Euclidean motions are fundamental in all areas of robotics research. Throughout the past several decades, investigations on some low-level tasks like parameterizing specific movements and generating effective motion plans have fostered high-level operations in an autonomous robotic system. In typical applications, before executing robot motions, a proper quantization of basic motion primitives could simplify online computations; a precise calibration of sensor readings could elevate the accuracy of the system controls. Of particular importance in the whole autonomous robotic task, a safe and efficient motion planning framework would make the whole system operate in a well-organized and effective way. All these modules encourage huge amounts of efforts in solving various fundamental problems, such as the uniformity of quantization in non-Euclidean manifolds, the calibration errors on unknown rigid transformations due to the lack of data correspondence and noise, the narrow passage and the curse of dimensionality bottlenecks in developing motion planning algorithms, etc. Therefore, the goal of this dissertation is to tackle these challenges in the topics of quantization, calibration and planning for Euclidean motions