Hand-eye calibration made easy through a closed-form two-stage method

© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksAn analysis of the existing hand-eye calibration methods reveals that most of them are far from trivial. And, what is worse, their intrinsic complexity makes it difficult to elucidate under which circumstances they fail to provide an accurate solution. Thus, although it might seem that hand-eye calibration problem is uninspiring because it is assumed to be well-solved, we show in this paper that there was still room for improvement, both in terms of simplicity and robustness. After reviewing the most representative methods, we analyze the situations in which they fail, and we introduce a simpler closed-form alternative that accurately solves the problem in all the identified critical circumstances. Its performance is evaluated using simulated and real experimental data.Peer ReviewedPostprint (author's final draft

Accuracy evaluation of hand-eye calibration techniques for vision-guided robots

Hand-eye calibration is an important step in controlling a vision-guided robot in applications like part assembly, bin picking and inspection operations etc. Many methods for estimating hand-eye transformations have been proposed in literature with varying degrees of complexity and accuracy. However, the success of a vision-guided application is highly impacted by the accuracy the hand-eye calibration of the vision system with the robot. The level of this accuracy depends on several factors such as rotation and translation noise, rotation and translation motion range that must be considered during calibration. Previous studies and benchmarking of the proposed algorithms have largely been focused on the combined effect of rotation and translation noise. This study provides insight on the impact of rotation and translation noise acting in isolation on the hand-eye calibration accuracy. This deviates from the most common method of assessing hand-eye calibration accuracy based on pose noise (combined rotation and translation noise). We also evaluated the impact of the robot motion range used during the hand-eye calibration operation which is rarely considered. We provide quantitative evaluation of our study using six commonly used algorithms from an implementation perspective. We comparatively analyse the performance of these algorithms through simulation case studies and experimental validation using the Universal Robot’s UR5e physical robots. Our results show that these different algorithms perform differently when the noise conditions vary rather than following a general trend. For example, the simultaneous methods are more resistant to rotation noise, whereas the separate methods are better at dealing with translation noise. Additionally, while increasing the robot rotation motion span during calibration enhances the accuracy of the separate methods, it has a negative effect on the simultaneous methods. Conversely, increasing the translation motion range improves the accuracy of simultaneous methods but degrades the accuracy of the separate methods. These findings suggest that those conditions should be considered when benchmarking algorithms or performing a calibration process for enhanced accuracy

Solving the nearest rotation matrix problem in three and four dimensions with applications in robotics

Aplicat embargament des de la data de defensa fins ei 31/5/2022Since the map from quaternions to rotation matrices is a 2-to-1 covering map, this map cannot be smoothly inverted. As a consequence, it is sometimes erroneously assumed that all inversions should necessarily contain singularities that arise in the form of quotients where the divisor can be arbitrarily small. This misconception was clarified when we found a new division-free conversion method. This result triggered the research work presented in this thesis. At first glance, the matrix to quaternion conversion does not seem to be a relevant problem. Actually, most researchers consider it as a well-solved problem whose revision is not likely to provide any new insight in any area of practical interest. Nevertheless, we show in this thesis how solving the nearest rotation matrix problem in Frobenius norm can be reduced to a matrix to quaternion conversion. Many problems, such as hand-eye calibration, camera pose estimation, location recognition, image stitching etc. require finding the nearest proper orthogonal matrix to a given matrix. Thus, the matrix to quaternion conversion becomes of paramount importance. While a rotation in 3D can be represented using a quaternion, a rotation in 4D can be represented using a double quaternion. As a consequence, the computation of the nearest rotation matrix in 4D, using our approach, essentially follow the same steps as in the 3D case. Although the 4D case might seem of theoretical interest only, we show in this thesis its practical relevance thanks to a little known mapping between 3D displacements and 4D rotations. In this thesis we focus our attention in obtaining closed-form solutions, in particular those that only require the four basic arithmetic operations because they can easily be implemented on microcomputers with limited computational resources. Moreover, closed-form methods are preferable for at least two reasons: they provide the most meaningful answer because they permit analyzing the influence of each variable on the result; and their computational cost, in terms of arithmetic operations, is fixed and assessable beforehand. We have actually derived closed-form methods specifically tailored for solving the hand-eye calibration and the pointcloud registration problems which outperform all previous approaches.Dado que la función que aplica a cada cuaternión su matrix de rotación correspondiente es 2 a 1, la inversa de esta función no es diferenciable en todo su dominio. Por consiguiente, a veces se asume erróneamente que todas las inversiones deben contener necesariamente singularidades que surgen en forma de cocientes donde el divisor puede ser arbitrariamente pequeño. Esta idea errónea se aclaró cuando encontramos un nuevo método de conversión sin división. Este resultado desencadenó el trabajo de investigación presentado en esta tesis. A primera vista, la conversión de matriz a cuaternión no parece un problema relevante. En realidad, la mayoría de los investigadores lo consideran un problema bien resuelto cuya revisión no es probable que proporcione nuevos resultados en ningún área de interés práctico. Sin embargo, mostramos en esta tesis cómo la resolución del problema de la matriz de rotación más cercana según la norma de Frobenius se puede reducir a una conversión de matriz a cuaternión. Muchos problemas, como el de la calibración mano-cámara, el de la estimación de la pose de una cámara, el de la identificación de una ubicación, el del solapamiento de imágenes, etc. requieren encontrar la matriz de rotación más cercana a una matriz dada. Por lo tanto, la conversión de matriz a cuaternión se vuelve de suma importancia. Mientras que una rotación en 3D se puede representar mediante un cuaternión, una rotación en 4D se puede representar mediante un cuaternión doble. Como consecuencia, el cálculo de la matriz de rotación más cercana en 4D, utilizando nuestro enfoque, sigue esencialmente los mismos pasos que en el caso 3D. Aunque el caso 4D pueda parecer de interés teórico únicamente, mostramos en esta tesis su relevancia práctica gracias a una función poco conocida que relaciona desplazamientos en 3D con rotaciones en 4D. En esta tesis nos centramos en la obtención de soluciones de forma cerrada, en particular aquellas que solo requieren las cuatro operaciones aritméticas básicas porque se pueden implementar fácilmente en microcomputadores con recursos computacionales limitados. Además, los métodos de forma cerrada son preferibles por al menos dos razones: proporcionan la respuesta más significativa porque permiten analizar la influencia de cada variable en el resultado; y su costo computacional, en términos de operaciones aritméticas, es fijo y evaluable de antemano. De hecho, hemos derivado nuevos métodos de forma cerrada diseñados específicamente para resolver el problema de la calibración mano-cámara y el del registro de nubes de puntos cuya eficiencia supera la de todos los métodos anteriores.Postprint (published version

Probabilistic Calibration and Catheter Tracking with Robotic Systems

A significant boost in robotics technology has been observed in recent years and more and more tasks are being automated by robots such as robotic surgery, autonomous driving, package delivery, etc. Not only has the precision of robots been improved, but the number of robots involved in a specific task has also grown in many scenarios. An important part in a robotic automated task involves the relative pose estimation among objects, and this often boils down to calibration and tracking. The dissertation begins with a robotic catheter tracking system and then focuses on calibration of robotic systems. The presentation first introduces a novel robotic catheter tracking system which uses an embedded active piezoelectric element at the tip of the catheter. Catheter intervention procedure is performed exclusively with X-ray, while ultrasound comes as an alternative modality which is radiation free. However, the catheter tip is usually very small and hard to be differentiated from human tissue in an ultrasound image. Moreover, an ultrasound photographer needs to hold the ultrasound probe during the procedure which can easily last for over an hour. The proposed system can tackle these issues using a robot arm and the active echo signal, and is, to the best knowledge of the author, the first robotic catheter tracking system using ultrasound. It is demonstrated in both the simulation and experiment that a robotic arm holding the ultrasound probe can track the catheter tip without image input. To better assist the tracking process, other procedures can be automated such as catheter insertion and phantom localization, etc. All these require introducing an extra robot and a precise calibration between robots and targets of interest. Out of many calibration approaches, the most classical one is called the hand-eye calibration problem formulated as AX = XB which takes in data from sensors in different locations to solve for an unknown rigid-body transformation. A generalization of this problem is the AX = YB robot-world and hand-eye calibration, where two unknowns need to be recovered simultaneously. The above two approaches mainly deal with the calibration of a single robot system. For multi-robot systems, a problem cast as the AXB = YCZ formulation arises where three unknowns need to be solved given three sensor data streams. The second portion of the presentation investigates in the probabilistic approaches toward all three problems above. Different methods based on the probabilistic theory on Lie group are developed to show their superior performance over non-probabilistic equivalents when there is partial knowledge of the correspondence among sensor data