Quasi-static Soft Fixture Analysis of Rigid and Deformable Objects

We present a sampling-based approach to reasoning about the caging-based manipulation of rigid and a simplified class of deformable 3D objects subject to energy constraints. Towards this end, we propose the notion of soft fixtures extending earlier work on energy-bounded caging to include a broader set of energy function constraints and settings, such as gravitational and elastic potential energy of 3D deformable objects. Previous methods focused on establishing provably correct algorithms to compute lower bounds or analytically exact estimates of escape energy for a very restricted class of known objects with low-dimensional C-spaces, such as planar polygons. We instead propose a practical sampling-based approach that is applicable in higher-dimensional C-spaces but only produces a sequence of upper-bound estimates that, however, appear to converge rapidly to actual escape energy. We present 8 simulation experiments demonstrating the applicability of our approach to various complex quasi-static manipulation scenarios. Quantitative results indicate the effectiveness of our approach in providing upper-bound estimates for escape energy in quasi-static manipulation scenarios. Two real-world experiments also show that the computed normalized escape energy estimates appear to correlate strongly with the probability of escape of an object under randomized pose perturbation.Comment: Paper submitted to ICRA 202

Sampling-Based Motion Planning: A Comparative Review

Sampling-based motion planning is one of the fundamental paradigms to generate robot motions, and a cornerstone of robotics research. This comparative review provides an up-to-date guideline and reference manual for the use of sampling-based motion planning algorithms. This includes a history of motion planning, an overview about the most successful planners, and a discussion on their properties. It is also shown how planners can handle special cases and how extensions of motion planning can be accommodated. To put sampling-based motion planning into a larger context, a discussion of alternative motion generation frameworks is presented which highlights their respective differences to sampling-based motion planning. Finally, a set of sampling-based motion planners are compared on 24 challenging planning problems. This evaluation gives insights into which planners perform well in which situations and where future research would be required. This comparative review thereby provides not only a useful reference manual for researchers in the field, but also a guideline for practitioners to make informed algorithmic decisions.Comment: 25 pages, 7 figures, Accepted for Volume 7 (2024) of the Annual Review of Control, Robotics, and Autonomous System

Studies on Spinal Fusion from Computational Modelling to ‘Smart’ Implants

Low back pain, the worldwide leading cause of disability, is commonly treated with lumbar interbody fusion surgery to address degeneration, instability, deformity, and trauma of the spine. Following fusion surgery, nearly 20% experience complications requiring reoperation while 1 in 3 do not experience a meaningful improvement in pain. Implant subsidence and pseudarthrosis in particular present a multifaceted challenge in the management of a patient’s painful symptoms. Given the diversity of fusion approaches, materials, and instrumentation, further inputs are required across the treatment spectrum to prevent and manage complications. This thesis comprises biomechanical studies on lumbar spinal fusion that provide new insights into spinal fusion surgery from preoperative planning to postoperative monitoring. A computational model, using the finite element method, is developed to quantify the biomechanical impact of temporal ossification on the spine, examining how the fusion mass stiffness affects loads on the implant and subsequent subsidence risk, while bony growth into the endplates affects load-distribution among the surrounding spinal structures. The computational modelling approach is extended to provide biomechanical inputs to surgical decisions regarding posterior fixation. Where a patient is not clinically pre-disposed to subsidence or pseudarthrosis, the results suggest unilateral fixation is a more economical choice than bilateral fixation to stabilise the joint. While finite element modelling can inform pre-surgical planning, effective postoperative monitoring currently remains a clinical challenge. Periodic radiological follow-up to assess bony fusion is subjective and unreliable. This thesis describes the development of a ‘smart’ interbody cage capable of taking direct measurements from the implant for monitoring fusion progression and complication risk. Biomechanical testing of the ‘smart’ implant demonstrated its ability to distinguish between graft and endplate stiffness states. The device is prepared for wireless actualisation by investigating sensor optimisation and telemetry. The results show that near-field communication is a feasible approach for wireless power and data transfer in this setting, notwithstanding further architectural optimisation required, while a combination of strain and pressure sensors will be more mechanically and clinically informative. Further work in computational modelling of the spine and ‘smart’ implants will enable personalised healthcare for low back pain, and the results presented in this thesis are a step in this direction

Learning to represent surroundings, anticipate motion and take informed actions in unstructured environments

Contemporary robots have become exceptionally skilled at achieving specific tasks in structured environments. However, they often fail when faced with the limitless permutations of real-world unstructured environments. This motivates robotics methods which learn from experience, rather than follow a pre-defined set of rules. In this thesis, we present a range of learning-based methods aimed at enabling robots, operating in dynamic and unstructured environments, to better understand their surroundings, anticipate the actions of others, and take informed actions accordingly

Path Signatures for Diversity in Probabilistic Trajectory Optimisation

Motion planning can be cast as a trajectory optimisation problem where a cost is minimised as a function of the trajectory being generated. In complex environments with several obstacles and complicated geometry, this optimisation problem is usually difficult to solve and prone to local minima. However, recent advancements in computing hardware allow for parallel trajectory optimisation where multiple solutions are obtained simultaneously, each initialised from a different starting point. Unfortunately, without a strategy preventing two solutions to collapse on each other, naive parallel optimisation can suffer from mode collapse diminishing the efficiency of the approach and the likelihood of finding a global solution. In this paper we leverage on recent advances in the theory of rough paths to devise an algorithm for parallel trajectory optimisation that promotes diversity over the range of solutions, therefore avoiding mode collapses and achieving better global properties. Our approach builds on path signatures and Hilbert space representations of trajectories, and connects parallel variational inference for trajectory estimation with diversity promoting kernels. We empirically demonstrate that this strategy achieves lower average costs than competing alternatives on a range of problems, from 2D navigation to robotic manipulators operating in cluttered environments

Meta-Routing: Synergistic Merging of Message Routing and Link Maintenance

The maintenance of network connectivity is essential for effective and efficient mobile team operations. Achieving robust mobile ad hoc networks (MANETs) connectivity requires a capable link maintenance mechanism especially if the network experiences expected intermittent connectivity due to a hostile environment. One applicable example of such network scenarios is multi-robot exploration for urban search and rescue (USAR). With the proliferation of these robotic networks, communication problems such as the link maintenance problem are subject to be raised quickly. Although various routing protocols for wireless ad hoc networks have been proposed, they solve the problems of message routing and link maintenance separately, resulting in additional overhead costs and long latency in network communication. Traditional routing protocols discover existing links, connect these links, find the best path and minimize the path cost. The limitation of previous routing protocols motivates us to develop a new concept of routing mechanism for a robotic network. This routing mechanism is named Meta-Routing. Meta-Routing expands current routing protocols to include not only the normal routing of packets, but also the maintenance of links in mobile agent scenarios. Thus, Meta-Routing minimizes the communication path cost and the overhead cost, the latter of which results from discovering a route, repairing a link or establishing a new communication path between nodes. This dissertation presents a method to achieve Meta-Routing by controlling robot motion based on the radio frequency (RF) environment recognition method and gradient descent method. Mobile robot controlled motion can effectively improve network performance by driving robots to favorable locations with strong links. Moreover, the gradient descent method is used in driving the robots into the direction of favorable positions for maximizing broken or failing links and maintaining network connectivity. The main accomplished goals of this thesis are summarized as follows: firstly, the Meta-Routing protocol, which integrates link maintenance into the normal message routing protocol cost function; secondly, the dissertation examines the unification of the syntax of message routing protocol and the link maintenance process through physical configuration of mobile network nodes by controlling their movement in the field; finally, the dissertation demonstrates that the utilization of the RF environment recognition and classification method improves route repair estimation for achieving link maintenance in the presented Meta-Routing protocol. The numerical experimental results demonstrate promising RF environment recognition and node controlled motion results, as well as confirm their abilities in robot movement control for link maintenance and reduction of the total path cost

Sequential Trajectory Re-planning with Tactile Information Gain for Dexterous Grasping under Object-pose Uncertainty

Abstract — Dexterous grasping of objects with uncertain pose is a hard unsolved problem in robotics. This paper solves this problem using information gain re-planning. First we show how tactile information, acquired during a failed attempt to grasp an object can be used to refine the estimate of that object’s pose. Second, we show how this information can be used to replan new reach to grasp trajectories for successive grasp attempts. Finally we show how reach-to-grasp trajectories can be modified, so that they maximise the expected tactile information gain, while simultaneously delivering the hand to the grasp configuration that is most likely to succeed. Our main novel outcome is thus to enable tactile information gain planning for Dexterous, high degree of freedom (DoFs) manipulators. We achieve this using a combination of information gain planning, hierarchical probabilistic roadmap planning, and belief updating from tactile sensors for objects with non-Gaussian pose uncertainty in 6 dimensions. The method is demonstrated in trials with simulated robots. Sequential replanning is shown to achieve a greater success rate than single grasp attempts, and trajectories that maximise information gain require fewer re-planning iterations than conventional planning methods before a grasp is achieved. I

Robust navigation for industrial service robots

Pla de Doctorats Industrials de la Generalitat de CatalunyaRobust, reliable and safe navigation is one of the fundamental problems of robotics. Throughout the present thesis, we tackle the problem of navigation for robotic industrial mobile-bases. We identify its components and analyze their respective challenges in order to address them. The research work presented here ultimately aims at improving the overall quality of the navigation stack of a commercially available industrial mobile-base. To introduce and survey the overall problem we first break down the navigation framework into clearly identified smaller problems. We examine the Simultaneous Localization and Mapping (SLAM) problem, recalling its mathematical grounding and exploring the state of the art. We then review the problem of planning the trajectory of a mobile-base toward a desired goal in the generated environment representation. Finally we investigate and clarify the use of the subset of the Lie theory that is useful in robotics. The first problem tackled is the recognition of place for closing loops in SLAM. Loop closure refers to the ability of a robot to recognize a previously visited location and infer geometrical information between its current and past locations. Using only a 2D laser range finder sensor, we address the problem using a technique borrowed from the field of Natural Language Processing (NLP) which has been successfully applied to image-based place recognition, namely the Bag-of-Words. We further improve the method with two proposals inspired from NLP. Firstly, the comparison of places is strengthened by considering the natural relative order of features in each individual sensor reading. Secondly, topological correspondences between places in a corpus of visited places are established in order to promote together instances that are ‘close’ to one another. We then tackle the problem of motion model calibration for odometry estimation. Given a mobile-base embedding an exteroceptive sensor able to observe ego-motion, we propose a novel formulation for estimating the intrinsic parameters of an odometry motion model. Resorting to an adaptation of the pre-integration theory initially developed for inertial motion sensors, we employ iterative nonlinear on-manifold optimization to estimate the wheel radii and wheel separation. The method is further extended to jointly estimate both the intrinsic parameters of the odometry model together with the extrinsic parameters of the embedded sensor. The method is shown to accommodate to variation in model parameters quickly when the vehicle is subject to physical changes during operation. Following the generation of a map in which the robot is localized, we address the problem of estimating trajectories for motion planning. We devise a new method for estimating a sequence of robot poses forming a smooth trajectory. Regardless of the Lie group considered, the trajectory is seen as a collection of states lying on a spline with non-vanishing n-th derivatives at each point. Formulated as a multi-objective nonlinear optimization problem, it allows for the addition of cost functions such as velocity and acceleration limits, collision avoidance and more. The proposed method is evaluated for two different motion planning tasks, the planning of trajectories for a mobile-base evolving in the SE(2) manifold, and the planning of the motion of a multi-link robotic arm whose end-effector evolves in the SE(3) manifold. From our study of Lie theory, we developed a new, ready to use, programming library called `manif’. The library is open source, publicly available and is developed following good software programming practices. It is designed so that it is easy to integrate and manipulate, and allows for flexible use while facilitating the possibility to extend it beyond the already implemented Lie groups.La navegación autónoma es uno de los problemas fundamentales de la robótica, y sus diferentes desafíos se han estudiado durante décadas. El desarrollo de métodos de navegación robusta, confiable y segura es un factor clave para la creación de funcionalidades de nivel superior en robots diseñados para operar en entornos con humanos. A lo largo de la presente tesis, abordamos el problema de navegación para bases robóticas móviles industriales; identificamos los elementos de un sistema de navegación; y analizamos y tratamos sus desafíos. El trabajo de investigación presentado aquí tiene como último objetivo mejorar la calidad general del sistema completo de navegación de una base móvil industrial disponible comercialmente. Para estudiar el problema de navegación, primero lo desglosamos en problemas menores claramente identificados. Examinamos el subproblema de mapeo del entorno y localización del robot simultáneamente (SLAM por sus siglas en ingles) y estudiamos el estado del arte del mismo. Al hacerlo, recordamos y detallamos la base matemática del problema de SLAM. Luego revisamos el subproblema de planificación de trayectorias hacia una meta deseada en la representación del entorno generada. Además, como una herramienta para las soluciones que se presentarán más adelante en el desarrollo de la tesis, investigamos y aclaramos el uso de teoría de Lie, centrándonos en el subconjunto de la teoría que es útil para la estimación de estados en robótica. Como primer elemento identificado para mejoras, abordamos el problema de reconocimiento de lugares para cerrar lazos en SLAM. El cierre de lazos se refiere a la capacidad de un robot para reconocer una ubicación visitada previamente e inferí información geométrica entre la ubicación actual del robot y aquellas reconocidas. Usando solo un sensor láser 2D, la tarea es desafiante ya que la percepción del entorno que proporciona el sensor es escasa y limitada. Abordamos el problema utilizando 'bolsas de palabras', una técnica prestada del campo de procesamiento del lenguaje natural (NLP) que se ha aplicado con éxito anteriormente al reconocimiento de lugares basado en imágenes. Nuestro método incluye dos nuevas propuestas inspiradas también en NLP. Primero, la comparación entre lugares candidatos se fortalece teniendo en cuenta el orden relativo natural de las características en cada lectura individual del sensor; y segundo, se establece un corpus de lugares visitados para promover juntos instancias que están "cerca" la una de la otra desde un punto de vista topológico. Evaluamos nuestras propuestas por separado y conjuntamente en varios conjuntos de datos, con y sin ruido, demostrando mejora en la detección de cierres de lazo para sensores láser 2D, con respecto al estado del arte. Luego abordamos el problema de la calibración del modelo de movimiento para la estimación de la edometría. Dado que nuestra base móvil incluye un sensor exteroceptivo capaz de observar el movimiento de la plataforma, proponemos una nueva formulación que permite estimar los parámetros intrínsecos del modelo cinemático de la plataforma durante el cómputo de la edometría del vehículo. Hemos recurrido a una adaptación de la teoría de reintegración inicialmente desarrollado para unidades inerciales de medida, y aplicado la técnica a nuestro modelo cinemático. El método nos permite, mediante optimización iterativa no lineal, la estimación del valor del radio de las ruedas de forma independiente y de la separación entre las mismas. El método se amplía posteriormente par idéntica de forma simultánea, estos parámetros intrínsecos junto con los parámetros extrínsecos que ubican el sensor láser con respecto al sistema de referencia de la base móvil. El método se valida en simulación y en un entorno real y se muestra que converge hacia los verdaderos valores de los parámetros. El método permite la adaptación de los parámetros intrínsecos del modelo cinemático de la plataforma derivados de cambios físicos durante la operación, tales como el impacto que el cambio de carga sobre la plataforma tiene sobre el diámetro de las ruedas. Como tercer subproblema de navegación, abordamos el reto de planificar trayectorias de movimiento de forma suave. Desarrollamos un método para planificar la trayectoria como una secuencia de configuraciones sobre una spline con n-ésimas derivadas en todos los puntos, independientemente del grupo de Lie considerado. Al ser formulado como un problema de optimización no lineal con múltiples objetivos, es posible agregar funciones de coste al problema de optimización que permitan añadir límites de velocidad o aceleración, evasión de colisiones, etc. El método propuesto es evaluado en dos tareas de planificación de movimiento diferentes, la planificación de trayectorias para una base móvil que evoluciona en la variedad SE(2), y la planificación del movimiento de un brazo robótico cuyo efector final evoluciona en la variedad SE(3). Además, cada tarea se evalúa en escenarios con complejidad de forma incremental, y se muestra un rendimiento comparable o mejor que el estado del arte mientras produce resultados más consistentes. Desde nuestro estudio de la teoría de Lie, desarrollamos una nueva biblioteca de programación llamada “manif”. La biblioteca es de código abierto, está disponible públicamente y se desarrolla siguiendo las buenas prácticas de programación de software. Esta diseñado para que sea fácil de integrar y manipular, y permite flexibilidad de uso mientras se facilita la posibilidad de extenderla más allá de los grupos de Lie inicialmente implementados. Además, la biblioteca se muestra eficiente en comparación con otras soluciones existentes. Por fin, llegamos a la conclusión del estudio de doctorado. Examinamos el trabajo de investigación y trazamos líneas para futuras investigaciones. También echamos un vistazo en los últimos años y compartimos una visión personal y experiencia del desarrollo de un doctorado industrial.Postprint (published version

Shape-correlated statistical modeling and analysis for respiratory motion estimation

Respiratory motion challenges image-guided radiation therapy (IGRT) with location uncertainties of important anatomical structures in the thorax. Effective and accurate respiration estimation is crucial to account for the motion effects on the radiation dose to tumors and organs at risk. Moreover, serious image artifacts present in treatment-guidance images such 4D cone-beam CT cause difficulties in identifying spatial variations. Commonly used non-linear dense image matching methods easily fail in regions where artifacts interfere. Learning-based linear motion modeling techniques have the advantage of incorporating prior knowledge for robust motion estimation. In this research shape-correlation deformation statistics (SCDS) capture strong correlations between the shape of the lung and the dense deformation field under breathing. Dimension reduction and linear regression techniques are used to extract the correlation statistics. Based on the assumption that the deformation correlations are consistent between planning and treatment time, patient-specific SCDS trained from a 4D planning image sequence is used to predict the respiratory motion in the patient's artifact-laden 4D treatment image sequence. Furthermore, a prediction-driven atlas formation method is developed to weaken the consistency assumption, by integrating intensity information from the target images and the SCDS predictions into a common optimization framework. The strategy of balancing between the prediction constraints and the intensity-matching forces makes the method less sensitive to variation in the correlation and utilizes intensity information besides the lung boundaries. This strategy thus provides improved motion estimation accuracy and robustness. The SCDS-based methods are shown to be effective in modeling and estimating respiratory motion in lung, with evaluations and comparisons carried out on both simulated images and patient images