Portable dVRK: an augmented V-REP simulator of the da Vinci Research Kit

The da Vinci Research Kit (dVRK) is a first generation da Vinci robot repurposed as a research platform and coupled with software and controllers developed by research users. An already quite wide community is currently sharing the dVRK (32 systems in 28 sites worldwide). The access to the robotic system for training surgeons and for developing new surgical procedures, tools and new control modalities is still difficult due to the limited availability and high maintenance costs. The development of simulation tools provides a low cost, easy and safe alternative to the use of the real platform for preliminary research and training activities. The Portable dVRK, which is described in this work, is based on a V-REP simulator of the dVRK patient side and endoscopic camera manipulators which are controlled through two haptic interfaces and a 3D viewer, respectively. The V-REP simulator is augmented with a physics engine allowing to render the interaction of new developed tools with soft objects. Full integration in the ROS control architecture makes the simulator flexible and easy to be interfaced with other possible devices. Several scenes have been implemented to illustrate performance and potentials of the developed simulator

Vision-based methods for state estimation and control of robotic systems with application to mobile and surgical robots

For autonomous systems that need to perceive the surrounding environment for the accomplishment of a given task, vision is a highly informative exteroceptive sensory source. When gathering information from the available sensors, in fact, the richness of visual data allows to provide a complete description of the environment, collecting geometrical and semantic information (e.g., object pose, distances, shapes, colors, lights). The huge amount of collected data allows to consider both methods exploiting the totality of the data (dense approaches), or a reduced set obtained from feature extraction procedures (sparse approaches). This manuscript presents dense and sparse vision-based methods for control and sensing of robotic systems. First, a safe navigation scheme for mobile robots, moving in unknown environments populated by obstacles, is presented. For this task, dense visual information is used to perceive the environment (i.e., detect ground plane and obstacles) and, in combination with other sensory sources, provide an estimation of the robot motion with a linear observer. On the other hand, sparse visual data are extrapolated in terms of geometric primitives, in order to implement a visual servoing control scheme satisfying proper navigation behaviours. This controller relies on visual estimated information and is designed in order to guarantee safety during navigation. In addition, redundant structures are taken into account to re-arrange the internal configuration of the robot and reduce its encumbrance when the workspace is highly cluttered. Vision-based estimation methods are relevant also in other contexts. In the field of surgical robotics, having reliable data about unmeasurable quantities is of great importance and critical at the same time. In this manuscript, we present a Kalman-based observer to estimate the 3D pose of a suturing needle held by a surgical manipulator for robot-assisted suturing. The method exploits images acquired by the endoscope of the robot platform to extrapolate relevant geometrical information and get projected measurements of the tool pose. This method has also been validated with a novel simulator designed for the da Vinci robotic platform, with the purpose to ease interfacing and employment in ideal conditions for testing and validation. The Kalman-based observers mentioned above are classical passive estimators, whose system inputs used to produce the proper estimation are theoretically arbitrary. This does not provide any possibility to actively adapt input trajectories in order to optimize specific requirements on the performance of the estimation. For this purpose, active estimation paradigm is introduced and some related strategies are presented. More specifically, a novel active sensing algorithm employing visual dense information is described for a typical Structure-from-Motion (SfM) problem. The algorithm generates an optimal estimation of a scene observed by a moving camera, while minimizing the maximum uncertainty of the estimation. This approach can be applied to any robotic platforms and has been validated with a manipulator arm equipped with a monocular camera

Contact aware robust semi-autonomous teleoperation of mobile manipulators

In the context of human-robot collaboration, cooperation and teaming, the use of mobile manipulators is widespread on applications involving unpredictable or hazardous environments for humans operators, like space operations, waste management and search and rescue on disaster scenarios. Applications where the manipulator's motion is controlled remotely by specialized operators. Teleoperation of manipulators is not a straightforward task, and in many practical cases represent a common source of failures. Common issues during the remote control of manipulators are: increasing control complexity with respect the mechanical degrees of freedom; inadequate or incomplete feedback to the user (i.e. limited visualization or knowledge of the environment); predefined motion directives may be incompatible with constraints or obstacles imposed by the environment. In the latter case, part of the manipulator may get trapped or blocked by some obstacle in the environment, failure that cannot be easily detected, isolated nor counteracted remotely. While control complexity can be reduced by the introduction of motion directives or by abstraction of the robot motion, the real-time constraint of the teleoperation task requires the transfer of the least possible amount of data over the system's network, thus limiting the number of physical sensors that can be used to model the environment. Therefore, it is of fundamental to define alternative perceptive strategies to accurately characterize different interaction with the environment without relying on specific sensory technologies. In this work, we present a novel approach for safe teleoperation, that takes advantage of model based proprioceptive measurement of the robot dynamics to robustly identify unexpected collisions or contact events with the environment. Each identified collision is translated on-the-fly into a set of local motion constraints, allowing the exploitation of the system redundancies for the computation of intelligent control laws for automatic reaction, without requiring human intervention and minimizing the disturbance of the task execution (or, equivalently, the operator efforts). More precisely, the described system consist in two different building blocks. The first, for detecting unexpected interactions with the environment (perceptive block). The second, for intelligent and autonomous reaction after the stimulus (control block). The perceptive block is responsible of the contact event identification. In short, the approach is based on the claim that a sensorless collision detection method for robot manipulators can be extended to the field of mobile manipulators, by embedding it within a statistical learning framework. The control deals with the intelligent and autonomous reaction after the contact or impact with the environment occurs, and consist on an motion abstraction controller with a prioritized set of constrains, where the highest priority correspond to the robot reconfiguration after a collision is detected; when all related dynamical effects have been compensated, the controller switch again to the basic control mode

Design of hybrid-kinematic mechanisms for machine tools

The machine tool industry is a well established, old and extremely important branch of today's manufacturing industry. With the ongoing globalization and the resulting increase of competition in this industry, the manufacturers have to push their technology to the limits in order to stay competitive. The architecture (kinematics) of most machine tools is based on a serial arrangement of joints and segments, like a human arm. The requirements regarding dynamics, stiffness and precision of these machines brought the scientists and industries to evaluate parallel kinematics for this type of application. Parallel kinematics possess a much higher potential to fulfill these demands, and they would therefore allow the access to a next level of machine performance. Whereas the success of parallel kinematics in domains like packaging is incontestable, it proved to be less evident in machine tools. The low rotation amplitudes and the complexity of the mechanism, the main weak points of parallel kinematics, slow down the development and integration of this kind of machines. In the last few years however, we could observe an increase in development, and more important, in the sales (1)(37)(54) of hybrid kinematic machines. Hybrid kinematics can, by appropriate combination of parallel and serial axes, present a well performing compromise, especially in the machine tool domain where 5 axes/mobilities and high rotation amplitudes are common. The present document is concerned with the mechanical, industrialized design of hybrid-kinematic machine tools and their mechanical elements, and will show that "Hybrid-kinematic mechanisms can outperform fully-parallel mechanisms considering all attributes for a successful and industrialized machine design." The work will point out the limits of fully-parallel mechanisms and justify the use of hybrid solutions. The most important elements of the mechanisms, thereof particularly the spherical and universal joints, will be treated in a detailed manner. Industrialization aspects will be analyzed, the difficulty for their integration will be shown, and solutions provided in order to increase the accessibility of hybrid and parallel mechanisms. A design methodology will be synthesized from all these elements and applied to three case studies. The methodology will point out important and often neglected steps and provide elements and tools to support the designer in the whole process of creation. Furthermore, by providing a broad catalogue of both new and existing hybrid and parallel kinematics, this work is intended to stimulate and inspire the creativity of the designer. The three final cases studies, each differing in their application domain and representing each an unpublished concept, will illustrate and validate the methodology. The work took place around multiple industrial projects and therefore always keeps in mind the practical feasibility, with respect to an industrial environment, and the economic aspects and risks

Robotics-Assisted Needle Steering for Percutaneous Interventions: Modeling and Experiments

Needle insertion and guidance plays an important role in medical procedures such as brachytherapy and biopsy. Flexible needles have the potential to facilitate precise targeting and avoid collisions during medical interventions while reducing trauma to the patient and post-puncture issues. Nevertheless, error introduced during guidance degrades the effectiveness of the planned therapy or diagnosis. Although steering using flexible bevel-tip needles provides great mobility and dexterity, a major barrier is the complexity of needle-tissue interaction that does not lend itself to intuitive control. To overcome this problem, a robotic system can be employed to perform trajectory planning and tracking by manipulation of the needle base. This research project focuses on a control-theoretic approach and draws on the rich literature from control and systems theory to model needle-tissue interaction and needle flexion and then design a robotics-based strategy for needle insertion/steering. The resulting solutions will directly benefit a wide range of needle-based interventions. The outcome of this computer-assisted approach will not only enable us to perform efficient preoperative trajectory planning, but will also provide more insight into needle-tissue interaction that will be helpful in developing advanced intraoperative algorithms for needle steering. Experimental validation of the proposed methodologies was carried out on a state of-the-art 5-DOF robotic system designed and constructed in-house primarily for prostate brachytherapy. The system is equipped with a Nano43 6-DOF force/torque sensor (ATI Industrial Automation) to measure forces and torques acting on the needle shaft. In our setup, an Aurora electromagnetic tracker (Northern Digital Inc.) is the sensing device used for measuring needle deflection. A multi-threaded application for control, sensor readings, data logging and communication over the ethernet was developed using Microsoft Visual C 2005, MATLAB 2007 and the QuaRC Toolbox (Quanser Inc.). Various artificial phantoms were developed so as to create a realistic medium in terms of elasticity and insertion force ranges; however, they simulated a uniform environment without exhibiting complexities of organic tissues. Experiments were also conducted on beef liver and fresh chicken breast, beef, and ham, to investigate the behavior of a variety biological tissues

Efficient modelling of RC walls for accurate simulations under earthquake loading

Remarkable advances have been achieved in earthquake engineering in the past decades, given the growing awareness and concern regarding the global seismic risk. In the wake of the damage wrought by the recent earthquakes, reinforced concrete (RC) walls are commonly employed as effective seismic resisting components in new building structures or retrofitting solutions to enhance the seismic performance of existing sub-standard frame buildings. Current codes of practice recommend using nonlinear dynamic analysis as the most accurate technique for the seismic evaluation of RC buildings under earthquake loading. This necessitates the development of reliable numerical strategies for accurate simulation of RC walls under cyclic loading conditions representing seismic actions. This research starts with a critical appraisal of currently available modelling strategies for RC walls associated with different levels of sophistication. They include: (i) the wide column approach with 1D beam elements, (ii) 2D FE models with nonlinear shell elements and (iii) detailed 3D FE descriptions with solid elements and embedded bar elements. Numerical simulations have been performed considering experimental slender and short wall specimens subjected to cyclic loading. Numerical-experimental comparisons highlight some drawbacks of existing modelling strategies as their inability to represent the actual degradation of strength and stiffness and the pinching characteristics of the cyclic behaviour, especially in the case of wall samples whose response is governed by flexure-shear interaction. In view of these limitations and to achieve more accurate response predictions, an efficient and practical 2D macro-element representation for RC walls is proposed in the second part of the research. It incorporates a biaxial concrete model based on the rotating crack approach to account for the nonlinear response under cyclic loading conditions. Accuracy and efficiency of the macro-element model have been demonstrated by validation studies, focusing on RC walls with different aspect ratios and an RC coupled wall system. The ability of the proposed model to predict the main cyclic response characteristics of RC walls, including stiffness and strength degradation, energy dissipation capacity, and pinched shapes of the hysteresis loops, has been confirmed by a favourable agreement between the numerical predictions and experimental findings. The final part of this research proceeds with an application study on seismic analysis of a realistic four-storey RC frame-wall building. The developed macro-element model accounting for shear deformability and potential shear damage and failure provides a more realistic representation for RC walls than the wide column approach widely used in practice. Moreover, the macro-element modelling strategy requires a comparable computational cost to the wide column approach, which renders it suitable for nonlinear dynamic analysis of large scale structures and realistic seismic assessment of RC buildings with shear walls.Open Acces

Advances in Robot Kinematics : Proceedings of the 15th international conference on Advances in Robot Kinematics

International audienceThe motion of mechanisms, kinematics, is one of the most fundamental aspect of robot design, analysis and control but is also relevant to other scientific domains such as biome- chanics, molecular biology, . . . . The series of books on Advances in Robot Kinematics (ARK) report the latest achievement in this field. ARK has a long history as the first book was published in 1991 and since then new issues have been published every 2 years. Each book is the follow-up of a single-track symposium in which the participants exchange their results and opinions in a meeting that bring together the best of world’s researchers and scientists together with young students. Since 1992 the ARK symposia have come under the patronage of the International Federation for the Promotion of Machine Science-IFToMM.This book is the 13th in the series and is the result of peer-review process intended to select the newest and most original achievements in this field. For the first time the articles of this symposium will be published in a green open-access archive to favor free dissemination of the results. However the book will also be o↵ered as a on-demand printed book.The papers proposed in this book show that robot kinematics is an exciting domain with an immense number of research challenges that go well beyond the field of robotics.The last symposium related with this book was organized by the French National Re- search Institute in Computer Science and Control Theory (INRIA) in Grasse, France

Of Priors and Particles: Structured and Distributed Approaches to Robot Perception and Control

Applications of robotic systems have expanded significantly in their scope, moving beyond the caged predictability of industrial automation and towards more open, unstructured environments. These agents must learn to reliably perceive their surroundings, efficiently integrate new information and quickly adapt to dynamic perturbations. To accomplish this, we require solutions which can effectively incorporate prior knowledge while maintaining the generality of learned representations. These systems must also contend with uncertainty in both their perception of the world and in predicting possible future outcomes. Efficient methods for probabilistic inference are then key to realizing robust, adaptive behavior. This thesis will first examine data-driven approaches for learning and combining perceptual models for both visual and tactile sensor modalities, common in robotics. Modern variational inference methods will then be examined in the context of online optimization and stochastic optimal control. Specifically, this thesis will contribute (1) data-driven visual and tactile perceptual models leveraging kinematic and dynamic priors, (2) a framework for joint inference with visuo-tactile sensing, (3) a family of particle-based, variational model predictive control and planning algorithms, and (4) a distributed inference scheme for online model adaptation.Ph.D