Fuzzy optimisation based symbolic grounding for service robots

A thesis submitted to the University of Bedfordshire in partial fulfilment of the requirements for the degree of Doctor of PhilosophySymbolic grounding is a bridge between task level planning and actual robot sensing and actuation. Uncertainties raised by unstructured environments make a bottleneck for integrating traditional artificial intelligence with service robotics. In this research, a fuzzy optimisation based symbolic grounding approach is presented. This approach can handle uncertainties and helps service robots to determine the most comfortable base region for grasping objects in a fetch and carry task. Novel techniques are applied to establish fuzzy objective function, to model fuzzy constraints and to perform fuzzy optimisation. The approach does not have the short comings of others’ work and the computation time is dramatically reduced in compare with other methods. The advantages of the proposed fuzzy optimisation based approach are evidenced by experiments that were undertaken in Care-O-bot 3 (COB 3) and Robot Operating System (ROS) platforms

Multi-robot cooperative platform : a task-oriented teleoperation paradigm

This thesis proposes the study and development of a teleoperation system based on multi-robot cooperation under the task oriented teleoperation paradigm: Multi-Robot Cooperative Paradigm, MRCP. In standard teleoperation, the operator uses the master devices to control the remote slave robot arms. These arms reproduce the desired movements and perform the task. With the developed work, the operator can virtually manipulate an object. MRCP automatically generates the arms orders to perform the task. The operator does not have to solve situations arising from possible restrictions that the slave arms may have. The research carried out is therefore aimed at improving the accuracy teleoperation tasks in complex environments, particularly in the field of robot assisted minimally invasive surgery. This field requires patient safety and the workspace entails many restrictions to teleoperation. MRCP can be defined as a platform composed of several robots that cooperate automatically to perform a teleoperated task, creating a robotic system with increased capacity (workspace volume, accessibility, dexterity ...). The cooperation is based on transferring the task between robots when necessary to enable a smooth task execution. The MRCP control evaluates the suitability of each robot to continue with the ongoing task and the optimal time to execute a task transfer between the current selected robot and the best candidate to continue with the task. From the operator¿s point of view, MRCP provides an interface that enables the teleoperation though the task-oriented paradigm: operator orders are translated into task actions instead of robot orders. This thesis is structured as follows: The first part is dedicated to review the current solutions in the teleoperation of complex tasks and compare them with those proposed in this research. The second part of the thesis presents and reviews in depth the different evaluation criteria to determine the suitability of each robot to continue with the execution of a task, considering the configuration of the robots and emphasizing the criterion of dexterity and manipulability. The study reviews the different required control algorithms to enable the task oriented telemanipulation. This proposed teleoperation paradigm is transparent to the operator. Then, the Thesis presents and analyses several experimental results using MRCP in the field of minimally invasive surgery. These experiments study the effectiveness of MRCP in various tasks requiring the cooperation of two hands. A type task is used: a suture using minimally invasive surgery technique. The analysis is done in terms of execution time, economy of movement, quality and patient safety (potential damage produced by undesired interaction between the tools and the vital tissues of the patient). The final part of the thesis proposes the implementation of different virtual aids and restrictions (guided teleoperation based on haptic visual and audio feedback, protection of restricted workspace regions, etc.) using the task oriented teleoperation paradigm. A framework is defined for implementing and applying a basic set of virtual aids and constraints within the framework of a virtual simulator for laparoscopic abdominal surgery. The set of experiments have allowed to validate the developed work. The study revealed the influence of virtual aids in the learning process of laparoscopic techniques. It has also demonstrated the improvement of learning curves, which paves the way for its implementation as a methodology for training new surgeons.Aquesta tesi doctoral proposa l'estudi i desenvolupament d'un sistema de teleoperació basat en la cooperació multi-robot sota el paradigma de la teleoperació orientada a tasca: Multi-Robot Cooperative Paradigm, MRCP. En la teleoperació clàssica, l'operador utilitza els telecomandaments perquè els braços robots reprodueixin els seus moviments i es realitzi la tasca desitjada. Amb el treball realitzat, l'operador pot manipular virtualment un objecte i és mitjançant el MRCP que s'adjudica a cada braç les ordres necessàries per realitzar la tasca, sense que l'operador hagi de resoldre les situacions derivades de possibles restriccions que puguin tenir els braços executors. La recerca desenvolupada està doncs orientada a millorar la teleoperació en tasques de precisió en entorns complexos i, en particular, en el camp de la cirurgia mínimament invasiva assistida per robots. Aquest camp imposa condicions de seguretat del pacient i l'espai de treball comporta moltes restriccions a la teleoperació. MRCP es pot definir com a una plataforma formada per diversos robots que cooperen de forma automàtica per dur a terme una tasca teleoperada, generant un sistema robòtic amb capacitats augmentades (volums de treball, accessibilitat, destresa,...). La cooperació es basa en transferir la tasca entre robots a partir de determinar quin és aquell que és més adequat per continuar amb la seva execució i el moment òptim per realitzar la transferència de la tasca entre el robot actiu i el millor candidat a continuar-la. Des del punt de vista de l'operari, MRCP ofereix una interfície de teleoperació que permet la realització de la teleoperació mitjançant el paradigma d'ordres orientades a la tasca: les ordres es tradueixen en accions sobre la tasca en comptes d'estar dirigides als robots. Aquesta tesi està estructurada de la següent manera: Primerament es fa una revisió de l'estat actual de les diverses solucions desenvolupades actualment en el camp de la teleoperació de tasques complexes, comparant-les amb les proposades en aquest treball de recerca. En el segon bloc de la tesi es presenten i s'analitzen a fons els diversos criteris per determinar la capacitat de cada robot per continuar l'execució d'una tasca, segons la configuració del conjunt de robots i fent especial èmfasi en el criteri de destresa i manipulabilitat. Seguint aquest estudi, es presenten els diferents processos de control emprats per tal d'assolir la telemanipulació orientada a tasca de forma transparent a l'operari. Seguidament es presenten diversos resultats experimentals aplicant MRCP al camp de la cirurgia mínimament invasiva. En aquests experiments s'estudia l'eficàcia de MRCP en diverses tasques que requereixen de la cooperació de dues mans. S'ha escollit una tasca tipus: sutura amb tècnica de cirurgia mínimament invasiva. L'anàlisi es fa en termes de temps d'execució, economia de moviment, qualitat i seguretat del pacient (potencials danys causats per la interacció no desitjada entre les eines i els teixits vitals del pacient). Finalment s'ha estudiat l'ús de diferents ajudes i restriccions virtuals (guiat de la teleoperació via retorn hàptic, visual o auditiu, protecció de regions de l'espai de treball, etc) dins el paradigma de teleoperació orientada a tasca. S'ha definint un marc d'aplicació base i implementant un conjunt de restriccions virtuals dins el marc d'un simulador de cirurgia laparoscòpia abdominal. El conjunt d'experiments realitzats han permès validar el treball realitzat. Aquest estudi ha permès determinar la influencia de les ajudes virtuals en el procés d'aprenentatge de les tècniques laparoscòpiques. S'ha evidenciat una millora en les corbes d'aprenentatge i obre el camí a la seva implantació com a metodologia d'entrenament de nous cirurgians.Postprint (published version

Kontextsensitive Körperregulierung für redundante Roboter

In the past few decades the classical 6 degrees of freedom manipulators' dominance has been challenged by the rise of 7 degrees of freedom redundant robots. Similarly, with increased availability of humanoid robots in academic research, roboticists suddenly have access to highly dexterous platforms with multiple kinematic chains capable of undertaking multiple tasks simultaneously. The execution of lower-priority tasks, however, are often done in task/scenario specific fashion. Consequently, these systems are not scalable and slight changes in the application often implies re-engineering the entire control system and deployment which impedes the development process over time. This thesis introduces an alternative systematic method of addressing the secondary tasks and redundancy resolution called, context aware body regulation. Contexts consist of one or multiple tasks, however, unlike the conventional definitions, the tasks within a context are not rigidly defined and maintain some level of abstraction. For instance, following a particular trajectory constitutes a concrete task while performing a Cartesian motion with the end-effector represents an abstraction of the same task and is more appropriate for context formulation. Furthermore, contexts are often made up of multiple abstract tasks that collectively describe a reoccurring situation. Body regulation is an umbrella term for a collection of schemes for addressing the robots' redundancy when a particular context occurs. Context aware body regulation offers several advantages over traditional methods. Most notably among them are reusability, scalability and composability of contexts and body regulation schemes. These three fundamental concerns are realized theoretically by in-depth study and through mathematical analysis of contexts and regulation strategies; and are practically implemented by a component based software architecture that complements the theoretical aspects. The findings of the thesis are applicable to any redundant manipulator and humanoids, and allow them to be used in real world applications. Proposed methodology presents an alternative approach for the control of robots and offers a new perspective for future deployment of robotic solutions.Im Verlauf der letzten Jahrzehnte wich der Einfluss klassischer Roboterarme mit 6 Freiheitsgraden zunehmend denen neuer und vielfältigerer Manipulatoren mit 7 Gelenken. Ebenso stehen der Forschung mit den neuartigen Humanoiden inzwischen auch hoch-redundante Roboterplattformen mit mehreren kinematischen Ketten zur Verfügung. Diese überaus flexiblen und komplexen Roboter-Kinematiken ermöglichen generell das gleichzeitige Verfolgen mehrerer priorisierter Bewegungsaufgaben. Die Steuerung der weniger wichtigen Aufgaben erfolgt jedoch oft in anwendungsspezifischer Art und Weise, welche die Skalierung der Regelung zu generellen Kontexten verhindert. Selbst kleine Änderungen in der Anwendung bewirken oft schon, dass große Teile der Robotersteuerung überarbeitet werden müssen, was wiederum den gesamten Entwicklungsprozess behindert. Diese Dissertation stellt eine alternative, systematische Methode vor um die Redundanz neuer komplexer Robotersysteme zu bewältigen und vielfältige, priorisierte Bewegungsaufgaben parallel zu steuern: Die so genannte kontextsensitive Körperregulierung. Darin bestehen Kontexte aus einer oder mehreren Bewegungsaufgaben. Anders als in konventionellen Anwendungen sind die Aufgaben nicht fest definiert und beinhalten eine gewisse Abstraktion. Beispielsweise stellt das Folgen einer bestimmten Trajektorie eine sehr konkrete Bewegungsaufgabe dar, während die Ausführung einer Kartesischen Bewegung mit dem Endeffektor eine Abstraktion darstellt, die für die Kontextformulierung besser geeignet ist. Kontexte setzen sich oft aus mehreren solcher abstrakten Aufgaben zusammen und beschreiben kollektiv eine sich wiederholende Situation. Durch die Verwendung der kontextsensitiven Körperregulierung ergeben sich vielfältige Vorteile gegenüber traditionellen Methoden: Wiederverwendbarkeit, Skalierbarkeit, sowie Komponierbarkeit von Konzepten. Diese drei fundamentalen Eigenschaften werden in der vorliegenden Arbeit theoretisch mittels gründlicher mathematischer Analyse aufgezeigt und praktisch mittels einer auf Komponenten basierenden Softwarearchitektur realisiert. Die Ergebnisse dieser Dissertation lassen sich auf beliebige redundante Manipulatoren oder humanoide Roboter anwenden und befähigen diese damit zur realen Anwendung außerhalb des Labors. Die hier vorgestellte Methode zur Regelung von Robotern stellt damit eine neue Perspektive für die zukünftige Entwicklung von robotischen Lösungen dar

Predictive Coordination in Goal-Oriented Shared Autonomy for Robotic Manipulators

Through shared autonomy, human intelligence can be combined with robot autonomy in order to enhance direct human teleoperation. This control paradigm involves two separate controllers -- a human operator and an assistive controller -- working in concert to operate the same physical robot. Generally, the advantage of shared autonomy systems over fully autonomous systems is that they retain a human operator's insight into the task to be completed, informed by the human's understanding of the broader context of the task and environment. However, introducing assistance into these robot control systems can make the system's performance complex due to the required coordination of these two control agents. Research into goal-oriented shared autonomy systems addresses this coordination through creating assistive controllers that model an operator's intended goal in order to provide assistance toward this goal. In this dissertation, I argue that these goal-oriented control systems can be extended through considering prediction of future actions in addition to the goal the control agents are working toward. Through modeling future expected operator control commands, assistive controllers can utilize novel methods for motion optimization that incorporate this expected future trajectory into motion planning. Analogously, an assistive controller can attempt to improve coordination with a human operator through communicating its own intended future commands. This dissertation discusses the use of shared autonomy systems for robotic manipulation, dividing these systems into classes according to how assistance is combined with operator commands. Then, novel systems are presented for each of these classes that apply modeling of future command actions to address problems critical to that respective class of control. Finally, the concluding chapter synthesizes results from these individual systems into general principles about the design and evaluation of shared autonomy systems. Designing shared autonomy systems that incorporate modeling of the two control agents' future commands allows the control agents to coordinate actions over predicted future trajectories. This dissertation develops and evaluates this predictive coordination approach to shared autonomy through designing novel systems that assist human operators in controlling robotic manipulators while utilizing models of future command actions to improve the coordination between assistive controllers and operators.</p

Collaborative mobile industrial manipulator : a review of system architecture and applications

This paper provides a comprehensive review of the development of Collaborative Mobile Industrial Manipulator (CMIM), which is currently in high demand. Such a review is necessary to have an overall understanding about CMIM advanced technology. This is the first review to combine the system architecture and application which is necessary in order to gain a full understanding of the system. The classical framework of CMIM is firstly discussed, including hardware and software. Subsystems that are typically involved in hardware such as mobile platform, manipulator, end-effector and sensors are presented. With regards to software, planner, controller, perception, interaction and so on are also described. Following this, the common applications (logistics, manufacturing and assembly) in industry are surveyed. Finally, the trends are predicted and issues are indicated as references for CMIM researchers. Specifically, more research is needed in the areas of interaction, fully autonomous control, coordination and standards. Besides, experiments in real environment would be performed more and novel collaborative robotic systems would be proposed in future. Additionally, some advanced technology in other areas would also be applied into the system. In all, the system would become more intelligent, collaborative and autonomous

Design and Control of Robotic Systems for Lower Limb Stroke Rehabilitation

Lower extremity stroke rehabilitation exhausts considerable health care resources, is labor intensive, and provides mostly qualitative metrics of patient recovery. To overcome these issues, robots can assist patients in physically manipulating their affected limb and measure the output motion. The robots that have been currently designed, however, provide assistance over a limited set of training motions, are not portable for in-home and in-clinic use, have high cost and may not provide sufficient safety or performance. This thesis proposes the idea of incorporating a mobile drive base into lower extremity rehabilitation robots to create a portable, inherently safe system that provides assistance over a wide range of training motions. A set of rehabilitative motion tasks were established and a six-degree-of-freedom (DOF) motion and force-sensing system was designed to meet high-power, large workspace, and affordability requirements. An admittance controller was implemented, and the feasibility of using this portable, low-cost system for movement assistance was shown through tests on a healthy individual. An improved version of the robot was then developed that added torque sensing and known joint elasticity for use in future clinical testing with a flexible-joint impedance controller

Kinematics and Robot Design IV, KaRD2021

This volume collects the papers published on the special issue “Kinematics and Robot Design IV, KaRD2021” (https://www.mdpi.com/journal/robotics/special_issues/KaRD2021), which is the forth edition of the KaRD special-issue series, hosted by the open-access journal “MDPI Robotics”. KaRD series is an open environment where researchers can present their works and discuss all the topics focused on the many aspects that involve kinematics in the design of robotic/automatic systems. Kinematics is so intimately related to the design of robotic/automatic systems that the admitted topics of the KaRD series practically cover all the subjects normally present in well-established international conferences on “mechanisms and robotics”. KaRD2021, after the peer-review process, accepted 12 papers. The accepted papers cover some theoretical and many design/applicative aspects

Robot Manipulators

Robot manipulators are developing more in the direction of industrial robots than of human workers. Recently, the applications of robot manipulators are spreading their focus, for example Da Vinci as a medical robot, ASIMO as a humanoid robot and so on. There are many research topics within the field of robot manipulators, e.g. motion planning, cooperation with a human, and fusion with external sensors like vision, haptic and force, etc. Moreover, these include both technical problems in the industry and theoretical problems in the academic fields. This book is a collection of papers presenting the latest research issues from around the world

Nonlinear Modeling and Control of Driving Interfaces and Continuum Robots for System Performance Gains

With the rise of (semi)autonomous vehicles and continuum robotics technology and applications, there has been an increasing interest in controller and haptic interface designs. The presence of nonlinearities in the vehicle dynamics is the main challenge in the selection of control algorithms for real-time regulation and tracking of (semi)autonomous vehicles. Moreover, control of continuum structures with infinite dimensions proves to be difficult due to their complex dynamics plus the soft and flexible nature of the manipulator body. The trajectory tracking and control of automobile and robotic systems requires control algorithms that can effectively deal with the nonlinearities of the system without the need for approximation, modeling uncertainties, and input disturbances. Control strategies based on a linearized model are often inadequate in meeting precise performance requirements. To cope with these challenges, one must consider nonlinear techniques. Nonlinear control systems provide tools and methodologies for enabling the design and realization of (semi)autonomous vehicle and continuum robots with extended specifications based on the operational mission profiles. This dissertation provides an insight into various nonlinear controllers developed for (semi)autonomous vehicles and continuum robots as a guideline for future applications in the automobile and soft robotics field. A comprehensive assessment of the approaches and control strategies, as well as insight into the future areas of research in this field, are presented.First, two vehicle haptic interfaces, including a robotic grip and a joystick, both of which are accompanied by nonlinear sliding mode control, have been developed and studied on a steer-by-wire platform integrated with a virtual reality driving environment. An operator-in-the-loop evaluation that included 30 human test subjects was used to investigate these haptic steering interfaces over a prescribed series of driving maneuvers through real time data logging and post-test questionnaires. A conventional steering wheel with a robust sliding mode controller was used for all the driving events for comparison. Test subjects operated these interfaces for a given track comprised of a double lane-change maneuver and a country road driving event. Subjective and objective results demonstrate that the driver’s experience can be enhanced up to 75.3% with a robotic steering input when compared to the traditional steering wheel during extreme maneuvers such as high-speed driving and sharp turn (e.g., hairpin turn) passing. Second, a cellphone-inspired portable human-machine-interface (HMI) that incorporated the directional control of the vehicle as well as the brake and throttle functionality into a single holistic device will be presented. A nonlinear adaptive control technique and an optimal control approach based on driver intent were also proposed to accompany the mechatronic system for combined longitudinal and lateral vehicle guidance. Assisting the disabled drivers by excluding extensive arm and leg movements ergonomically, the device has been tested in a driving simulator platform. Human test subjects evaluated the mechatronic system with various control configurations through obstacle avoidance and city road driving test, and a conventional set of steering wheel and pedals were also utilized for comparison. Subjective and objective results from the tests demonstrate that the mobile driving interface with the proposed control scheme can enhance the driver’s performance by up to 55.8% when compared to the traditional driving system during aggressive maneuvers. The system’s superior performance during certain vehicle maneuvers and approval received from the participants demonstrated its potential as an alternative driving adaptation for disabled drivers. Third, a novel strategy is designed for trajectory control of a multi-section continuum robot in three-dimensional space to achieve accurate orientation, curvature, and section length tracking. The formulation connects the continuum manipulator dynamic behavior to a virtual discrete-jointed robot whose degrees of freedom are directly mapped to those of a continuum robot section under the hypothesis of constant curvature. Based on this connection, a computed torque control architecture is developed for the virtual robot, for which inverse kinematics and dynamic equations are constructed and exploited, with appropriate transformations developed for implementation on the continuum robot. The control algorithm is validated in a realistic simulation and implemented on a six degree-of-freedom two-section OctArm continuum manipulator. Both simulation and experimental results show that the proposed method could manage simultaneous extension/contraction, bending, and torsion actions on multi-section continuum robots with decent tracking performance (e.g. steady state arc length and curvature tracking error of 3.3mm and 130mm-1, respectively). Last, semi-autonomous vehicles equipped with assistive control systems may experience degraded lateral behaviors when aggressive driver steering commands compete with high levels of autonomy. This challenge can be mitigated with effective operator intent recognition, which can configure automated systems in context-specific situations where the driver intends to perform a steering maneuver. In this article, an ensemble learning-based driver intent recognition strategy has been developed. A nonlinear model predictive control algorithm has been designed and implemented to generate haptic feedback for lateral vehicle guidance, assisting the drivers in accomplishing their intended action. To validate the framework, operator-in-the-loop testing with 30 human subjects was conducted on a steer-by-wire platform with a virtual reality driving environment. The roadway scenarios included lane change, obstacle avoidance, intersection turns, and highway exit. The automated system with learning-based driver intent recognition was compared to both the automated system with a finite state machine-based driver intent estimator and the automated system without any driver intent prediction for all driving events. Test results demonstrate that semi-autonomous vehicle performance can be enhanced by up to 74.1% with a learning-based intent predictor. The proposed holistic framework that integrates human intelligence, machine learning algorithms, and vehicle control can help solve the driver-system conflict problem leading to safer vehicle operations