Development of new intelligent autonomous robotic assistant for hospitals

Continuous technological development in modern societies has increased the quality of life and average life-span of people. This imposes an extra burden on the current healthcare infrastructure, which also creates the opportunity for developing new, autonomous, assistive robots to help alleviate this extra workload. The research question explored the extent to which a prototypical robotic platform can be created and how it may be implemented in a hospital environment with the aim to assist the hospital staff with daily tasks, such as guiding patients and visitors, following patients to ensure safety, and making deliveries to and from rooms and workstations. In terms of major contributions, this thesis outlines five domains of the development of an actual robotic assistant prototype. Firstly, a comprehensive schematic design is presented in which mechanical, electrical, motor control and kinematics solutions have been examined in detail. Next, a new method has been proposed for assessing the intrinsic properties of different flooring-types using machine learning to classify mechanical vibrations. Thirdly, the technical challenge of enabling the robot to simultaneously map and localise itself in a dynamic environment has been addressed, whereby leg detection is introduced to ensure that, whilst mapping, the robot is able to distinguish between people and the background. The fourth contribution is geometric collision prediction into stabilised dynamic navigation methods, thus optimising the navigation ability to update real-time path planning in a dynamic environment. Lastly, the problem of detecting gaze at long distances has been addressed by means of a new eye-tracking hardware solution which combines infra-red eye tracking and depth sensing. The research serves both to provide a template for the development of comprehensive mobile assistive-robot solutions, and to address some of the inherent challenges currently present in introducing autonomous assistive robots in hospital environments.Open Acces

Advances in Robot Navigation

Robot navigation includes different interrelated activities such as perception - obtaining and interpreting sensory information; exploration - the strategy that guides the robot to select the next direction to go; mapping - the construction of a spatial representation by using the sensory information perceived; localization - the strategy to estimate the robot position within the spatial map; path planning - the strategy to find a path towards a goal location being optimal or not; and path execution, where motor actions are determined and adapted to environmental changes. This book integrates results from the research work of authors all over the world, addressing the abovementioned activities and analyzing the critical implications of dealing with dynamic environments. Different solutions providing adaptive navigation are taken from nature inspiration, and diverse applications are described in the context of an important field of study: social robotics

Multi-robot cooperative platform : a task-oriented teleoperation paradigm

This thesis proposes the study and development of a teleoperation system based on multi-robot cooperation under the task oriented teleoperation paradigm: Multi-Robot Cooperative Paradigm, MRCP. In standard teleoperation, the operator uses the master devices to control the remote slave robot arms. These arms reproduce the desired movements and perform the task. With the developed work, the operator can virtually manipulate an object. MRCP automatically generates the arms orders to perform the task. The operator does not have to solve situations arising from possible restrictions that the slave arms may have. The research carried out is therefore aimed at improving the accuracy teleoperation tasks in complex environments, particularly in the field of robot assisted minimally invasive surgery. This field requires patient safety and the workspace entails many restrictions to teleoperation. MRCP can be defined as a platform composed of several robots that cooperate automatically to perform a teleoperated task, creating a robotic system with increased capacity (workspace volume, accessibility, dexterity ...). The cooperation is based on transferring the task between robots when necessary to enable a smooth task execution. The MRCP control evaluates the suitability of each robot to continue with the ongoing task and the optimal time to execute a task transfer between the current selected robot and the best candidate to continue with the task. From the operator¿s point of view, MRCP provides an interface that enables the teleoperation though the task-oriented paradigm: operator orders are translated into task actions instead of robot orders. This thesis is structured as follows: The first part is dedicated to review the current solutions in the teleoperation of complex tasks and compare them with those proposed in this research. The second part of the thesis presents and reviews in depth the different evaluation criteria to determine the suitability of each robot to continue with the execution of a task, considering the configuration of the robots and emphasizing the criterion of dexterity and manipulability. The study reviews the different required control algorithms to enable the task oriented telemanipulation. This proposed teleoperation paradigm is transparent to the operator. Then, the Thesis presents and analyses several experimental results using MRCP in the field of minimally invasive surgery. These experiments study the effectiveness of MRCP in various tasks requiring the cooperation of two hands. A type task is used: a suture using minimally invasive surgery technique. The analysis is done in terms of execution time, economy of movement, quality and patient safety (potential damage produced by undesired interaction between the tools and the vital tissues of the patient). The final part of the thesis proposes the implementation of different virtual aids and restrictions (guided teleoperation based on haptic visual and audio feedback, protection of restricted workspace regions, etc.) using the task oriented teleoperation paradigm. A framework is defined for implementing and applying a basic set of virtual aids and constraints within the framework of a virtual simulator for laparoscopic abdominal surgery. The set of experiments have allowed to validate the developed work. The study revealed the influence of virtual aids in the learning process of laparoscopic techniques. It has also demonstrated the improvement of learning curves, which paves the way for its implementation as a methodology for training new surgeons.Aquesta tesi doctoral proposa l'estudi i desenvolupament d'un sistema de teleoperació basat en la cooperació multi-robot sota el paradigma de la teleoperació orientada a tasca: Multi-Robot Cooperative Paradigm, MRCP. En la teleoperació clàssica, l'operador utilitza els telecomandaments perquè els braços robots reprodueixin els seus moviments i es realitzi la tasca desitjada. Amb el treball realitzat, l'operador pot manipular virtualment un objecte i és mitjançant el MRCP que s'adjudica a cada braç les ordres necessàries per realitzar la tasca, sense que l'operador hagi de resoldre les situacions derivades de possibles restriccions que puguin tenir els braços executors. La recerca desenvolupada està doncs orientada a millorar la teleoperació en tasques de precisió en entorns complexos i, en particular, en el camp de la cirurgia mínimament invasiva assistida per robots. Aquest camp imposa condicions de seguretat del pacient i l'espai de treball comporta moltes restriccions a la teleoperació. MRCP es pot definir com a una plataforma formada per diversos robots que cooperen de forma automàtica per dur a terme una tasca teleoperada, generant un sistema robòtic amb capacitats augmentades (volums de treball, accessibilitat, destresa,...). La cooperació es basa en transferir la tasca entre robots a partir de determinar quin és aquell que és més adequat per continuar amb la seva execució i el moment òptim per realitzar la transferència de la tasca entre el robot actiu i el millor candidat a continuar-la. Des del punt de vista de l'operari, MRCP ofereix una interfície de teleoperació que permet la realització de la teleoperació mitjançant el paradigma d'ordres orientades a la tasca: les ordres es tradueixen en accions sobre la tasca en comptes d'estar dirigides als robots. Aquesta tesi està estructurada de la següent manera: Primerament es fa una revisió de l'estat actual de les diverses solucions desenvolupades actualment en el camp de la teleoperació de tasques complexes, comparant-les amb les proposades en aquest treball de recerca. En el segon bloc de la tesi es presenten i s'analitzen a fons els diversos criteris per determinar la capacitat de cada robot per continuar l'execució d'una tasca, segons la configuració del conjunt de robots i fent especial èmfasi en el criteri de destresa i manipulabilitat. Seguint aquest estudi, es presenten els diferents processos de control emprats per tal d'assolir la telemanipulació orientada a tasca de forma transparent a l'operari. Seguidament es presenten diversos resultats experimentals aplicant MRCP al camp de la cirurgia mínimament invasiva. En aquests experiments s'estudia l'eficàcia de MRCP en diverses tasques que requereixen de la cooperació de dues mans. S'ha escollit una tasca tipus: sutura amb tècnica de cirurgia mínimament invasiva. L'anàlisi es fa en termes de temps d'execució, economia de moviment, qualitat i seguretat del pacient (potencials danys causats per la interacció no desitjada entre les eines i els teixits vitals del pacient). Finalment s'ha estudiat l'ús de diferents ajudes i restriccions virtuals (guiat de la teleoperació via retorn hàptic, visual o auditiu, protecció de regions de l'espai de treball, etc) dins el paradigma de teleoperació orientada a tasca. S'ha definint un marc d'aplicació base i implementant un conjunt de restriccions virtuals dins el marc d'un simulador de cirurgia laparoscòpia abdominal. El conjunt d'experiments realitzats han permès validar el treball realitzat. Aquest estudi ha permès determinar la influencia de les ajudes virtuals en el procés d'aprenentatge de les tècniques laparoscòpiques. S'ha evidenciat una millora en les corbes d'aprenentatge i obre el camí a la seva implantació com a metodologia d'entrenament de nous cirurgians.Postprint (published version

Advanced Robot Path Planning (RRT)

Tato diplomová práce práce se zabývá plánováním cesty všesměrového mobilního robotu pomocí algoritmu RRT (Rapidly-exploring Random Tree – Rychle rostoucí náhodný strom). Teoretická část popisuje základní algoritmy plánování cesty a prezentuje bližší pohled na RRT a jeho potenciál. Praktická část práce řeší návrh a tvorbu v zásadě multiplatformní C++ aplikace v prostředí Windows 7 za použití aplikačního frameworku Qt 4.8.0, která implementuje pokročilé RRT algoritmy s parametrizovatelným řešičem a speciálním dávkovým režimem. Tento mód slouží k testování efektivnosti nastavení řešiče pro dané úlohy a je založen na post-processingu a vizualizaci výstupu měřených úloh pomocí jazyka Python. Vypočtené cesty mohou být vylepšeny pomocí zkracovacích algoritmů a výsledná trajektorie odeslána do pohonů Maxon Compact Drive všesměrové mobilní platformy pomocí CANopen. Aplikace klade důraz na moderní grafické uživatelské rozhraní se spolehlivým a výkonným 2D grafickým engine.This master's thesis deals with path planning of omnidirectional mobile robot using the RRT algorithm (Rapidly-exploring Random Tree). Theoretical part describes basic algorithms of path planning and presents closer view on RRT and its potential. Practical part deals with designing and creation of essentially multiplatform C++ application in Windows 7 environment with Qt 4.8.0 application framework, which implements advanced RRT algorithms with user-programmable solver and special batch mode. This mode is used for testing the effectiveness of solver on given tasks and it is based on postprocessing and visualization of measurement tasks output by Python language. Computed paths can be enhanced by shortening algorithms and result trajectory sent to Maxon Compact Drives of omnidirectional platform via the CANopen. Application puts emphasis on modern GUI with reliable and powerful 2D graphics engine.

Mobile Robots Navigation

Mobile robots navigation includes different interrelated activities: (i) perception, as obtaining and interpreting sensory information; (ii) exploration, as the strategy that guides the robot to select the next direction to go; (iii) mapping, involving the construction of a spatial representation by using the sensory information perceived; (iv) localization, as the strategy to estimate the robot position within the spatial map; (v) path planning, as the strategy to find a path towards a goal location being optimal or not; and (vi) path execution, where motor actions are determined and adapted to environmental changes. The book addresses those activities by integrating results from the research work of several authors all over the world. Research cases are documented in 32 chapters organized within 7 categories next described

Coordinated multi-robot formation control

Tese de doutoramento. Engenharia Electrotécnica e de Computadores. Faculdade de Engenharia. Universidade do Porto. 201

Reactive Control Of Autonomous Dynamical Systems

This thesis mainly consists of five independent papers concerning the reactive control design of autonomous mobile robots in the context of target tracking and cooperative formation keeping with obstacle avoidance in the static/dynamic environment. Technical contents of this thesis are divided into three parts. The first part consists of the first two papers, which consider the target-tracking and obstacle avoidance in the static environment. Especially, in the static environment, a fundamental issue of reactive control design is the local minima problem(LMP) inherent in the potential field methods(PFMs). Through introducing a state-dependent planned goal, the first paper proposes a switching control strategy to tackle this problem. The control law for the planned goal is presented. When trapped into local minima, the robot can escape from local minima by following the planned goal. The proposed control law also takes into account the presence of possible saturation constraints. In addition, a time-varying continuous control law is proposed in the second paper to tackle this problem. Challenges of finding continuous control solutions of LMP are discussed and explicit design strategies are then proposed. The second part of this thesis deals with target-tracking and obstacle avoidance in the dynamic environment. In the third paper, a reactive control design is presented for omnidirectional mobile robots with limited sensor range to track targets while avoiding static and moving obstacles in a dynamically evolving environment. Towards this end, a multiiii objective control problem is formulated and control is synthesized by generating a potential field force for each objective and combining them through analysis and design. Different from standard potential field methods, the composite potential field described in this paper is time-varying and planned to account for moving obstacles and vehicle motion. In order to accommodate a larger class of mobile robots, the fourth paper proposes a reactive control design for unicycle-type mobile robots. With the relative motion among the mobile robot, targets, and obstacles being formulated in polar coordinates, kinematic control laws achieving target-tracking and obstacle avoidance are synthesized using Lyapunov based technique, and more importantly, the proposed control laws also take into account possible kinematic control saturation constraints. The third part of this thesis investigates the cooperative formation control with collision avoidance. In the fifth paper, firstly, the target tracking and collision avoidance problem for a single agent is studied. Instead of directly extending the single agent controls to the multiagents case, the single agent controls are incorporated with the cooperative control design presented in [1]. The proposed decentralized control is reactive, considers the formation feedback and changes in the communication networks. The proposed control is based on a potential field method, its inherent oscillation problem is also studied to improve group transient performance

Motion Planning For Micro Aerial Vehicles

A Micro Aerial Vehicle (MAV) is capable of agile motion in 3D making it an ideal platform for developments of planning and control algorithms. For fully autonomous MAV systems, it is essential to plan motions that are both dynamically feasible and collision-free in cluttered environments. Recent work demonstrates precise control of MAVs using time-parameterized trajectories that satisfy feasibility and safety requirements. However, planning such trajectories is non-trivial, especially when considering constraints, such as optimality and completeness. For navigating in unknown environments, the capability for fast re-planning is also critical. Considering all of these requirements, motion planning for MAVs is a challenging problem. In this thesis, we examine trajectory planning algorithms for MAVs and present methodologies that solve a wide range of planning problems. We first introduce path planning and geometric control methods, which produce spatial paths that are inadequate for high speed flight, but can be used to guide trajectory optimization. We then describe optimization-based trajectory planning and demonstrate this method for solving navigation problems in complex 3D environments. When the initial state is not fixed, an optimization-based method is prone to generate sub-optimal trajectories. To address this challenge, we propose a search-based approach using motion primitives to plan resolution complete and sub-optimal trajectories. This algorithm can also be used to solve planning problems with constraints such as motion uncertainty, limited field-of-view and moving obstacles. The proposed methods can run in real time and are applicable for real-world autonomous navigation, even with limited on-board computational resources. This thesis includes a carefully analysis of the strengths and weaknesses of our planning paradigm and algorithms, and demonstration of their performance through simulation and experiments

Visuohaptic Simulation of a Borescope for Aircraft Engine Inspection

Consisting of a long, fiber optic probe containing a small CCD camera controlled by hand-held articulation interface, a video borescope is used for remote visual inspection of hard to reach components in an aircraft. The knowledge and psychomotor skills, specifically the hand-eye coordination, required for effective inspection are hard to acquire through limited exposure to the borescope in aviation maintenance schools. Inexperienced aircraft maintenance technicians gain proficiency through repeated hands-on learning in the workplace along a steep learning curve while transitioning from the classroom to the workforce. Using an iterative process combined with focused user evaluations, this dissertation details the design, implementation and evaluation of a novel visuohaptic simulator for training novice aircraft maintenance technicians in the task of engine inspection using a borescope. First, we describe the development of the visual components of the simulator, along with the acquisition and modeling of a representative model of a {PT-6} aircraft engine. Subjective assessments with both expert and novice aircraft maintenance engineers evaluated the visual realism and the control interfaces of the simulator. In addition to visual feedback, probe contact feedback is provided through a specially designed custom haptic interface that simulates tip contact forces as the virtual probe intersects with the {3D} model surfaces of the engine. Compared to other haptic interfaces, the custom design is unique in that it is inexpensive and uses a real borescope probe to simulate camera insertion and withdrawal. User evaluation of this simulator with probe tip feedback suggested a trend of improved performance with haptic feedback. Next, we describe the development of a physically-based camera model for improved behavioral realism of the simulator. Unlike a point-based camera, the enhanced camera model simulates the interaction of the borescope probe, including multiple points of contact along the length of the probe. We present visual comparisons of a real probe\u27s motion with the simulated probe model and develop a simple algorithm for computing the resultant contact forces. User evaluation comparing our custom haptic device with two commonly available haptic devices, the Phantom Omni and the Novint Falcon, suggests that the improved camera model as well as probe contact feedback with the 3D engine model plays a significant role in the overall engine inspection process. Finally, we present results from a skill transfer study comparing classroom-only instruction with both simulator and hands-on training. Students trained using the simulator and the video borescope completed engine inspection using the real video borescope significantly faster than students who received classroom-only training. The speed improvements can be attributed to reduced borescope probe maneuvering time within the engine and improved psychomotor skills due to training. Given the usual constraints of limited time and resources, simulator training may provide beneficial skills needed by novice aircraft maintenance technicians to augment classroom instruction, resulting in a faster transition into the aviation maintenance workforce