Prototypical Arm Motions from Human Demonstration for Upper-Limb Prosthetic Device Control

Controlling a complex upper limb prosthesis, akin to a healthy arm, is still an open challenge due to the inadequate number of inputs available to amputees. Designs have therefore largely focused on a limited number of controllable degrees of freedom, developing a complex hand and grasp functionality rather than the wrist. This thesis investigates joint coordination based on human demonstrations that aims to vastly simplify the controls of wrist, elbow-wrist, and shoulder-elbow wrist devices.The wide range of motions performed by the human arm during daily tasks makes it desirable to find representative subsets to reduce the dimensionality of these movements for a variety of applications, including the design and control of robotic and prosthetic devices. Here I present the results of an extensive human subjects study and two methods that were used to obtain representative categories of arm use that span naturalistic motions during activities of daily living. First, I sought to identify sets of prototypical upper-limb motions that are functions of a single variable, allowing, for instance, an entire prosthetic or robotic arm to be controlled with a single input from a user, along with a means to select between motions for different tasks. Second, I decouple the orientation from the location of the hand and analyze the hand location in three ways and orientation in three reference frames. Both of these analyses are an application of data driven approaches that reduce the wide range of hand and arm use to a smaller representative set. Together these provide insight into our arm usage in daily life and inform an implementation in prosthetic or robotic devices without the need for additional hardware. To demonstrate the control efficacy of prototypical arm motions in upper-limb prosthetic devices, I developed an immersive virtual reality environment where able-bodied participants tested out different devices and controls. I coined prototypical arm motion control as trajectory control, and I found that as device complexity increased from 3 DOF wrist to 4 DOF elbow-wrist and 7 DOF shoulder-elbow-wrist, it enables users to complete tasks faster with a more intuitive interface without additional body compensation, while featuring better movement cosmesis when compared to standard controls

Task-adaptable, Pervasive Perception for Robots Performing Everyday Manipulation

Intelligent robotic agents that help us in our day-to-day chores have been an aspiration of robotics researchers for decades. More than fifty years since the creation of the first intelligent mobile robotic agent, robots are still struggling to perform seemingly simple tasks, such as setting or cleaning a table. One of the reasons for this is that the unstructured environments these robots are expected to work in impose demanding requirements on a robota s perception system. Depending on the manipulation task the robot is required to execute, different parts of the environment need to be examined, the objects in it found and functional parts of these identified. This is a challenging task, since the visual appearance of the objects and the variety of scenes they are found in are large. This thesis proposes to treat robotic visual perception for everyday manipulation tasks as an open question-asnswering problem. To this end RoboSherlock, a framework for creating task-adaptable, pervasive perception systems is presented. Using the framework, robot perception is addressed from a systema s perspective and contributions to the state-of-the-art are proposed that introduce several enhancements which scale robot perception toward the needs of human-level manipulation. The contributions of the thesis center around task-adaptability and pervasiveness of perception systems. A perception task-language and a language interpreter that generates task-relevant perception plans is proposed. The task-language and task-interpreter leverage the power of knowledge representation and knowledge-based reasoning in order to enhance the question-answering capabilities of the system. Pervasiveness, a seamless integration of past, present and future percepts, is achieved through three main contributions: a novel way for recording, replaying and inspecting perceptual episodic memories, a new perception component that enables pervasive operation and maintains an object belief state and a novel prospection component that enables robots to relive their past experiences and anticipate possible future scenarios. The contributions are validated through several real world robotic experiments that demonstrate how the proposed system enhances robot perception

Tightly-coupled manipulation pipelines: Combining traditional pipelines and end-to-end learning

Traditionally, robot manipulation tasks are solved by engineering solutions in a modular fashion --- typically consisting of object detection, pose estimation, grasp planning, motion planning, and finally run a control algorithm to execute the planned motion. This traditional approach to robot manipulation separates the hard problem of manipulation into several self-contained stages, which can be developed independently, and gives interpretable outputs at each stage of the pipeline. However, this approach comes with a plethora of issues, most notably, their generalisability to a broad range of tasks; it is common that as tasks get more difficult, the systems become increasingly complex. To combat the flaws of these systems, recent trends have seen robots visually learning to predict actions and grasp locations directly from sensor input in an end-to-end manner using deep neural networks, without the need to explicitly model the in-between modules. This thesis investigates a sample of methods, which fall somewhere on a spectrum from pipelined to fully end-to-end, which we believe to be more advantageous for developing a general manipulation system; one that could eventually be used in highly dynamic and unpredictable household environments. The investigation starts at the far end of the spectrum, where we explore learning an end-to-end controller in simulation and then transferring to the real world by employing domain randomisation, and finish on the other end, with a new pipeline, where the individual modules bear little resemblance to the "traditional" ones. The thesis concludes with a proposition of a new paradigm: Tightly-coupled Manipulation Pipelines (TMP). Rather than learning all modules implicitly in one large, end-to-end network or conversely, having individual, pre-defined modules that are developed independently, TMPs suggest taking the best of both world by tightly coupling actions to observations, whilst still maintaining structure via an undefined number of learned modules, which do not have to bear any resemblance to the modules seen in "traditional" systems.Open Acces

Modeling And Control For Robotic Assistants: Single And Multi-Robot Manipulation

As advances are made in robotic hardware, the complexity of tasks they are capable of performing also increases. One goal of modern robotics is to introduce robotic platforms that require very little augmentation of their environments to be effective and robust. Therefore the challenge for a roboticist is to develop algorithms and control strategies that leverage knowledge of the task while retaining the ability to be adaptive, adjusting to perturbations in the environment and task assumptions. This work considers approaches to these challenges in the context of a wet-lab robotic assistant. The tasks considered are cooperative transport with limited communication between team members, and robot-assisted rapid experiment preparation requiring pouring reagents from open containers useful for research and development scientists. For cooperative transport, robots must be able to plan collision-free trajectories and agree on a final destination to minimize internal forces on the carried load. Robot teammates are considered, where robots must reach consensus to minimize internal forces. The case of a human leader, and robot follower is then considered, where robots must use non-verbal information to estimate the human leader\u27s intended pose for the carried load. For experiment preparation, the robot must pour precisely from open containers with known fluid in a single attempt. Two scenarios examined are when the geometries of the pouring and receiving containers and behaviors are known, and when the pourer must be approximated. An analytical solution is presented for a given geometry in the first instance. In the second instance, a combination of online system identification and leveraging of model priors is used to achieve the precision-pour in a single attempt with considerations for long-term robot deployment. The main contributions of this work are considerations and implementations for making robots capable of performing complex tasks with an emphasis on combining model-based and data-driven approaches for best performance

Design and Development of Assistive Robots for Close Interaction with People with Disabilities

People with mobility and manipulation impairments wish to live and perform tasks as independently as possible; however, for many tasks, compensatory technology does not exist, to do so. Assistive robots have the potential to address this need. This work describes various aspects of the development of three novel assistive robots: the Personal Mobility and Manipulation Appliance (PerMMA), the Robotic Assisted Transfer Device (RATD), and the Mobility Enhancement Robotic Wheelchair (MEBot). PerMMA integrates mobility with advanced bi-manual manipulation to assist people with both upper and lower extremity impairments. The RATD is a wheelchair mounted robotic arm that can lift higher payloads and its primary aim is to assist caregivers of people who cannot independently transfer from their electric powered wheelchair to other surfaces such as a shower bench or toilet. MEBot is a wheeled robot that has highly reconfigurable kinematics, which allow it to negotiate challenging terrain, such as steep ramps, gravel, or stairs. A risk analysis was performed on all three robots which included a Fault Tree Analysis (FTA) and a Failure Mode Effect Analysis (FMEA) to identify potential risks and inform strategies to mitigate them. Identified risks or PerMMA include dropping sharp or hot objects. Critical risks identified for RATD included tip over, crush hazard, and getting stranded mid-transfer, and risks for MEBot include getting stranded on obstacles and tip over. Lastly, several critical factors, such as early involvement of people with disabilities, to guide future assistive robot design are presented