Learning Interaction Primitives for Biomechanical Prediction

abstract: This dissertation is focused on developing an algorithm to provide current state estimation and future state predictions for biomechanical human walking features. The goal is to develop a system which is capable of evaluating the current action a subject is taking while walking and then use this to predict the future states of biomechanical features. This work focuses on the exploration and analysis of Interaction Primitives (Amor er al, 2014) and their relevance to biomechanical prediction for human walking. Built on the framework of Probabilistic Movement Primitives, Interaction Primitives utilize an EKF SLAM algorithm to localize and map a distribution over the weights of a set of basis functions. The prediction properties of Bayesian Interaction Primitives were utilized to predict real-time foot forces from a 9 degrees of freedom IMUs mounted to a subjects tibias. This method shows that real-time human biomechanical features can be predicted and have a promising link to real-time controls applications.Dissertation/ThesisMasters Thesis Electrical Engineering 201

Aggressive Aerial Grasping using a Soft Drone with Onboard Perception

Contrary to the stunning feats observed in birds of prey, aerial manipulation and grasping with flying robots still lack versatility and agility. Conventional approaches using rigid manipulators require precise positioning and are subject to large reaction forces at grasp, which limit performance at high speeds. The few reported examples of aggressive aerial grasping rely on motion capture systems, or fail to generalize across environments and grasp targets. We describe the first example of a soft aerial manipulator equipped with a fully onboard perception pipeline, capable of robustly localizing and grasping visually and morphologically varied objects. The proposed system features a novel passively closing tendon-actuated soft gripper that enables fast closure at grasp, while compensating for position errors, complying to the target-object morphology, and dampening reaction forces. The system includes an onboard perception pipeline that combines a neural-network-based semantic keypoint detector with a state-of-the-art robust 3D object pose estimator, whose estimate is further refined using a fixed-lag smoother. The resulting pose estimate is passed to a minimum-snap trajectory planner, tracked by an adaptive controller that fully compensates for the added mass of the grasped object. Finally, a finite-element-based controller determines optimal gripper configurations for grasping. Rigorous experiments confirm that our approach enables dynamic, aggressive, and versatile grasping. We demonstrate fully onboard vision-based grasps of a variety of objects, in both indoor and outdoor environments, and up to speeds of 2.0 m/s -- the fastest vision-based grasp reported in the literature. Finally, we take a major step in expanding the utility of our platform beyond stationary targets, by demonstrating motion-capture-based grasps of targets moving up to 0.3 m/s, with relative speeds up to 1.5 m/s

Autonomous Robots in Dynamic Indoor Environments: Localization and Person-Following

Autonomous social robots have many tasks that they need to address such as localization, mapping, navigation, person following, place recognition, etc. In this thesis we focus on two key components required for the navigation of autonomous robots namely, person following behaviour and localization in dynamic human environments. We propose three novel approaches to address these components; two approaches for person following and one for indoor localization. A convolutional neural networks based approach and an Ada-boost based approach are developed for person following. We demonstrate the results by showing the tracking accuracy over time for this behaviour. For the localization task, we propose a novel approach which can act as a wrapper for traditional visual odometry based approaches to improve the localization accuracy in dynamic human environments. We evaluate this approach by showing how the performance varies with increasing number of dynamic agents present in the scene. This thesis provides qualitative and quantitative evaluations for each of the approaches proposed and show that we perform better than the current approaches

Robust Grasp with Compliant Multi-Fingered Hand

As robots find more and more applications in unstructured environments, the need for grippers able to grasp and manipulate a large variety of objects has brought consistent attention to the use of multi-fingered hands. The hardware development and the control of these devices have become one of the most active research subjects in the field of grasping and dexterous manipulation. Despite a large number of publications on grasp planning, grasping frameworks that strongly depend on information collected by touching the object are getting attention only in recent years. The objective of this thesis focuses on the development of a controller for a robotic system composed of a 7-dof collaborative arm + a 16-dof torque-controlled multi-fingered hand to successfully and robustly grasp various objects. The robustness of the grasp is increased through active interaction between the object and the arm/hand robotic system. Algorithms that rely on the kinematic model of the arm/hand system and its compliance characteristics are proposed and tested on real grasping applications. The obtained results underline the importance of taking advantage of information from hand-object contacts, which is necessary to achieve human-like abilities in grasping tasks

Robotic Crop Interaction in Agriculture for Soft Fruit Harvesting

Autonomous tree crop harvesting has been a seemingly attainable, but elusive, robotics goal for the past several decades. Limiting grower reliance on uncertain seasonal labour is an economic driver of this, but the ability of robotic systems to treat each plant individually also has environmental benefits, such as reduced emissions and fertiliser use. Over the same time period, effective grasping and manipulation (G&M) solutions to warehouse product handling, and more general robotic interaction, have been demonstrated. Despite research progress in general robotic interaction and harvesting of some specific crop types, a commercially successful robotic harvester has yet to be demonstrated. Most crop varieties, including soft-skinned fruit, have not yet been addressed. Soft fruit, such as plums, present problems for many of the techniques employed for their more robust relatives and require special focus when developing autonomous harvesters. Adapting existing robotics tools and techniques to new fruit types, including soft skinned varieties, is not well explored. This thesis aims to bridge that gap by examining the challenges of autonomous crop interaction for the harvesting of soft fruit. Aspects which are known to be challenging include mixed obstacle planning with both hard and soft obstacles present, poor outdoor sensing conditions, and the lack of proven picking motion strategies. Positioning an actuator for harvesting requires solving these problems and others specific to soft skinned fruit. Doing so effectively means addressing these in the sensing, planning and actuation areas of a robotic system. Such areas are also highly interdependent for grasping and manipulation tasks, so solutions need to be developed at the system level. In this thesis, soft robotics actuators, with simplifying assumptions about hard obstacle planes, are used to solve mixed obstacle planning. Persistent target tracking and filtering is used to overcome challenging object detection conditions, while multiple stages of object detection are applied to refine these initial position estimates. Several picking motions are developed and tested for plums, with varying degrees of effectiveness. These various techniques are integrated into a prototype system which is validated in lab testing and extensive field trials on a commercial plum crop. Key contributions of this thesis include I. The examination of grasping & manipulation tools, algorithms, techniques and challenges for harvesting soft skinned fruit II. Design, development and field-trial evaluation of a harvester prototype to validate these concepts in practice, with specific design studies of the gripper type, object detector architecture and picking motion for this III. Investigation of specific G&M module improvements including: o Application of the autocovariance least squares (ALS) method to noise covariance matrix estimation for visual servoing tasks, where both simulated and real experiments demonstrated a 30% improvement in state estimation error using this technique. o Theory and experimentation showing that a single range measurement is sufficient for disambiguating scene scale in monocular depth estimation for some datasets. o Preliminary investigations of stochastic object completion and sampling for grasping, active perception for visual servoing based harvesting, and multi-stage fruit localisation from RGB-Depth data. Several field trials were carried out with the plum harvesting prototype. Testing on an unmodified commercial plum crop, in all weather conditions, showed promising results with a harvest success rate of 42%. While a significant gap between prototype performance and commercial viability remains, the use of soft robotics with carefully chosen sensing and planning approaches allows for robust grasping & manipulation under challenging conditions, with both hard and soft obstacles

Optimal Control of Image Based Visual Servoing (IBVS) for High Precision Visual Inspection Applications

Visual servoing is a control technique that uses image data as feedback in a motion control loop. This technique is useful in tasks that require robots or other automated motion systems to automatically inspect parts or structures in motion. One specific method of visual servoing is Image Based Visual Servoing (IBVS), a method that simply minimizes the differences between an observed image orientation and a desired one. This method works well for orientations where the differences are small, but in the case where the desired orientation is more difficult to reach, the system can become unstable, either driving to infinity through a phenomenon known as camera retreat or following non-optimal and non-repeatable trajectories. This work attempts to address camera retreat and other non-optimal paths by applying dynamic programming, an optimal control method that can determine an optimal trajectory by partitioning possible trajectories into multiple smaller trajectories. Using a cost function to penalize undesirable sub trajectories, the optimal overall trajectory can be determined and initiated. This work attempts to explore an optimized portioned approach using dynamic programming to address camera retreat. The motivation for this is to create a high precision visual servoing sequence suitable for high tolerance automated processes; specifically, quality inspection of airplane wire harnesses

Search Methods for Mobile Manipulator Performance Measurement

Mobile manipulators are a potential solution to the increasing need for additional flexibility and mobility in industrial robotics applications. However, they tend to lack the accuracy and precision achieved by fixed manipulators, especially in scenarios where both the manipulator and the autonomous vehicle move simultaneously. This thesis analyzes the problem of dynamically evaluating the positioning error of mobile manipulators. In particular, it investigates the use of Bayesian methods to predict the position of the end-effector in the presence of uncertainty propagated from the mobile platform. Simulations and real-world experiments are carried out to test the proposed method against a deterministic approach. These experiments are carried out on two mobile manipulators - a proof-of-concept research platform and an industrial mobile manipulator - using ROS and Gazebo. The precision of the mobile manipulator is evaluated through its ability to intercept retroreflective markers using a photoelectric sensor attached to the end-effector. Compared to the deterministic search approach, we observed improved interception capability with comparable search times, thereby enabling the effective performance measurement of the mobile manipulator

First quasi-elastic polarized electron scattering measurement of polarization transfer to protons in a complex nucleus: Oxygen-16

The first polarization transfer measurement in the (e,e\u27 p) reaction on a complex nucleus, a study of the exclusive reaction O16e&ar; ,e\u27p&ar; 15N* , is discussed in this thesis. This experiment was performed at Jefferson Lab, using the Hall A Focal Plane Polarimeter. This was the first experiment to use polarized beam at Jefferson Lab. Simultaneous measurements were also made for the He&ar;, e\u27p&ar; reaction allowing a precise comparison between the form factor ratios for free and bound protons. Since the helicity-dependent, longitudinal P\u27l and transverse P\u27t components of the polarization of the recoil proton are sensitive to the electromagnetic form factors GE and GM of the proton inside the nuclear medium, these polarization transfer observables can be used to look for possible medium modification effects on the form factor ratio for the bound proton. The systematic uncertainties involved in performing a recoil polarimetry measurement are minimal, and the theoretical uncertainty (mainly corrections to the Impulse Approximation) are also small. Therefore this measurement provides a sensitive test of the predictions of medium-induced changes in the form factor ratio. Results were obtained for two recoil momentum points, 85 and 140 MeV/c, at a Q2 of 0.8 (GeV/c)2 in quasielastic, perpendicular kinematics. Within statistical uncertainties, the results are in good agreement with theoretical calculations for the polarization observables obtained assuming free values for the form factors, i.e., the results are consistent with the absence of medium modifications of the nucleon electromagnetic structure. This experiment provides a strong basis for the forthcoming high precision measurements of the ratio GE/GM in the nuclear medium

Robot introspection through learned hidden Markov models

In this paper we describe a machine learning approach for acquiring a model of a robot behaviour from raw sensor data. We are interested in automating the acquisition of behavioural models to provide a robot with an introspective capability. We assume that the behaviour of a robot in achieving a task can be modelled as a finite stochastic state transition system. Beginning with data recorded by a robot in the execution of a task, we use unsupervised learning techniques to estimate a hidden Markov model (HMM) that can be used both for predicting and explaining the behaviour of the robot in subsequent executions of the task. We demonstrate that it is feasible to automate the entire process of learning a high quality HMM from the data recorded by the robot during execution of its task.The learned HMM can be used both for monitoring and controlling the behaviour of the robot. The ultimate purpose of our work is to learn models for the full set of tasks associated with a given problem domain, and to integrate these models with a generative task planner. We want to show that these models can be used successfully in controlling the execution of a plan. However, this paper does not develop the planning and control aspects of our work, focussing instead on the learning methodology and the evaluation of a learned model. The essential property of the models we seek to construct is that the most probable trajectory through a model, given the observations made by the robot, accurately diagnoses, or explains, the behaviour that the robot actually performed when making these observations. In the work reported here we consider a navigation task. We explain the learning process, the experimental setup and the structure of the resulting learned behavioural models. We then evaluate the extent to which explanations proposed by the learned models accord with a human observer's interpretation of the behaviour exhibited by the robot in its execution of the task