Learning Multi-step Robotic Manipulation Tasks through Visual Planning

Multi-step manipulation tasks in unstructured environments are extremely challenging for a robot to learn. Such tasks interlace high-level reasoning that consists of the expected states that can be attained to achieve an overall task and low-level reasoning that decides what actions will yield these states. A model-free deep reinforcement learning method is proposed to learn multi-step manipulation tasks. This work introduces a novel Generative Residual Convolutional Neural Network (GR-ConvNet) model that can generate robust antipodal grasps from n-channel image input at real-time speeds (20ms). The proposed model architecture achieved a state-of-the-art accuracy on three standard grasping datasets. The adaptability of the proposed approach is demonstrated by directly transferring the trained model to a 7 DoF robotic manipulator with a grasp success rate of 95.4% and 93.0% on novel household and adversarial objects, respectively. A novel Robotic Manipulation Network (RoManNet) is introduced, which is a vision-based model architecture, to learn the action-value functions and predict manipulation action candidates. A Task Progress based Gaussian (TPG) reward function is defined to compute the reward based on actions that lead to successful motion primitives and progress towards the overall task goal. To balance the ratio of exploration/exploitation, this research introduces a Loss Adjusted Exploration (LAE) policy that determines actions from the action candidates according to the Boltzmann distribution of loss estimates. The effectiveness of the proposed approach is demonstrated by training RoManNet to learn several challenging multi-step robotic manipulation tasks in both simulation and real-world. Experimental results show that the proposed method outperforms the existing methods and achieves state-of-the-art performance in terms of success rate and action efficiency. The ablation studies show that TPG and LAE are especially beneficial for tasks like multiple block stacking

Improving robotic grasping performance using machine learning techniques

Robots are being utilized in ever more complex tasks and environments to help humans with difficult or dangerous tasks. However, robotic grasping is still in its infancy and is one of the limiting factors which prevent the deployment of robots in the home and other assisted living scenarios. Traditional methods for grasp planning use grasp metrics, which are numerical computations of the kinematic arrangement of the hand and object. However, they are insufficient alone for accounting for all of the variables involved in the grasping process shown by their poor performance when implemented on a robotic platform. We use grasp testing data, along with a machine learning algorithm, in order to learn the complex relationship among all of the grasp metrics so as to improve grasp prediction performance. We then evaluate the resulting machine algorithm to validate the results and compare them to the individual metrics and state of the art grasp planners

Método para evaluación de una estrategia de control realimentado en la funcionalidad de agarre de poder con una prótesis de mano robótica

Objective: In the design of a robotic hand prosthesis, a variety of problems arise for which there are multiple solutions proposed in the literature. One of them is the selection of a strategy for the automatic control of the movement of the hand. Within the functionalities of a prosthesis, the power grip functionality is one of the most important. Given the wide variety of proposals in terms of control techniques for power grip, the designer is faced with the dilemma of which of them is the most suitable for his particular design. Although there are metrics to quantify the quality of the grip, those that have been proposed address various aspects independently, which does not allow making a decision about the best option for the particular case. In this work, a method is proposed to calculate a composite indicator that allows the evaluation of the performance of control techniques in the power grip based on the following individual metrics that evaluate different aspects in the execution of the power grip: finger strength, grip strength, grip efficiency, grip cycle time, slip resistance, following error, and transient response overshoot. Thanks to the method proposed in this work, the question is answered: of several control options to be tested, which one offers better functional performance? Methodology: A set of metrics are adopted with which to obtain quantitative data related to the quality of the power grip and to the performance of the control target to govern the prosthesis; subsequently, selected individual metrics are calculated for each of the grip control techniques to be evaluated on a virtual environment, consisting of a robotic hand and an object to be grasped; then two composite indicators are constructed to obtain a quantitative assessment of the quality of the grip based on a statistical analysis, and the results are contrasted against the individual metrics used. Results: A method is proposed for the construction of a composite indicator, which allows the evaluation of the performance of control techniques, in power grips in robotic hands. By implementing this method, the best performance values were found in hybrid controllers. Conclusions: In this work, a method has been proposed to facilitate the decision-making of the designer regarding the most adequate control technique, among several available, to achieve the power grip with a specific prosthesis. The method seeks to build a composite indicator that groups together various metrics to evaluate particular grip functionality, and also to quantify the achievement of following instructions, facilitating decision-making about the incidence of the control technique in the achievement of the final objective. Financing: This work was supported by University of Cauca under the Master of Science in Automation program.Objetivo: En el diseño de una prótesis de mano robótica surgen variedad de problemáticas para las cuales existen múltiples soluciones propuestas en la literatura. Una de las problemáticas es la selección de una estrategia para el control automático del movimiento de la mano. Entre las funcionalidades de una prótesis, la de agarre de poder es una de las más importantes. Dada la gran cantidad de propuestas en cuanto técnicas de control para el agarre de poder, el diseñador se enfrenta al dilema de cuál de ellas es la más adecuada para su diseño particular. Aunque hay métricas para cuantificar la calidad del agarre, las que se han propuesto abordan diversos aspectos de manera independiente, lo cual no permite tomar una decisión acerca de la mejor opción para el caso particular. En este trabajo se propone un método para calcular un indicador compuesto que permite la evaluación del desempeño de técnicas de control en el agarre de poder, con base en las siguientes métricas individuales que evalúan diferentes aspectos en la ejecución del agarre de poder: la fuerza del dedo, la fuerza del agarre, la eficiencia del agarre, el tiempo de ciclo de agarre, la resistencia al deslizamiento, el error de seguimiento y el sobreimpulso de la respuesta transitoria. Gracias al método propuesto en este trabajo, se da respuesta a la pregunta: “De varias opciones de control a probar, ¿cuál ofrece un mejor desempeño funcional?”. Metodología: Se adoptó un conjunto de métricas con las cuales obtener datos cuantitativos relacionadas con la calidad del agarre de poder y con el desempeño del objetivo de control para gobernar la prótesis. Posteriormente, se calcularon métricas individuales seleccionadas para cada una de las técnicas de control de agarre a evaluar, sobre un entorno virtual, constituido por una mano robótica y un objeto a agarrar. Luego se construyeron dos indicadores compuestos para obtener una valoración cuantitativa de la calidad del agarre a partir de un análisis estadístico, y se contrastaron los resultados contra las métricas individuales utilizadas. Resultados: Se planteó un método para la construcción de un indicador compuesto que permitiera la evaluación del desempeño de técnicas de control, en agarres de poder en manos robóticas. Al implementar dicho método, se encontraron los mejores valores de desempeño en controladores híbridos. Conclusiones: En este trabajo se sugiere una alternativa tendiente a facilitar la toma de decisiones del diseñador en cuanto a la técnica de control más adecuada, entre varias disponibles, para el logro del agarre de poder con una prótesis específica. El método busca construir un indicador compuesto que agrupa variadas métricas para evaluar funcionalidad particular de agarre, y también para cuantificar el logro de seguimiento de consignas, facilitando la toma de decisiones acerca de la incidencia de la técnica de control en el logro del objetivo final

Algorithms for Neural Prosthetic Applications

abstract: In the last 15 years, there has been a significant increase in the number of motor neural prostheses used for restoring limb function lost due to neurological disorders or accidents. The aim of this technology is to enable patients to control a motor prosthesis using their residual neural pathways (central or peripheral). Recent studies in non-human primates and humans have shown the possibility of controlling a prosthesis for accomplishing varied tasks such as self-feeding, typing, reaching, grasping, and performing fine dexterous movements. A neural decoding system comprises mainly of three components: (i) sensors to record neural signals, (ii) an algorithm to map neural recordings to upper limb kinematics and (iii) a prosthetic arm actuated by control signals generated by the algorithm. Machine learning algorithms that map input neural activity to the output kinematics (like finger trajectory) form the core of the neural decoding system. The choice of the algorithm is thus, mainly imposed by the neural signal of interest and the output parameter being decoded. The various parts of a neural decoding system are neural data, feature extraction, feature selection, and machine learning algorithm. There have been significant advances in the field of neural prosthetic applications. But there are challenges for translating a neural prosthesis from a laboratory setting to a clinical environment. To achieve a fully functional prosthetic device with maximum user compliance and acceptance, these factors need to be addressed and taken into consideration. Three challenges in developing robust neural decoding systems were addressed by exploring neural variability in the peripheral nervous system for dexterous finger movements, feature selection methods based on clinically relevant metrics and a novel method for decoding dexterous finger movements based on ensemble methods.Dissertation/ThesisDoctoral Dissertation Bioengineering 201

Tactile Perception And Visuotactile Integration For Robotic Exploration

As the close perceptual sibling of vision, the sense of touch has historically received less than deserved attention in both human psychology and robotics. In robotics, this may be attributed to at least two reasons. First, it suffers from the vicious cycle of immature sensor technology, which causes industry demand to be low, and then there is even less incentive to make existing sensors in research labs easy to manufacture and marketable. Second, the situation stems from a fear of making contact with the environment, avoided in every way so that visually perceived states do not change before a carefully estimated and ballistically executed physical interaction. Fortunately, the latter viewpoint is starting to change. Work in interactive perception and contact-rich manipulation are on the rise. Good reasons are steering the manipulation and locomotion communities’ attention towards deliberate physical interaction with the environment prior to, during, and after a task. We approach the problem of perception prior to manipulation, using the sense of touch, for the purpose of understanding the surroundings of an autonomous robot. The overwhelming majority of work in perception for manipulation is based on vision. While vision is a fast and global modality, it is insufficient as the sole modality, especially in environments where the ambient light or the objects therein do not lend themselves to vision, such as in darkness, smoky or dusty rooms in search and rescue, underwater, transparent and reflective objects, and retrieving items inside a bag. Even in normal lighting conditions, during a manipulation task, the target object and fingers are usually occluded from view by the gripper. Moreover, vision-based grasp planners, typically trained in simulation, often make errors that cannot be foreseen until contact. As a step towards addressing these problems, we present first a global shape-based feature descriptor for object recognition using non-prehensile tactile probing alone. Then, we investigate in making the tactile modality, local and slow by nature, more efficient for the task by predicting the most cost-effective moves using active exploration. To combine the local and physical advantages of touch and the fast and global advantages of vision, we propose and evaluate a learning-based method for visuotactile integration for grasping

Variable autonomy assignment algorithms for human-robot interactions.

As robotic agents become increasingly present in human environments, task completion rates during human-robot interaction has grown into an increasingly important topic of research. Safe collaborative robots executing tasks under human supervision often augment their perception and planning capabilities through traded or shared control schemes. However, such systems are often proscribed only at the most abstract level, with the meticulous details of implementation left to the designer\u27s prerogative. Without a rigorous structure for implementing controls, the work of design is frequently left to ad hoc mechanism with only bespoke guarantees of systematic efficacy, if any such proof is forthcoming at all. Herein, I present two quantitatively defined models for implementing sliding-scale variable autonomy, in which levels of autonomy are determined by the relative efficacy of autonomous subroutines. I experimentally test the resulting Variable Autonomy Planning (VAP) algorithm and against a traditional traded control scheme in a pick-and-place task, and apply the Variable Autonomy Tasking algorithm to the implementation of a robot performing a complex sanitation task in real-world environs. Results show that prioritizing autonomy levels with higher success rates, as encoded into VAP, allows users to effectively and intuitively select optimal autonomy levels for efficient task completion. Further, the Pareto optimal design structure of the VAP+ algorithm allows for significant performance improvements to be made through intervention planning based on systematic input determining failure probabilities through sensorized measurements. This thesis describes the design, analysis, and implementation of these two algorithms, with a particular focus on the VAP+ algorithm. The core conceit is that they are methods for rigorously defining locally optimal plans for traded control being shared between a human and one or more autonomous processes. It is derived from an earlier algorithmic model, the VAP algorithm, developed to address the issue of rigorous, repeatable assignment of autonomy levels based on system data which provides guarantees on basis of the failure-rate sorting of paired autonomous and manual subtask achievement systems. Using only probability ranking to define levels of autonomy, the VAP algorithm is able to sort modules into optimizable ordered sets, but is limited to only solving sequential task assignments. By constructing a joint cost metric for the entire plan, and by implementing a back-to-front calculation scheme for this metric, it is possible for the VAP+ algorithm to generate optimal planning solutions which minimize the expected cost, as amortized over time, funds, accuracy, or any metric combination thereof. The algorithm is additionally very efficient, and able to perform on-line assessments of environmental changes to the conditional probabilities associated with plan choices, should a suitable model for determining these probabilities be present. This system, as a paired set of two algorithms and a design augmentation, form the VAP+ algorithm in full