537 research outputs found

    Benchmarking Cerebellar Control

    Get PDF
    Cerebellar models have long been advocated as viable models for robot dynamics control. Building on an increasing insight in and knowledge of the biological cerebellum, many models have been greatly refined, of which some computational models have emerged with useful properties with respect to robot dynamics control. Looking at the application side, however, there is a totally different picture. Not only is there not one robot on the market which uses anything remotely connected with cerebellar control, but even in research labs most testbeds for cerebellar models are restricted to toy problems. Such applications hardly ever exceed the complexity of a 2 DoF simulated robot arm; a task which is hardly representative for the field of robotics, or relates to realistic applications. In order to bring the amalgamation of the two fields forwards, we advocate the use of a set of robotics benchmarks, on which existing and new computational cerebellar models can be comparatively tested. It is clear that the traditional approach to solve robotics dynamics loses ground with the advancing complexity of robotic structures; there is a desire for adaptive methods which can compete as traditional control methods do for traditional robots. In this paper we try to lay down the successes and problems in the fields of cerebellar modelling as well as robot dynamics control. By analyzing the common ground, a set of benchmarks is suggested which may serve as typical robot applications for cerebellar models

    Models for reinforcement learning and design of a soft robot inspired by Drosophila larvae

    Get PDF
    Designs for robots are often inspired by animals, as they are designed mimicking animals’ mechanics, motions, behaviours and learning. The Drosophila, known as the fruit fly, is a well-studied model animal. In this thesis, the Drosophila larva is studied and the results are applied to robots. More specifically: a part of the Drosophila larva’s neural circuit for operant learning is modelled, based on which a synaptic plasticity model and a neural circuit model for operant learning, as well as a dynamic neural network for robot reinforcement learning, are developed; then Drosophila larva’s motor system for locomotion is studied, and based on it a soft robot system is designed. Operant learning is a concept similar to reinforcement learning in computer science, i.e. learning by reward or punishment for behaviour. Experiments have shown that a wide range of animals is capable of operant learning, including animal with only a few neurons, such as Drosophila. The fact implies that operant learning can establish without a large number of neurons. With it as an assumption, the structure and dynamics of synapses are investigated, and a synaptic plasticity model is proposed. The model includes nonlinear dynamics of synapses, especially receptor trafficking which affects synaptic strength. Tests of this model show it can enable operant learning at the neuron level and apply to a broad range of NNs, including feedforward, recurrent and spiking NNs. The mushroom body is a learning centre of the insect brain known and modelled for associative learning, but not yet for operant learning. To investigate whether it participates in operant learning, Drosophila larvae are studied with a transgenic tool by my collaborators. Based on the experiment and the results, a mushroom body model capable of operant learning is modelled. The proposed neural circuit model can reproduce the operant learning of the turning behaviour of Drosophila larvae. Then the synaptic plasticity model is simplified for robot learning. With the simplified model, a recurrent neural network with internal neural dynamics can learn to control a planar bipedal robot in a benchmark reinforcement learning task which is called bipedal walker by OpenAI. Benefiting efficiency in parameter space exploration instead of action space exploration, it is the first known solution to the task with reinforcement learning approaches. Although existing pneumatic soft robots can have multiple muscles embedded in a component, it is far less than the muscles in the Drosophila larva, which are well-organised in a tiny space. A soft robot system is developed based on the muscle pattern of the Drosophila larva, to explore the possibility to embed a high density of muscles in a limited space. Three versions of the body wall with pneumatic muscles mimicking the muscle pattern are designed. A pneumatic control system and embedded control system are also developed for controlling the robot. With a bioinspired body wall will a large number of muscles, the robot performs lifelike motions in experiments

    Design of a pneumatic soft robotic actuator using model-based optimization

    Get PDF
    In this thesis, the design and optimization process of a novel soft intelligent modular pad (IntelliPad) for the purpose of pressure injury prevention is presented. The structure of the IntelliPad consists of multiple individual multi-chamber soft pneumatic-driven actuators that use pressurized air and vacuum. Each actuator is able to provide both vertical and horizontal motions that can be controlled independently. An analytical modeling approach using multiple cantilever beams and virtual springs connected in a closed formed structure was developed to analyze the mechanical performance of the actuator. The analytical approach was validated by a finite element analysis. For optimizing the actuator\u27s mechanical performance, firefly algorithm and deep reinforcement learning-based design optimization frameworks were developed with the purpose of maximizing the horizontal motion of the top surface of the actuators, while minimizing its corresponding effect on the vertical motion. Four optimized designs were fabricated. The actuators were tested and validated experimentally to demonstrate their required mechanical performance in order to regulate normal and shear stresses at the skin-pad interface for pressure injury prevention applications

    Locomotion Optimization of Photoresponsive Small-scale Robot: A Deep Reinforcement Learning Approach

    Get PDF
    Soft robots comprise of elastic and flexible structures, and actuatable soft materials are often used to provide stimuli-responses, remotely controlled with different kinds of external stimuli, which is beneficial for designing small-scale devices. Among different stimuli-responsive materials, liquid crystal networks (LCNs) have gained a significant amount of attention for soft small-scale robots in the past decade being stimulated and actuated by light, which is clean energy, able to transduce energy remotely, easily available and accessible to sophisticated control. One of the persistent challenges in photoresponsive robotics is to produce controllable autonomous locomotion behavior. In this Thesis, different types of photoresponsive soft robots were used to realize light-powered locomotion, and an artificial intelligence-based approach was developed for controlling the movement. A robot tracking system, including an automatic laser steering function, was built for efficient robotic feature detection and steering the laser beam automatically to desired locations. Another robot prototype, a swimmer robot, driven by the automatically steered laser beam, showed directional movements including some degree of uncertainty and randomness in their locomotion behavior. A novel approach is developed to deal with the challenges related to the locomotion of photoresponsive swimmer robots. Machine learning, particularly deep reinforcement learning method, was applied to develop a control policy for autonomous locomotion behavior. This method can learn from its experiences by interacting with the robot and its environment without explicit knowledge of the robot structure, constituent material, and robotic mechanics. Due to the requirement of a large number of experiences to correlate the goodness of behavior control, a simulator was developed, which mimicked the uncertain and random movement behavior of the swimmer robots. This approach effectively adapted the random movement behaviors and developed an optimal control policy to reach different destination points autonomously within a simulated environment. This work has successfully taken a step towards the autonomous locomotion control of soft photoresponsive robots

    Multi-modal Skill Memories for Online Learning of Interactive Robot Movement Generation

    Get PDF
    Queißer J. Multi-modal Skill Memories for Online Learning of Interactive Robot Movement Generation. Bielefeld: Universität Bielefeld; 2018.Modern robotic applications pose complex requirements with respect to the adaptation of actions regarding the variability in a given task. Reinforcement learning can optimize for changing conditions, but relearning from scratch is hardly feasible due to the high number of required rollouts. This work proposes a parameterized skill that generalizes to new actions for changing task parameters. The actions are encoded by a meta-learner that provides parameters for task-specific dynamic motion primitives. Experimental evaluation shows that the utilization of parameterized skills for initialization of the optimization process leads to a more effective incremental task learning. A proposed hybrid optimization method combines a fast coarse optimization on a manifold of policy parameters with a fine-grained parameter search in the unrestricted space of actions. It is shown that the developed algorithm reduces the number of required rollouts for adaptation to new task conditions. Further, this work presents a transfer learning approach for adaptation of learned skills to new situations. Application in illustrative toy scenarios, for a 10-DOF planar arm, a humanoid robot point reaching task and parameterized drumming on a pneumatic robot validate the approach. But parameterized skills that are applied on complex robotic systems pose further challenges: the dynamics of the robot and the interaction with the environment introduce model inaccuracies. In particular, high-level skill acquisition on highly compliant robotic systems such as pneumatically driven or soft actuators is hardly feasible. Since learning of the complete dynamics model is not feasible due to the high complexity, this thesis examines two alternative approaches: First, an improvement of the low-level control based on an equilibrium model of the robot. Utilization of an equilibrium model reduces the learning complexity and this thesis evaluates its applicability for control of pneumatic and industrial light-weight robots. Second, an extension of parameterized skills to generalize for forward signals of action primitives that result in an enhanced control quality of complex robotic systems. This thesis argues for a shift in the complexity of learning the full dynamics of the robot to a lower dimensional task-related learning problem. Due to the generalization in relation to the task variability, online learning for complex robots as well as complex scenarios becomes feasible. An experimental evaluation investigates the generalization capabilities of the proposed online learning system for robot motion generation. Evaluation is performed through simulation of a compliant 2-DOF arm and scalability to a complex robotic system is demonstrated for a pneumatically driven humanoid robot with 8-DOF

    The design and mathematical model of a novel variable stiffness extensor-contractor pneumatic artificial muscle

    Get PDF
    This article presents the design of a novel Extensor-Contractor Pneumatic Artificial Muscle (ECPAM). This new actuator has numerous advantages over traditional pneumatic artificial muscles. These include the ability to both contract and extend relative to a nominal initial length, the ability to generate both contraction and extension forces and the ability to vary stiffness at any actuator length. A kinematic analysis of the ECPAM is presented in this article. A new output force mathematical model has been developed for the ECPAM based on its kinematic analysis and the theory of energy conservation. The correlation between experimental results and the new mathematical model has been investigated and show good correlation. Numerous stiffness experiments have been conducted to validate the variable stiffness ability of the actuator at a series of specific fixed lengths. This has proven that actuator stiffness can be adjusted independently of actuator length. Finally a stiffness-position controller has been developed to validate the effectiveness of the novel actuator

    Surrogate models for the design and control of soft mechanical systems

    Get PDF
    Soft mechanical systems constitute stretchable skins, tissue-like appendages, fibers and fluids, and utilize material deformation to transmit forces or motion to perform a mechanical task. These systems may possess infinite degrees of freedom with finite modes of actuation and sensing, and this creates challenges in modeling, design and controls. This thesis explores the use of surrogate models to approximate the complex physics between the inputs and outputs of a soft mechanical system composed of a ubiquitous soft building block known as Fiber Reinforced Elastomeric Enclosures (FREEs). Towards this the thesis is divided into two parts, with the first part investigating reduced order models for design and the other part investigating reinforcement learning (RL) framework for controls. The reduced order models for design is motivated by the need for repeated quick and accurate evaluation of the system performance. Two mechanics-based models are investigated: (a) A Pseudo Rigid Body model (PRB) with lumped spring and link elements, and (b) a Homogenized Strain Induced (HIS) model that can be implemented in a finite element framework. The parameters of the two models are fit either directly with experiments on FREE prototypes or with a high fidelity robust finite element model. These models capture fundamental insights on design by isolating a fundamental dyad building block of contracting FREEs that can be configured to either obtain large stroke (displacement) or large force. Furthermore, the thesis proposes a novel building block-based design framework where soft FREE actuators are systematically integrated in a compliant system to yield a given motion requirement. The design process is deemed useful in shape morphing adaptive structures such as airfoils, soft skins, and wearable devices for the upper extremities. Soft robotic systems such as manipulators are challenging to control because of their flexibility, ability to undergo large spatial deformations that are dependent on the external load. The second part of this work focuses on the control of a unique soft continuum arm known as the BR2 manipulator using reinforcement learning (RL). The BR2 manipulator has a unique parallel architecture with a combined bending mode and torsional modes, and its inherent asymmetric nature precludes well defined analytical models to capture its forward kinematics. Two RL-based frameworks are evaluated on the BR2 manipulator and their efficacy in carrying out position control using simple state feedback is reported in this work. The results highlight external load invariance of the learnt control policies which is a significant factor for deformable continuum arms for applications involving pick and place operations. The manipulator is deemed useful in berry harvesting and other agricultural applications

    Adaptive control of compliant robots with Reservoir Computing

    Get PDF
    In modern society, robots are increasingly used to handle dangerous, repetitive and/or heavy tasks with high precision. Because of the nature of the tasks, either being dangerous, high precision or simply repetitive, robots are usually constructed with high torque motors and sturdy materials, that makes them dangerous for humans to handle. In a car-manufacturing company, for example, a large cage is placed around the robot’s workspace that prevents humans from entering its vicinity. In the last few decades, efforts have been made to improve human-robot interaction. Often the movement of robots is characterized as not being smooth and clearly dividable into sub-movements. This makes their movement rather unpredictable for humans. So, there exists an opportunity to improve the motion generation of robots to enhance human-robot interaction. One interesting research direction is that of imitation learning. Here, human motions are recorded and demonstrated to the robot. Although the robot is able to reproduce such movements, it cannot be generalized to other situations. Therefore, a dynamical system approach is proposed where the recorded motions are embedded into the dynamics of the system. Shaping these nonlinear dynamics, according to recorded motions, allows for dynamical system to generalize beyond demonstration. As a result, the robot can generate motions of other situations not included in the recorded human demonstrations. In this dissertation, a Reservoir Computing approach is used to create a dynamical system in which such demonstrations are embedded. Reservoir Computing systems are Recurrent Neural Network-based approaches that are efficiently trained by considering only the training of the readout connections and retaining all other connections of such a network unchanged given their initial randomly chosen values. Although they have been used to embed periodic motions before, they were extended to embed discrete motions, or both. This work describes how such a motion pattern-generating system is built, investigates the nature of the underlying dynamics and evaluates their robustness in the face of perturbations. Additionally, a dynamical system approach to obstacle avoidance is proposed that is based on vector fields in the presence of repellers. This technique can be used to extend the motion abilities of the robot without need for changing the trained Motion Pattern Generator (MPG). Therefore, this approach can be applied in real-time on any system that generates a certain movement trajectory. Assume that the MPG system is implemented on an industrial robotic arm, similar to the ones used in a car factory. Even though the obstacle avoidance strategy presented is able to modify the generated motion of the robot’s gripper in such a way that it avoids obstacles, it does not guarantee that other parts of the robot cannot collide with a human. To prevent this, engineers have started to use advanced control algorithms that measure the amount of torque that is applied on the robot. This allows the robot to be aware of external perturbations. However, it turns out that, even with fast control loops, the adaptation to compensate for a sudden perturbation, is too slow to prevent high interaction forces. To reduce such forces, researchers started to use mechanical elements that are passively compliant (e.g., springs) and light-weight flexible materials to construct robots. Although such compliant robots are much safer and inherently energy efficient to use, their control becomes much harder. Most control approaches use model information about the robot (e.g., weight distribution and shape). However, when constructing a compliant robot it is hard to determine the dynamics of these materials. Therefore, a model-free adaptive control framework is proposed that assumes no prior knowledge about the robot. By interacting with the robot it learns an inverse robot model that is used as controller. The more it interacts, the better the control be- comes. Appropriately, this framework is called Inverse Modeling Adaptive (IMA) control framework. I have evaluated the IMA controller’s tracking ability on sev- eral tasks, investigating its model independence and stability. Furthermore, I have shown its fast learning ability and comparable performance to taskspecific designed controllers. Given both the MPG and IMA controllers, it is possible to improve the inter- actability of a compliant robot in a human-friendly environment. When the robot is to perform human-like motions for a large set of tasks, we need to demonstrate motion examples of all these tasks. However, biological research concerning the motion generation of animals and humans revealed that a limited set of motion patterns, called motion primitives, are modulated and combined to generate advanced motor/motion skills that humans and animals exhibit. Inspired by these interesting findings, I investigate if a single motion primitive indeed can be modulated to achieve a desired motion behavior. By some elementary experiments, where an MPG is controlled by an IMA controller, a proof of concept is presented. Furthermore, a general hierarchy is introduced that describes how a robot can be controlled in a biology-inspired manner. I also investigated how motion primitives can be combined to produce a desired motion. However, I was unable to get more advanced implementations to work. The results of some simple experiments are presented in the appendix. Another approach I investigated assumes that the primitives themselves are undefined. Instead, only a high-level description is given, which describes that every primitive on average should contribute equally, while still allowing for a single primitive to specialize in a part of the motion generation. Without defining the behavior of a primitive, only a set of untrained IMA controllers is used of which each will represent a single primitive. As a result of the high-level heuristic description, the task space is tiled into sub-regions in an unsupervised manner. Resulting in controllers that indeed represent a part of the motion generation. I have applied this Modular Architecture with Control Primitives (MACOP) on an inverse kinematic learning task and investigated the emerged primitives. Thanks to the tiling of the task space, it becomes possible to control redundant systems, because redundant solutions can be spread over several control primitives. Within each sub region of the task space, a specific control primitive is more accurate than in other regions allowing for the task complexity to be distributed over several less complex tasks. Finally, I extend the use of an IMA-controller, which is tracking controller, to the control of under-actuated systems. By using a sample-based planning algorithm it becomes possible to explore the system dynamics in which a path to a desired state can be planned. Afterwards, MACOP is used to incorporate feedback and to learn the necessary control commands corresponding to the planned state space trajectory, even if it contains errors. As a result, the under-actuated control of a cart pole system was achieved. Furthermore, I presented the concept of a simulation based control framework that allows the learning of the system dynamics, planning and feedback control iteratively and simultaneously

    Modeling, simulation, and control of soft robots

    Get PDF
    2019 Fall.Includes bibliographical references.Soft robots are a new type of robot with deformable bodies and muscle-like actuations, which are fundamentally different from traditional robots with rigid links and motor-based actuators. Owing to their elasticity, soft robots outperform rigid ones in safety, maneuverability, and adaptability. With their advantages, many soft robots have been developed for manipulation and locomotion in recent years. However, the current state of soft robotics has significant design and development work, but lags behind in modeling and control due to the complex dynamic behavior of the soft bodies. This complexity prevents a unified dynamics model that captures the dynamic behavior, computationally-efficient algorithms to simulate the dynamics in real-time, and closed-loop control algorithms to accomplish desired dynamic responses. In this thesis, we address the three problems of modeling, simulation, and control of soft robots. For the modeling, we establish a general modeling framework for the dynamics by integrating Cosserat theory with Hamilton's principle. Such a framework can accommodate different actuation methods (e.g., pneumatic, cable-driven, artificial muscles, etc.). To simulate the proposed models, we develop efficient numerical algorithms and implement them in C++ to simulate the dynamics of soft robots in real-time. These algorithms consider qualities of the dynamics that are typically neglected (e.g., numerical damping, group structure). Using the developed numerical algorithms, we investigate the control of soft robots with the goal of achieving real-time and closed-loop control policies. Several control approaches are tested (e.g., model predictive control, reinforcement learning) for a few key tasks: reaching various points in a soft manipulator's workspace and tracking a given trajectory. The results show that model predictive control is possible but is computationally demanding, while reinforcement learning techniques are more computationally effective but require a substantial number of training samples. The modeling, simulation, and control framework developed in this thesis will lay a solid foundation to unleash the potential of soft robots for various applications, such as manipulation and locomotion
    • …
    corecore