4,190 research outputs found
Autonomous Robust Skill Generation Using Reinforcement Learning with Plant Variation
This paper discusses an autonomous space robot for a truss structure assembly using some reinforcement learning. It is difficult for a space robot to complete contact tasks within a real environment, for example, a peg-in-hole task, because of error between the real environment and the controller model. In order to solve problems, we propose an autonomous space robot able to obtain proficient and robust skills by overcoming error to complete a task. The proposed approach develops skills by reinforcement learning that considers plant variation, that is, modeling error. Numerical simulations and experiments show the proposed method is useful in real environments
Multi-agent evolutionary systems for the generation of complex virtual worlds
Modern films, games and virtual reality applications are dependent on
convincing computer graphics. Highly complex models are a requirement for the
successful delivery of many scenes and environments. While workflows such as
rendering, compositing and animation have been streamlined to accommodate
increasing demands, modelling complex models is still a laborious task. This
paper introduces the computational benefits of an Interactive Genetic Algorithm
(IGA) to computer graphics modelling while compensating the effects of user
fatigue, a common issue with Interactive Evolutionary Computation. An
intelligent agent is used in conjunction with an IGA that offers the potential
to reduce the effects of user fatigue by learning from the choices made by the
human designer and directing the search accordingly. This workflow accelerates
the layout and distribution of basic elements to form complex models. It
captures the designer's intent through interaction, and encourages playful
discovery
Reinforcement Learning Approaches in Social Robotics
This article surveys reinforcement learning approaches in social robotics.
Reinforcement learning is a framework for decision-making problems in which an
agent interacts through trial-and-error with its environment to discover an
optimal behavior. Since interaction is a key component in both reinforcement
learning and social robotics, it can be a well-suited approach for real-world
interactions with physically embodied social robots. The scope of the paper is
focused particularly on studies that include social physical robots and
real-world human-robot interactions with users. We present a thorough analysis
of reinforcement learning approaches in social robotics. In addition to a
survey, we categorize existent reinforcement learning approaches based on the
used method and the design of the reward mechanisms. Moreover, since
communication capability is a prominent feature of social robots, we discuss
and group the papers based on the communication medium used for reward
formulation. Considering the importance of designing the reward function, we
also provide a categorization of the papers based on the nature of the reward.
This categorization includes three major themes: interactive reinforcement
learning, intrinsically motivated methods, and task performance-driven methods.
The benefits and challenges of reinforcement learning in social robotics,
evaluation methods of the papers regarding whether or not they use subjective
and algorithmic measures, a discussion in the view of real-world reinforcement
learning challenges and proposed solutions, the points that remain to be
explored, including the approaches that have thus far received less attention
is also given in the paper. Thus, this paper aims to become a starting point
for researchers interested in using and applying reinforcement learning methods
in this particular research field
On the Utility of Koopman Operator Theory in Learning Dexterous Manipulation Skills
Despite impressive dexterous manipulation capabilities enabled by
learning-based approaches, we are yet to witness widespread adoption beyond
well-resourced laboratories. This is likely due to practical limitations, such
as significant computational burden, inscrutable learned behaviors, sensitivity
to initialization, and the considerable technical expertise required for
implementation. In this work, we investigate the utility of Koopman operator
theory in alleviating these limitations. Koopman operators are simple yet
powerful control-theoretic structures to represent complex nonlinear dynamics
as linear systems in higher dimensions. Motivated by the fact that complex
nonlinear dynamics underlie dexterous manipulation, we develop a Koopman
operator-based imitation learning framework to learn the desired motions of
both the robotic hand and the object simultaneously. We show that Koopman
operators are surprisingly effective for dexterous manipulation and offer a
number of unique benefits. Notably, policies can be learned analytically,
drastically reducing computation burden and eliminating sensitivity to
initialization and the need for painstaking hyperparameter optimization. Our
experiments reveal that a Koopman operator-based approach can perform
comparably to state-of-the-art imitation learning algorithms in terms of
success rate and sample efficiency, while being an order of magnitude faster.
Policy videos can be viewed at https://sites.google.com/view/kodex-corl.Comment: This work has been accepted for an oral presentation at CORL 202
Synergy-based policy improvement with path integrals for anthropomorphic hands
In this work, a synergy-based reinforcement learning
algorithm has been developed to confer autonomous grasping
capabilities to anthropomorphic hands. In the presence of
high degrees of freedom, classical machine learning techniques
require a number of iterations that increases with the size of the
problem, thus convergence of the solution is not ensured. The
use of postural synergies determines dimensionality reduction
of the search space and allows recent learning techniques, such
as Policy Improvement with Path Integrals, to become easily
applicable. A key point is the adoption of a suitable reward
function representing the goal of the task and ensuring onestep
performance evaluation. Force-closure quality of the grasp
in the synergies subspace has been chosen as a cost function
for performance evaluation. The experiments conducted on the
SCHUNK 5-Finger Hand demonstrate the effectiveness of the
algorithm showing skills comparable to human capabilities in
learning new grasps and in performing a wide variety from
power to high precision grasps of very small objects
Learning to reach and reaching to learn: a unified approach to path planning and reactive control through reinforcement learning
The next generation of intelligent robots will need to be able to plan reaches. Not just ballistic point to point reaches, but reaches around things such as the edge of a table, a nearby human, or any other known object in the robotâs workspace. Planning reaches may seem easy to us humans, because we do it so intuitively, but it has proven to be a challenging problem, which continues to limit the versatility of what robots can do today. In this document, I propose a novel intrinsically motivated RL system that draws on both Path/Motion Planning and Reactive Control. Through Reinforcement Learning, it tightly integrates these two previously disparate approaches to robotics. The RL system is evaluated on a task, which is as yet unsolved by roboticists in practice. That is to put the palm of the iCub humanoid robot on arbitrary target objects in its workspace, start- ing from arbitrary initial configurations. Such motions can be generated by planning, or searching the configuration space, but this typically results in some kind of trajectory, which must then be tracked by a separate controller, and such an approach offers a brit- tle runtime solution because it is inflexible. Purely reactive systems are robust to many problems that render a planned trajectory infeasible, but lacking the capacity to search, they tend to get stuck behind constraints, and therefore do not replace motion planners. The planner/controller proposed here is novel in that it deliberately plans reaches without the need to track trajectories. Instead, reaches are composed of sequences of reactive motion primitives, implemented by my Modular Behavioral Environment (MoBeE), which provides (fictitious) force control with reactive collision avoidance by way of a realtime kinematic/geometric model of the robot and its workspace. Thus, to the best of my knowledge, mine is the first reach planning approach to simultaneously offer the best of both the Path/Motion Planning and Reactive Control approaches. By controlling the real, physical robot directly, and feeling the influence of the con- straints imposed by MoBeE, the proposed system learns a stochastic model of the iCubâs configuration space. Then, the model is exploited as a multiple query path planner to find sensible pre-reach poses, from which to initiate reaching actions. Experiments show that the system can autonomously find practical reaches to target objects in workspace and offers excellent robustness to changes in the workspace configuration as well as noise in the robotâs sensory-motor apparatus
Outdoor operations of multiple quadrotors in windy environment
Coordinated multiple small unmanned aerial vehicles (sUAVs) offer several advantages over a single sUAV platform. These advantages include improved task efficiency, reduced task completion time, improved fault tolerance, and higher task flexibility. However, their deployment in an outdoor environment is challenging due to the presence of wind gusts. The coordinated motion of a multi-sUAV system in the presence of wind disturbances is a challenging problem when considering collision avoidance (safety), scalability, and communication connectivity. Performing wind-agnostic motion planning for sUAVs may produce a sizeable cross-track error if the wind on the planned route leads to actuator saturation. In a multi-sUAV system, each sUAV has to locally counter the wind disturbance while maintaining the safety of the system. Such continuous manipulation of the control effort for multiple sUAVs under uncertain environmental conditions is computationally taxing and can lead to reduced efficiency and safety concerns. Additionally, modern day sUAV systems are susceptible to cyberattacks due to their use of commercial wireless communication infrastructure.
This dissertation aims to address these multi-faceted challenges related to the operation of outdoor rotor-based multi-sUAV systems. A comprehensive review of four representative techniques to measure and estimate wind speed and direction using rotor-based sUAVs is discussed. After developing a clear understanding of the role wind gusts play in quadrotor motion, two decentralized motion planners for a multi-quadrotor system are implemented and experimentally evaluated in the presence of wind disturbances. The first planner is rooted in the reinforcement learning (RL) technique of state-action-reward-state-action (SARSA) to provide generalized path plans in the presence of wind disturbances. While this planner provides feasible trajectories for the quadrotors, it does not provide guarantees of collision avoidance. The second planner implements a receding horizon (RH) mixed-integer nonlinear programming (MINLP) model that is integrated with control barrier functions (CBFs) to guarantee collision-free transit of the multiple quadrotors in the presence of wind disturbances. Finally, a novel communication protocol using Ethereum blockchain-based smart contracts is presented to address the challenge of secure wireless communication.
The U.S. sUAV market is expected to be worth $92 Billion by 2030. The Association for Unmanned Vehicle Systems International (AUVSI) noted in its seminal economic report that UAVs would be responsible for creating 100,000 jobs by 2025 in the U.S. The rapid proliferation of drone technology in various applications has led to an increasing need for professionals skilled in sUAV piloting, designing, fabricating, repairing, and programming. Engineering educators have recognized this demand for certified sUAV professionals.
This dissertation aims to address this growing sUAV-market need by evaluating two active learning-based instructional approaches designed for undergraduate sUAV education. The two approaches leverages the interactive-constructive-active-passive (ICAP) framework of engagement and explores the use of Competition based Learning (CBL) and Project based Learning (PBL). The CBL approach is implemented through a drone building and piloting competition that featured 97 students from undergraduate and graduate programs at NJIT. The competition focused on 1) drone assembly, testing, and validation using commercial off-the-shelf (COTS) parts, 2) simulation of drone flight missions, and 3) manual and semi-autonomous drone piloting were implemented. The effective student learning experience from this competition served as the basis of a new undergraduate course on drone science fundamentals at NJIT. This undergraduate course focused on the three foundational pillars of drone careers: 1) drone programming using Python, 2) designing and fabricating drones using Computer-Aided Design (CAD) and rapid prototyping, and 3) the US Federal Aviation Administration (FAA) Part 107 Commercial small Unmanned Aerial Vehicles (sUAVs) pilot test. Multiple assessment methods are applied to examine the studentsâ gains in sUAV skills and knowledge and student attitudes towards an active learning-based approach for sUAV education. The use of active learning techniques to address these challenges lead to meaningful student engagement and positive gains in the learning outcomes as indicated by quantitative and qualitative assessments
- âŠ