304 research outputs found
Learning High-Level Policies for Model Predictive Control
The combination of policy search and deep neural networks holds the promise
of automating a variety of decision-making tasks. Model Predictive
Control~(MPC) provides robust solutions to robot control tasks by making use of
a dynamical model of the system and solving an optimization problem online over
a short planning horizon. In this work, we leverage probabilistic
decision-making approaches and the generalization capability of artificial
neural networks to the powerful online optimization by learning a deep
high-level policy for the MPC~(High-MPC). Conditioning on robot's local
observations, the trained neural network policy is capable of adaptively
selecting high-level decision variables for the low-level MPC controller, which
then generates optimal control commands for the robot. First, we formulate the
search of high-level decision variables for MPC as a policy search problem,
specifically, a probabilistic inference problem. The problem can be solved in a
closed-form solution. Second, we propose a self-supervised learning algorithm
for learning a neural network high-level policy, which is useful for online
hyperparameter adaptations in highly dynamic environments. We demonstrate the
importance of incorporating the online adaption into autonomous robots by using
the proposed method to solve a challenging control problem, where the task is
to control a simulated quadrotor to fly through a swinging gate. We show that
our approach can handle situations that are difficult for standard MPC
Flymation: Interactive Animation for Flying Robots
Trajectory visualization and animation play critical roles in robotics
research. However, existing data visualization and animation tools often lack
flexibility, scalability, and versatility, resulting in limited capability to
fully explore and analyze flight data. To address this limitation, we introduce
Flymation, a new flight trajectory visualization and animation tool. Built on
the Unity3D engine, Flymation is an intuitive and interactive tool that allows
users to visualize and analyze flight data in real time. Users can import data
from various sources, including flight simulators and real-world data, and
create customized visualizations with high-quality rendering. With Flymation,
users can choose between trajectory snapshot and animation; both provide
valuable insights into the behavior of the underlying autonomous system.
Flymation represents an exciting step toward visualizing and interacting with
large-scale data in robotics research.Comment: This work was presented at Workshop at ICRA 2023 ( The Role of
Robotics Simulators for Unmanned Aerial Vehicles
Contrastive Initial State Buffer for Reinforcement Learning
In Reinforcement Learning, the trade-off between exploration and exploitation
poses a complex challenge for achieving efficient learning from limited
samples. While recent works have been effective in leveraging past experiences
for policy updates, they often overlook the potential of reusing past
experiences for data collection. Independent of the underlying RL algorithm, we
introduce the concept of a Contrastive Initial State Buffer, which
strategically selects states from past experiences and uses them to initialize
the agent in the environment in order to guide it toward more informative
states. We validate our approach on two complex robotic tasks without relying
on any prior information about the environment: (i) locomotion of a quadruped
robot traversing challenging terrains and (ii) a quadcopter drone racing
through a track. The experimental results show that our initial state buffer
achieves higher task performance than the nominal baseline while also speeding
up training convergence
Autonomous Drone Racing with Deep Reinforcement Learning
In many robotic tasks, such as drone racing, the goal is to travel through a
set of waypoints as fast as possible. A key challenge for this task is planning
the minimum-time trajectory, which is typically solved by assuming perfect
knowledge of the waypoints to pass in advance. The resulting solutions are
either highly specialized for a single-track layout, or suboptimal due to
simplifying assumptions about the platform dynamics. In this work, a new
approach to minimum-time trajectory generation for quadrotors is presented.
Leveraging deep reinforcement learning and relative gate observations, this
approach can adaptively compute near-time-optimal trajectories for random track
layouts. Our method exhibits a significant computational advantage over
approaches based on trajectory optimization for non-trivial track
configurations. The proposed approach is evaluated on a set of race tracks in
simulation and the real world, achieving speeds of up to 17 m/s with a physical
quadrotor
The influence of additive content on microstructure and mechanical properties on the Csf/SiC composites after annealed treatment
AbstractIn this paper, micrometers long and 20–100nm diameter SiC nanowires had been synthesized in the short cut fiber toughened SiC composites (Csf/SiC) by annealing treatment. The present work demonstrated that it was possible to fabricate the in situ SiC nanowires toughened Csf/SiC composites by annealed treatment. The “vapor–liquid–solid” growth mechanism of the SiC nanowires was proposed. The mainly toughened mechanism concluded grain bridging, crack deflection, fiber debonding and SiC nanowires, which can improve fracture toughness
Learning Minimum-Time Flight in Cluttered Environments
We tackle the problem of minimum-time flight for a quadrotor through a
sequence of waypoints in the presence of obstacles while exploiting the full
quadrotor dynamics. Early works relied on simplified dynamics or polynomial
trajectory representations that did not exploit the full actuator potential of
the quadrotor, and, thus, resulted in suboptimal solutions. Recent works can
plan minimum-time trajectories; yet, the trajectories are executed with control
methods that do not account for obstacles. Thus, a successful execution of such
trajectories is prone to errors due to model mismatch and in-flight
disturbances. To this end, we leverage deep reinforcement learning and
classical topological path planning to train robust neural-network controllers
for minimum-time quadrotor flight in cluttered environments. The resulting
neural network controller demonstrates significantly better performance of up
to 19% over state-of-the-art methods. More importantly, the learned policy
solves the planning and control problem simultaneously online to account for
disturbances, thus achieving much higher robustness. As such, the presented
method achieves 100% success rate of flying minimum-time policies without
collision, while traditional planning and control approaches achieve only 40%.
The proposed method is validated in both simulation and the real world
Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flight
Scene transfer for vision-based mobile robotics applications is a highly
relevant and challenging problem. The utility of a robot greatly depends on its
ability to perform a task in the real world, outside of a well-controlled lab
environment. Existing scene transfer end-to-end policy learning approaches
often suffer from poor sample efficiency or limited generalization
capabilities, making them unsuitable for mobile robotics applications. This
work proposes an adaptive multi-pair contrastive learning strategy for visual
representation learning that enables zero-shot scene transfer and real-world
deployment. Control policies relying on the embedding are able to operate in
unseen environments without the need for finetuning in the deployment
environment. We demonstrate the performance of our approach on the task of
agile, vision-based quadrotor flight. Extensive simulation and real-world
experiments demonstrate that our approach successfully generalizes beyond the
training domain and outperforms all baselines
- …