4,430 research outputs found
Scaling reinforcement learning to the unconstrained multi-agent domain
Reinforcement learning is a machine learning technique designed to mimic the
way animals learn by receiving rewards and punishment. It is designed to train
intelligent agents when very little is known about the agent’s environment, and consequently
the agent’s designer is unable to hand-craft an appropriate policy. Using
reinforcement learning, the agent’s designer can merely give reward to the agent when
it does something right, and the algorithm will craft an appropriate policy automatically.
In many situations it is desirable to use this technique to train systems of agents
(for example, to train robots to play RoboCup soccer in a coordinated fashion). Unfortunately,
several significant computational issues occur when using this technique
to train systems of agents. This dissertation introduces a suite of techniques that
overcome many of these difficulties in various common situations.
First, we show how multi-agent reinforcement learning can be made more tractable
by forming coalitions out of the agents, and training each coalition separately. Coalitions
are formed by using information-theoretic techniques, and we find that by using
a coalition-based approach, the computational complexity of reinforcement-learning
can be made linear in the total system agent count. Next we look at ways to integrate
domain knowledge into the reinforcement learning process, and how this can signifi-cantly improve the policy quality in multi-agent situations. Specifically, we find that
integrating domain knowledge into a reinforcement learning process can overcome training data deficiencies and allow the learner to converge to acceptable solutions
when lack of training data would have prevented such convergence without domain
knowledge. We then show how to train policies over continuous action spaces, which
can reduce problem complexity for domains that require continuous action spaces
(analog controllers) by eliminating the need to finely discretize the action space. Finally,
we look at ways to perform reinforcement learning on modern GPUs and show
how by doing this we can tackle significantly larger problems. We find that by offloading
some of the RL computation to the GPU, we can achieve almost a 4.5 speedup
factor in the total training process
A Survey on Physics Informed Reinforcement Learning: Review and Open Problems
The inclusion of physical information in machine learning frameworks has
revolutionized many application areas. This involves enhancing the learning
process by incorporating physical constraints and adhering to physical laws. In
this work we explore their utility for reinforcement learning applications. We
present a thorough review of the literature on incorporating physics
information, as known as physics priors, in reinforcement learning approaches,
commonly referred to as physics-informed reinforcement learning (PIRL). We
introduce a novel taxonomy with the reinforcement learning pipeline as the
backbone to classify existing works, compare and contrast them, and derive
crucial insights. Existing works are analyzed with regard to the
representation/ form of the governing physics modeled for integration, their
specific contribution to the typical reinforcement learning architecture, and
their connection to the underlying reinforcement learning pipeline stages. We
also identify core learning architectures and physics incorporation biases
(i.e., observational, inductive and learning) of existing PIRL approaches and
use them to further categorize the works for better understanding and
adaptation. By providing a comprehensive perspective on the implementation of
the physics-informed capability, the taxonomy presents a cohesive approach to
PIRL. It identifies the areas where this approach has been applied, as well as
the gaps and opportunities that exist. Additionally, the taxonomy sheds light
on unresolved issues and challenges, which can guide future research. This
nascent field holds great potential for enhancing reinforcement learning
algorithms by increasing their physical plausibility, precision, data
efficiency, and applicability in real-world scenarios
Modeling, simulation, and control of soft robots
2019 Fall.Includes bibliographical references.Soft robots are a new type of robot with deformable bodies and muscle-like actuations, which are fundamentally different from traditional robots with rigid links and motor-based actuators. Owing to their elasticity, soft robots outperform rigid ones in safety, maneuverability, and adaptability. With their advantages, many soft robots have been developed for manipulation and locomotion in recent years. However, the current state of soft robotics has significant design and development work, but lags behind in modeling and control due to the complex dynamic behavior of the soft bodies. This complexity prevents a unified dynamics model that captures the dynamic behavior, computationally-efficient algorithms to simulate the dynamics in real-time, and closed-loop control algorithms to accomplish desired dynamic responses. In this thesis, we address the three problems of modeling, simulation, and control of soft robots. For the modeling, we establish a general modeling framework for the dynamics by integrating Cosserat theory with Hamilton's principle. Such a framework can accommodate different actuation methods (e.g., pneumatic, cable-driven, artificial muscles, etc.). To simulate the proposed models, we develop efficient numerical algorithms and implement them in C++ to simulate the dynamics of soft robots in real-time. These algorithms consider qualities of the dynamics that are typically neglected (e.g., numerical damping, group structure). Using the developed numerical algorithms, we investigate the control of soft robots with the goal of achieving real-time and closed-loop control policies. Several control approaches are tested (e.g., model predictive control, reinforcement learning) for a few key tasks: reaching various points in a soft manipulator's workspace and tracking a given trajectory. The results show that model predictive control is possible but is computationally demanding, while reinforcement learning techniques are more computationally effective but require a substantial number of training samples. The modeling, simulation, and control framework developed in this thesis will lay a solid foundation to unleash the potential of soft robots for various applications, such as manipulation and locomotion
Roadmap on Machine learning in electronic structure
AbstractIn recent years, we have been witnessing a paradigm shift in computational materials science. In fact, traditional methods, mostly developed in the second half of the XXth century, are being complemented, extended, and sometimes even completely replaced by faster, simpler, and often more accurate approaches. The new approaches, that we collectively label by machine learning, have their origins in the fields of informatics and artificial intelligence, but are making rapid inroads in all other branches of science. With this in mind, this Roadmap article, consisting of multiple contributions from experts across the field, discusses the use of machine learning in materials science, and share perspectives on current and future challenges in problems as diverse as the prediction of materials properties, the construction of force-fields, the development of exchange correlation functionals for density-functional theory, the solution of the many-body problem, and more. In spite of the already numerous and exciting success stories, we are just at the beginning of a long path that will reshape materials science for the many challenges of the XXIth century
Simulation Intelligence: Towards a New Generation of Scientific Methods
The original "Seven Motifs" set forth a roadmap of essential methods for the
field of scientific computing, where a motif is an algorithmic method that
captures a pattern of computation and data movement. We present the "Nine
Motifs of Simulation Intelligence", a roadmap for the development and
integration of the essential algorithms necessary for a merger of scientific
computing, scientific simulation, and artificial intelligence. We call this
merger simulation intelligence (SI), for short. We argue the motifs of
simulation intelligence are interconnected and interdependent, much like the
components within the layers of an operating system. Using this metaphor, we
explore the nature of each layer of the simulation intelligence operating
system stack (SI-stack) and the motifs therein: (1) Multi-physics and
multi-scale modeling; (2) Surrogate modeling and emulation; (3)
Simulation-based inference; (4) Causal modeling and inference; (5) Agent-based
modeling; (6) Probabilistic programming; (7) Differentiable programming; (8)
Open-ended optimization; (9) Machine programming. We believe coordinated
efforts between motifs offers immense opportunity to accelerate scientific
discovery, from solving inverse problems in synthetic biology and climate
science, to directing nuclear energy experiments and predicting emergent
behavior in socioeconomic settings. We elaborate on each layer of the SI-stack,
detailing the state-of-art methods, presenting examples to highlight challenges
and opportunities, and advocating for specific ways to advance the motifs and
the synergies from their combinations. Advancing and integrating these
technologies can enable a robust and efficient hypothesis-simulation-analysis
type of scientific method, which we introduce with several use-cases for
human-machine teaming and automated science
- …