Deep Lagrangian Networks for end-to-end learning of energy-based control for under-actuated systems

Applying Deep Learning to control has a lot of potential for enabling the intelligent design of robot control laws. Unfortunately common deep learning approaches to control, such as deep reinforcement learning, require an unrealistic amount of interaction with the real system, do not yield any performance guarantees, and do not make good use of extensive insights from model-based control. In particular, common black-box approaches -- that abandon all insight from control -- are not suitable for complex robot systems. We propose a deep control approach as a bridge between the solid theoretical foundations of energy-based control and the flexibility of deep learning. To accomplish this goal, we extend Deep Lagrangian Networks (DeLaN) to not only adhere to Lagrangian Mechanics but also ensure conservation of energy and passivity of the learned representation. This novel extension is embedded within generic model-based control laws to enable energy control of under-actuated systems. The resulting DeLaN for energy control (DeLaN 4EC) is the first model learning approach using generic function approximation that is capable of learning energy control. DeLaN 4EC exhibits excellent real-time control on the physical Furuta Pendulum and learns to swing-up the pendulum while the control law using system identification does not.Comment: Published at IROS 201

Data-driven discovery of interpretable Lagrangian of stochastically excited dynamical systems

Exploring the intersection of deterministic and stochastic dynamics, this paper delves into Lagrangian discovery for conservative and non-conservative systems under stochastic excitation. Traditional Lagrangian frameworks, adept at capturing deterministic behavior, are extended to incorporate stochastic excitation. The study critically evaluates recent computational methodologies for learning Lagrangians from observed data, highlighting the limitations in interpretability and the exclusion of stochastic excitation. To address these gaps, an automated data-driven framework is proposed for the simultaneous yet uncoupled discovery of Lagrange densities and the volatility function of stochastic excitation by leveraging the sparse regression. This novel framework offers several advantages over existing approaches. Firstly, it provides an interpretable description of the underlying Lagrange density, allowing for a deeper understanding of system dynamics under stochastic excitations. Secondly, it identifies the interpretable form of the generalized stochastic force, addressing the limitations of existing deterministic approaches. Additionally, the framework demonstrates robustness and versatility through numerical case studies encompassing both stochastic differential equations (SDEs) and stochastic partial differential equations (SPDEs), with results showing almost exact approximations to true system behavior and minimal relative error in derived equations of motion

On Optimal Behavior Under Uncertainty in Humans and Robots

Despite significant progress in robotics and automation in the recent decades, there still remains a noticeable gap in performance compared to humans. Although the computation capabilities are growing every year, and are even projected to exceed the capacities of biological systems, the behaviors generated using current computational paradigms are arguably not catching up with the available resources. Why is that? It appears that we are still lacking some fundamental understanding of how living organisms are making decisions, and therefore we are unable to replicate intelligent behavior in artificial systems. Therefore, in this thesis, we attempted to develop a framework for modeling human and robot behavior based on statistical decision theory. Different features of this approach, such as risk-sensitivity, exploration, learning, control, were investigated in a number of publications. First, we considered the problem of learning new skills and developed a framework of entropic regularization of Markov decision processes (MDP). Utilizing a generalized concept of entropy, we were able to realize the trade-off between exploration and exploitation via a choice of a single scalar parameter determining the divergence function. Second, building on the theory of partially observable Markov decision process (POMDP), we proposed and validated a model of human ball catching behavior. Crucially, information seeking behavior was identified as a key feature enabling the modeling of observed human catches. Thus, entropy reduction was seen to play an important role in skillful human behavior. Third, having extracted the modeling principles from human behavior and having developed an information-theoretic framework for reinforcement learning, we studied the real-robot applications of the learning-based controllers in tactile-rich manipulation tasks. We investigated vision-based tactile sensors and the capability of learning algorithms to autonomously extract task-relevant features for manipulation tasks. The specific feature of tactile-based control that perception and action are tightly connected at the point of contact, enabled us to gather insights into the strengths and limitations of the statistical learning approach to real-time robotic manipulation. In conclusion, this thesis presents a series of investigations into the applicability of the statistical decision theory paradigm to modeling the behavior of humans and for synthesizing the behavior of robots. We conclude that a number of important features related to information processing can be represented and utilized in artificial systems for generating more intelligent behaviors. Nevertheless, these are only the first steps and we acknowledge that the road towards artificial general intelligence and skillful robotic applications will require more innovations and potentially transcendence of the probabilistic modeling paradigm