Versatile Inverse Reinforcement Learning via Cumulative Rewards

Inverse Reinforcement Learning infers a reward function from expert demonstrations, aiming to encode the behavior and intentions of the expert. Current approaches usually do this with generative and uni-modal models, meaning that they encode a single behavior. In the common setting, where there are various solutions to a problem and the experts show versatile behavior this severely limits the generalization capabilities of these methods. We propose a novel method for Inverse Reinforcement Learning that overcomes these problems by formulating the recovered reward as a sum of iteratively trained discriminators. We show on simulated tasks that our approach is able to recover general, high-quality reward functions and produces policies of the same quality as behavioral cloning approaches designed for versatile behavior

Decentralized Network Based Mobility Management: Framework, System Design and Evaluation

wird nachgereich

BUSINESS INTELLIGENCE FOR BUSINESS PROCESSES: THE CASE OF IT INCIDENT MANAGEMENT

IT service desks have become an integral part of intra-enterprise ecosystems, keeping IT hardware and software services within the company running. Business Intelligence methods have an enormous potential to support IT helpdesk employees by making implicit knowledge explicit, accelerating business processes throughout the entire company, and retaining the knowledge of experienced employees upon retirement. In this paper, we investigate these benefits by showing how analytics can automate the assignment of helpdesk tasks, enable early warning mechanisms for accumulated incidents, and enhance knowledge sharing among helpdesk users. For this purpose, we use a combination of topic modeling and predictive analytics, which is applied to an extensive dataset of support tickets from a global automotive supplier. Our approach identifies relevant topics and assigns these to helpdesk tickets, thereby decoding implicit knowledge into formal rules and business processes

Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors

Humans intuitively solve tasks in versatile ways, varying their behavior in terms of trajectory-based planning and for individual steps. Thus, they can easily generalize and adapt to new and changing environments. Current Imitation Learning algorithms often only consider unimodal expert demonstrations and act in a state-action-based setting, making it difficult for them to imitate human behavior in case of versatile demonstrations. Instead, we combine a mixture of movement primitives with a distribution matching objective to learn versatile behaviors that match the expert's behavior and versatility. To facilitate generalization to novel task configurations, we do not directly match the agent's and expert's trajectory distributions but rather work with concise geometric descriptors which generalize well to unseen task configurations. We empirically validate our method on various robot tasks using versatile human demonstrations and compare to imitation learning algorithms in a state-action setting as well as a trajectory-based setting. We find that the geometric descriptors greatly help in generalizing to new task configurations and that combining them with our distribution-matching objective is crucial for representing and reproducing versatile behavior.Comment: Accepted as a poster at the 6th Conference on Robot Learning (CoRL), 202

Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors

Humans intuitively solve tasks in versatile ways, varying their behavior in terms of trajectory-based planning and for individual steps. Thus, they can easily generalize and adapt to new and changing environments. Current Imitation Learning algorithms often only consider unimodal expert demonstrations and act in a state-action-based setting, making it difficult for them to imitate human behavior in case of versatile demonstrations. Instead, we combine a mixture of movement primitives with a distribution matching objective to learn versatile behaviors that match the expert's behavior and versatility. To facilitate generalization to novel task configurations, we do not directly match the agent's and expert's trajectory distributions but rather work with concise geometric descriptors which generalize well to unseen task configurations. We empirically validate our method on various robot tasks using versatile human demonstrations and compare to imitation learning algorithms in a state-action setting as well as a trajectory-based setting. We find that the geometric descriptors greatly help in generalizing to new task configurations and that combining them with our distribution-matching objective is crucial for representing and reproducing versatile behavior.Comment: Accepted as a poster at the 6th Conference on Robot Learning (CoRL), 202

Anticipating Injuries and Health Problems in Elite Soccer Players Using Dynamic Complexity

BACKGROUND/AIM: Injuries and health problems of soccer players may appear abruptly and are often unexpected. However, hypotheses from complex systems theory suggest that these events can be preceded by certain Early Warning Signals (EWSs).1 We tested whether injuries and health problems can be anticipated with a specific type of EWS, that is, an increase in dynamic complexity (DC).2METHODS:Over two competitive seasons, we collected psychological and physiological self-reports (i.e., self-efficacy, motivation, mood, rating of own performance, enjoyment, and recovery) and data from heart rate sensors on every training and match day from 14 youth soccer players. We recorded time-loss injuries daily and players filled in the Sports Trauma Research Center Questionnaire on Health Problems (OSTRC-H2) once a week. We then calculated the DC of the self-reports and sensor data in a seven-day window to test for increased variability and complexity over time before injuries and health problems.RESULTS:Players experienced 5.6 injuries and 8.4 health problems on average across two seasons (range=1-18 and range=2-26, respectively). Results showed that increases in DC could often anticipate the occurrence of injuries and health problems. In 55% and 37% of the players DC increased up to five days before injuries and health problems, respectively (SD=39% and SD=25%, Min=0% and Min=0%, Max=100% and Max=83%).CONCLUSIONS:Results of this study suggest that EWSs can be used for real-time anticipation of injuries and health problems in daily soccer practice. Future research should test for the robustness of these results within and between individuals and perform sensitivity and specificity tests. In addition, finding out how warning signals can be communicated to soccer players and staff is an interesting avenue.REFERENCES1. Den Hartigh RJR, Meerhoff LRA, Van Yperen NW, et al. Resilience in Sports: A Multidisciplinary, Dynamic, and Personalized Perspective. Int Rev Sport Exerc Psychol. 2022. doi:https://doi.org/10.1080/1750984X.2022.20397492. Olthof M, Hasselman F, Strunk G, et al. Critical Fluctuations as an Early-Warning Signal for Sudden Gains and Losses in Patients Receiving Psychotherapy for Mood Disorders. Clin Psychol Sci. 2020;8(1):25-35. doi:10.1177/2167702619865969<br/