Visual Imitation Learning with Recurrent Siamese Networks

It would be desirable for a reinforcement learning (RL) based agent to learn behaviour by merely watching a demonstration. However, defining rewards that facilitate this goal within the RL paradigm remains a challenge. Here we address this problem with Siamese networks, trained to compute distances between observed behaviours and the agent's behaviours. Given a desired motion such Siamese networks can be used to provide a reward signal to an RL agent via the distance between the desired motion and the agent's motion. We experiment with an RNN-based comparator model that can compute distances in space and time between motion clips while training an RL policy to minimize this distance. Through experimentation, we have had also found that the inclusion of multi-task data and an additional image encoding loss helps enforce the temporal consistency. These two components appear to balance reward for matching a specific instance of behaviour versus that behaviour in general. Furthermore, we focus here on a particularly challenging form of this problem where only a single demonstration is provided for a given task -- the one-shot learning setting. We demonstrate our approach on humanoid agents in both 2D with $10$ degrees of freedom (DoF) and 3D with $38$ DoF.Comment: PrePrin

Applied Deep Learning: Automated segmentation of White Matter Hyperintensities (WMH) on brain MR images

Small vessel disease plays a crucial role in stroke, dementia, and ageing. White matter hyperintensities (WMH) of vascular origin are one of the main consequences of small vessel disease and well visible on brain MR images. Quantification of WMH volume, location, and shape is of key importance in clinical research studies and likely to find its way into clinical practice; supporting diagnosis, prognosis, and monitoring of treatment for dementia and other neurodegenerative diseases. It has been noted that visual rating of WMH has important limitations and hence a more detailed segmentation of WMH is preferred. Various automated WMH segmentation techniques have been developed, to provide quantitative measurements and replace time-consuming, observer-dependent delineation procedures. NLP LOGIX developed an automated algorithm for automatically segmenting white matter hyperintensities using an advanced modeling technique called deep learning

Optimizing Simulated Crowd Behaviour

In the context of crowd simulation, there is a diverse set of algorithms that model steering, the ability of an agent to navigate between spatial locations, while avoiding static and dynamic obstacles. The performance of steering approaches, both in terms of quality of results and computational efficiency, depends on internal parameters that are manually tuned to satisfy application-specific requirements. This work investigates the effect that these parameters have on an algorithm's performance. Using three representative steering algorithms and a set of established performance criteria, we perform a number of large scale optimization experiments that optimize an algorithm's parameters for a range of objectives. For example, our method automatically finds optimal parameters to minimize turbulence at bottlenecks, reduce building evacuation times, produce emergent patterns, and increase the computational efficiency of an algorithm. Our study includes a statistical analysis of the correlations between algorithmic parameters, and performance criteria. We also propose using the Pareto Optimal Front as an efficient way of modelling optimal relationships between multiple objectives, and demonstrate its effectiveness by estimating optimal parameters for interactively defined combinations of the associated objectives. The proposed methodologies are general and can be applied to any steering algorithm using any set of performance criteria

Attitudes of Participants Toward the Brookings Middle School Interscholastic Athletic Program

The purpose of this study was to determine the attitudes of participants toward their experience of the interscholastic athletic program at the Brookings Middle School, in Brookings, South Dakota. This involved a comparative analysis of reactions to questionnaire items to determine the influence of sex and grade level on attitudes toward the athletic program. The questionnaire was designed to investigate attitudes in three areas: 1 the initial motivations which led the student to participate in athletics; 2 ways in which the actual athletic experience coincided or contrasted with the student’s expectations; and 3 the place of athletics in each student’s value system. In general, members of both sexes and grade levels reacted positively to statements which were supportive of the program, and negatively to those statements that were critical of the program. A high percentage of students (approximately 70%) indicated that their parents encouraged them to participate in athletics, and that they believed that their parents were proud of them because they were in athletics. Peer group pressure and popularity were found to have little influence on the level of athletic participation. Female and seventh grade students did not feel that their experience or their performance in athletic paralleled their pre-season expectations to the degree that it did for the males and eighth grade students. In the eyes of the students, participating in athletics enabled them to feel better about themselves. A large percentage of students believe that athletics was an important part of their educational experience and would continue to have high priority throughout their adult life

Torque-based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfer

In this paper, we review the question of which action space is best suited for controlling a real biped robot in combination with Sim2Real training. Position control has been popular as it has been shown to be more sample efficient and intuitive to combine with other planning algorithms. However, for position control gain tuning is required to achieve the best possible policy performance. We show that instead, using a torque-based action space enables task-and-robot agnostic learning with less parameter tuning and mitigates the sim-to-reality gap by taking advantage of torque control's inherent compliance. Also, we accelerate the torque-based-policy training process by pre-training the policy to remain upright by compensating for gravity. The paper showcases the first successful sim-to-real transfer of a torque-based deep reinforcement learning policy on a real human-sized biped robot. The video is available at https://youtu.be/CR6pTS39VRE