Deep R-Learning for Continual Area Sweeping

This publication is by UT affiliates that was featured in the October Good Systems Network Digest in 2020.Office of the VP for Researc

Deep R learning for continual area sweeping

In order to maintain robustness, autonomous robots need to constantly update their knowledge of the environment, which can be expensive when they are deployed in large, dynamic spaces. The continual area sweeping task formalizes the problem of a robot continually patrolling an area in a non-uniform way in order to efficiently use travel time. However, the existing problem formulation makes strong assumptions about the environment, and to date only a sub-optimal greedy approach has been proposed. We generalize the continual area sweeping formulation to include fewer environmental constraints, and propose a novel reinforcement learning approach. We evaluate our approach in an abstract simulation and in a high fidelity Gazebo simulation, which shows significant improvement upon the initial approach in general settingsComputational Science, Engineering, and Mathematic

Temporal-Logic-Based Reward Shaping for Continuing Learning Tasks

In continuing tasks, average-reward reinforcement learning may be a more appropriate problem formulation than the more common discounted reward formulation. As usual, learning an optimal policy in this setting typically requires a large amount of training experiences. Reward shaping is a common approach for incorporating domain knowledge into reinforcement learning in order to speed up convergence to an optimal policy. However, to the best of our knowledge, the theoretical properties of reward shaping have thus far only been established in the discounted setting. This paper presents the first reward shaping framework for average-reward learning and proves that, under standard assumptions, the optimal policy under the original reward function can be recovered. In order to avoid the need for manual construction of the shaping function, we introduce a method for utilizing domain knowledge expressed as a temporal logic formula. The formula is automatically translated to a shaping function that provides additional reward throughout the learning process. We evaluate the proposed method on three continuing tasks. In all cases, shaping speeds up the average-reward learning rate without any reduction in the performance of the learned policy compared to relevant baselines

Defining a relevant architecture in South Africa

Architecture in South Africa is at a crossroads. Afteryears of repression and isolation during which contemporary architecture lost its way, there is now a desperate need for architects to respond to the social a nd cultural challenges of a society riven by massive material contrasts. Within architecture schools, a student body more representative of society than hitherto is engaged in projects which reflect the very diverse needs of the community. Central to the effectiveness of such teaching programmes is the presence of teachers fully engaged in practice, creating a responsible architecture fora renewed nation

If deep learning is the answer, then what is the question?

Neuroscience research is undergoing a minor revolution. Recent advances in machine learning and artificial intelligence (AI) research have opened up new ways of thinking about neural computation. Many researchers are excited by the possibility that deep neural networks may offer theories of perception, cognition and action for biological brains. This perspective has the potential to radically reshape our approach to understanding neural systems, because the computations performed by deep networks are learned from experience, not endowed by the researcher. If so, how can neuroscientists use deep networks to model and understand biological brains? What is the outlook for neuroscientists who seek to characterise computations or neural codes, or who wish to understand perception, attention, memory, and executive functions? In this Perspective, our goal is to offer a roadmap for systems neuroscience research in the age of deep learning. We discuss the conceptual and methodological challenges of comparing behaviour, learning dynamics, and neural representation in artificial and biological systems. We highlight new research questions that have emerged for neuroscience as a direct consequence of recent advances in machine learning.Comment: 4 Figures, 17 Page