Using machine learning to learn from demonstration: application to the AR.Drone quadrotor control

A dissertation submitted to the Faculty of Science, University of the Witwatersrand, Johannesburg, in fulfilment of requirements for the degree of Master of Science. December 14, 2015Developing a robot that can operate autonomously is an active area in robotics research. An autonomously operating robot can have a tremendous number of applications such as: surveillance and inspection; search and rescue; and operating in hazardous environments. Reinforcement learning, a branch of machine learning, provides an attractive framework for developing robust control algorithms since it is less demanding in terms of both knowledge and programming effort. Given a reward function, reinforcement learning employs a trial-and-error concept to make an agent learn. It is computationally intractable in practice for an agent to learn “de novo”, thus it is important to provide the learning system with “a priori” knowledge. Such prior knowledge would be in the form of demonstrations performed by the teacher. However, prior knowledge does not necessarily guarantee that the agent will perform well. The performance of the agent usually depends on the reward function, since the reward function describes the formal specification of the control task. However, problems arise with complex reward function that are difficult to specify manually. In order to address these problems, apprenticeship learning via inverse reinforcement learning is used. Apprenticeship learning via inverse reinforcement learning can be used to extract a reward function from the set of demonstrations so that the agent can optimise its performance with respect to that reward function. In this research, a flight controller for the Ar.Drone quadrotor was created using a reinforcement learning algorithm and function approximators with some prior knowledge. The agent was able to perform a manoeuvre that is similar to the one demonstrated by the teacher

Compassion and Merit in Early Buddhism with the Focus on the Aṅguttara Nikāya and the Ekottarika Āgama

Of the four Nikāyas in Pali and the four Āgamas in Chinese, the numerical collections, i.e. the Aṅguttara Nikāya and the Ekottarika Āgama, are the most adaptable and considerate of individual needs according to ancient Indian/Chinese and modern American monks. Therefore, these two collections contain a considerable proportion of suttas/sūtras that are closely connected with the notion of compassion (karuṇā/anukampā). These two collections include many suttas addressed to Buddhists dealing with the ethical and spiritual concerns of life within the world, and thus involves the issues of merit (puñña). In this study I have illustrated the significant but often underestimated position of compassion with merit in early Buddhist doctrine. The soteriological function of compassion associated with merit is expounded in the early suttas/sūtras, particularly those in the Aṅguttara Nikāya and the Ekottarika Āgama. On the other hand, many discourses in these two collections reify great compassion by extending Buddhist concern from monastics to the laity, caring for all beings’ worldly welfare based on an ethical system of merit

Non-autoregressive Transformer-based End-to-end ASR using BERT

Transformer-based models have led to a significant innovation in various classic and practical subjects, including speech processing, natural language processing, and computer vision. On top of the transformer, the attention-based end-to-end automatic speech recognition (ASR) models have become a popular fashion in recent years. Specifically, the non-autoregressive modeling, which can achieve fast inference speed and comparable performance when compared to conventional autoregressive methods, is an emergent research topic. In the context of natural language processing, the bidirectional encoder representations from transformers (BERT) model has received widespread attention, partially due to its ability to infer contextualized word representations and to obtain superior performances of downstream tasks by performing only simple fine-tuning. In order to not only inherit the advantages of non-autoregressive ASR modeling, but also receive benefits from a pre-trained language model (e.g., BERT), a non-autoregressive transformer-based end-to-end ASR model based on BERT is presented in this paper. A series of experiments conducted on the AISHELL-1 dataset demonstrates competitive or superior results of the proposed model when compared to state-of-the-art ASR systems

Order-Free RNN with Visual Attention for Multi-Label Classification

In this paper, we propose the joint learning attention and recurrent neural network (RNN) models for multi-label classification. While approaches based on the use of either model exist (e.g., for the task of image captioning), training such existing network architectures typically require pre-defined label sequences. For multi-label classification, it would be desirable to have a robust inference process, so that the prediction error would not propagate and thus affect the performance. Our proposed model uniquely integrates attention and Long Short Term Memory (LSTM) models, which not only addresses the above problem but also allows one to identify visual objects of interests with varying sizes without the prior knowledge of particular label ordering. More importantly, label co-occurrence information can be jointly exploited by our LSTM model. Finally, by advancing the technique of beam search, prediction of multiple labels can be efficiently achieved by our proposed network model.Comment: Accepted at 32nd AAAI Conference on Artificial Intelligence (AAAI-18

Particle production during Inflation with a non-minimally coupled spectator scalar field

We study the inflationary model with a spectator scalar field $\chi$ coupled to both the inflaton and Ricci scalar. The interaction between the $\chi$ field and the gravity, denoted by $\xi R\chi^2$, can trigger the tachyonic instability of certain modes of the $\chi$ field. As a result, the $\chi$ field perturbations are amplified and serve as a gravitational wave (GW) source. When considering the backreaction of the $\chi$ field, an upper bound on the coupling parameter $\xi$ must be imposed to ensure that inflation does not end prematurely. In this case, we find that the inflaton's evolution experiences a sudden slowdown due to the production of $\chi$ particles, resulting in a unique oscillating structure in the power spectrum of curvature perturbations at specific scales. Moreover, the GW signal induced by the $\chi$ field is more significant than primordial GWs at around its peak scale, leading to a noticeable bump in the overall energy spectrum of GWs

Some reflections on translating the pali texts: Literary conventions, Buddhist thought, cultural background and textual history

For over a century many Buddhist texts in Pali have been translated into English, the four main Nikāyas at least twice. Significant improvements have been made in regard to English translations of Pali texts. This paper provides five case studies that illustrate the problems and complexities involved in translating Pali texts. Examples are taken from four suttas of the Aṅguttara Nikāya. Various issues are addressed using textual and contextualised analyses. I attempt to offer solutions to some problems related to translating the Pali through different approaches, including style, philology, history, Buddhist thought and inter-religious relation