Improving Search through A3C Reinforcement Learning based Conversational Agent

We develop a reinforcement learning based search assistant which can assist users through a set of actions and sequence of interactions to enable them realize their intent. Our approach caters to subjective search where the user is seeking digital assets such as images which is fundamentally different from the tasks which have objective and limited search modalities. Labeled conversational data is generally not available in such search tasks and training the agent through human interactions can be time consuming. We propose a stochastic virtual user which impersonates a real user and can be used to sample user behavior efficiently to train the agent which accelerates the bootstrapping of the agent. We develop A3C algorithm based context preserving architecture which enables the agent to provide contextual assistance to the user. We compare the A3C agent with Q-learning and evaluate its performance on average rewards and state values it obtains with the virtual user in validation episodes. Our experiments show that the agent learns to achieve higher rewards and better states.Comment: 17 pages, 7 figure

Interactive Machine Comprehension with Information Seeking Agents

Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA). We argue that this stems from the nature of MRC datasets: most of these are static environments wherein the supporting documents and all necessary information are fully observed. In this paper, we propose a simple method that reframes existing MRC datasets as interactive, partially observable environments. Specifically, we "occlude" the majority of a document's text and add context-sensitive commands that reveal "glimpses" of the hidden text to a model. We repurpose SQuAD and NewsQA as an initial case study, and then show how the interactive corpora can be used to train a model that seeks relevant information through sequential decision making. We believe that this setting can contribute in scaling models to web-level QA scenarios.Comment: ACL202

Interactive Language Learning by Question Answering

Humans observe and interact with the world to acquire knowledge. However, most existing machine reading comprehension (MRC) tasks miss the interactive, information-seeking component of comprehension. Such tasks present models with static documents that contain all necessary information, usually concentrated in a single short substring. Thus, models can achieve strong performance through simple word- and phrase-based pattern matching. We address this problem by formulating a novel text-based question answering task: Question Answering with Interactive Text (QAit). In QAit, an agent must interact with a partially observable text-based environment to gather information required to answer questions. QAit poses questions about the existence, location, and attributes of objects found in the environment. The data is built using a text-based game generator that defines the underlying dynamics of interaction with the environment. We propose and evaluate a set of baseline models for the QAit task that includes deep reinforcement learning agents. Experiments show that the task presents a major challenge for machine reading systems, while humans solve it with relative ease.Comment: EMNLP 201

Tracking brain dynamics across transitions of consciousness

How do we lose and regain consciousness? The space between healthy wakefulness and unconsciousness encompasses a series of gradual and rapid changes in brain activity. In this thesis, I investigate computational measures applicable to the electroencephalogram to quantify the loss and recovery of consciousness from the perspective of modern theoretical frameworks. I examine three different transitions of consciousness caused by natural, pharmacological and pathological factors: sleep, sedation and coma. First, I investigate the neural dynamics of falling asleep. By combining the established methods of phase-lag brain connectivity and EEG microstates in a group of healthy subjects, a unique microstate is identified, whose increased duration predicts behavioural unresponsiveness to auditory stimuli during drowsiness. This microstate also uniquely captures an increase in frontoparietal theta connectivity, a putative marker of the loss of consciousness prior to sleep onset. I next examine the loss of behavioural responsiveness in healthy subjects undergoing mild and moderate sedation. The Lempel-Ziv compression algorithm is employed to compute signal complexity and symbolic mutual information to assess information integration. An intriguing dissociation between responsiveness and drug level in blood during sedation is revealed: responsiveness is best predicted by the temporal complexity of the signal at single- channel and low-frequency integration, whereas drug level is best predicted by the complexity of spatial patterns and high-frequency integration. Finally, I investigate brain connectivity in the overnight EEG recordings of a group of patients in acute coma. Graph theory is applied on alpha, theta and delta networks to find that increased variability in delta network integration early after injury predicts the eventual coma recovery score. A case study is also described where the re-emergence of frontoparietal connectivity predicted a full recovery long before behavioural improvement. The findings of this thesis inform prospective clinical applications for tracking states of consciousness and advance our understanding of the slow and fast brain dynamics underlying its transitions. Collating these findings under a common theoretical framework, I argue that the diversity of dynamical states, in particular in temporal domain, and information integration across brain networks are fundamental in sustaining consciousness.My PhD was funded by the Cambridge Trust and a MariaMarina award from Lucy Cavendish College