Genie: A Generator of Natural Language Semantic Parsers for Virtual Assistant Commands

To understand diverse natural language commands, virtual assistants today are trained with numerous labor-intensive, manually annotated sentences. This paper presents a methodology and the Genie toolkit that can handle new compound commands with significantly less manual effort. We advocate formalizing the capability of virtual assistants with a Virtual Assistant Programming Language (VAPL) and using a neural semantic parser to translate natural language into VAPL code. Genie needs only a small realistic set of input sentences for validating the neural model. Developers write templates to synthesize data; Genie uses crowdsourced paraphrases and data augmentation, along with the synthesized data, to train a semantic parser. We also propose design principles that make VAPL languages amenable to natural language translation. We apply these principles to revise ThingTalk, the language used by the Almond virtual assistant. We use Genie to build the first semantic parser that can support compound virtual assistants commands with unquoted free-form parameters. Genie achieves a 62% accuracy on realistic user inputs. We demonstrate Genie's generality by showing a 19% and 31% improvement over the previous state of the art on a music skill, aggregate functions, and access control.Comment: To appear in PLDI 201

Prototype of a Conversational Assistant for Satellite Mission Operations

The very first artificial satellite, Sputnik, was launched in 1957 marking a new era. Concurrently, satellite mission operations emerged. These start at launch and finish at the end of mission, when the spacecraft is decommissioned. Running a satellite mission requires the monitoring and control of telemetry data, to verify and maintain satellite health, reconfigure and command the spacecraft, detect, identify and resolve anomalies and perform launch and early orbit operations. The very first chatbot, ELIZA was created in 1966, and also marked a new era of Artificial Intelligence Systems. Said systems answer users’ questions in the most diverse domains, interpreting the human language input and responding in the same manner. Nowadays, these systems are everywhere, and the list of possible applications seems endless. The goal of the present master’s dissertation is to develop a prototype of a chatbot for mission operations. For this purpose implementing a Natural Language Processing (NLP) model for satellite missions allied to a dialogue flow model. The performance of the conversational assistant is evaluated with its implementation on a mission operated by the European Space Agency (ESA), implying the generation of the spacecraft’s Database Knowledge Graph (KG). Throughout the years, many tools have been developed and added to the systems used to monitor and control spacecrafts helping Flight Control Teams (FCT) either by maintaining a comprehensive overview of the spacecraft’s status and health, speeding up failure investigation, or allowing to easily correlate time series of telemetry data. However, despite all the advances made which facilitate the daily tasks, the teams still need to navigate through thousands of parameters and events spanning years of data, using purposely built user interfaces and relying on filters and time series plots. The solution presented in this dissertation and proposed by VisionSpace Technologies focuses on improving operational efficiency whilst dealing with the mission’s complex and extensive databases.O primeiro satélite artificial, Sputnik, foi lançado em 1957 e marcou o início de uma nova era. Simultaneamente, surgiram as operações de missão de satélites. Estas iniciam com o lançamento e terminam com desmantelamento do veículo espacial, que marca o fim da missão. A operação de satélites exige o acompanhamento e controlo de dados de telemetria, com o intuito de verificar e manter a saúde do satélite, reconfigurar e comandar o veículo, detetar, identificar e resolver anomalias e realizar o lançamento e as operações iniciais do satélite. Em 1966, o primeiro Chatbot foi criado, ELIZA, e também marcou uma nova era, de sistemas dotados de Inteligência Artificial. Tais sistemas respondem a perguntas nos mais diversos domínios, para tal interpretando linguagem humana e repondendo de forma similar. Hoje em dia, é muito comum encontrar estes sistemas e a lista de aplicações possíveis parece infindável. O objetivo da presente dissertação de mestrado consiste em desenvolver o protótipo de um Chatbot para operação de satélites. Para este proposito, criando um modelo de Processamento de Linguagem Natural (NLP) aplicado a missoões de satélites aliado a um modelo de fluxo de diálogo. O desempenho do assistente conversacional será avaliado com a sua implementação numa missão operada pela Agência Espacial Europeia (ESA), o que implica a elaboração do grafico de conhecimentos associado à base de dados da missão. Ao longo dos anos, várias ferramentas foram desenvolvidas e adicionadas aos sistemas que acompanham e controlam veículos espaciais, que colaboram com as equipas de controlo de missão, mantendo uma visão abrangente sobre a condição do satélite, acelerando a investigação de falhas, ou permitindo correlacionar séries temporais de dados de telemetria. No entanto, apesar de todos os progressos que facilitam as tarefas diárias, as equipas ainda necessitam de navegar por milhares de parametros e eventos que abrangem vários anos de recolha de dados, usando interfaces para esse fim e dependendo da utilização de filtros e gráficos de series temporais. A solução apresentada nesta dissertação e proposta pela VisionSpace Technologies tem como foco melhorar a eficiência operacional lidando simultaneamente com as suas complexas e extensas bases de dados

Parallel Architectures for Planetary Exploration Requirements (PAPER)

The Parallel Architectures for Planetary Exploration Requirements (PAPER) project is essentially research oriented towards technology insertion issues for NASA's unmanned planetary probes. It was initiated to complement and augment the long-term efforts for space exploration with particular reference to NASA/LaRC's (NASA Langley Research Center) research needs for planetary exploration missions of the mid and late 1990s. The requirements for space missions as given in the somewhat dated Advanced Information Processing Systems (AIPS) requirements document are contrasted with the new requirements from JPL/Caltech involving sensor data capture and scene analysis. It is shown that more stringent requirements have arisen as a result of technological advancements. Two possible architectures, the AIPS Proof of Concept (POC) configuration and the MAX Fault-tolerant dataflow multiprocessor, were evaluated. The main observation was that the AIPS design is biased towards fault tolerance and may not be an ideal architecture for planetary and deep space probes due to high cost and complexity. The MAX concepts appears to be a promising candidate, except that more detailed information is required. The feasibility for adding neural computation capability to this architecture needs to be studied. Key impact issues for architectural design of computing systems meant for planetary missions were also identified

Voicing Kinship with Machines: Diffractive Empathetic Listening to Synthetic Voices in Performance.

This thesis contributes to the field of voice studies by analyzing the design and production of synthetic voices in performance. The work explores six case studies, consisting of different performative experiences of the last decade (2010- 2020) that featured synthetic voice design. It focusses on the political and social impact of synthetic voices, starting from yet challenging the concepts of voice in the machine and voice of the machine. The synthetic voices explored are often playing the role of simulated artificial intelligences, therefore this thesis expands its questions towards technology at large. The analysis of the case studies follows new materialist and posthumanist premises, yet it tries to confute the patriarchal and neoliberal approach towards technological development through feminist and de-colonial approaches, developing a taxonomy for synthetic voices in performance. Chapter 1 introduces terms and explains the taxonomy. Chapter 2 looks at familiar representations of fictional AI. Chapter 3 introduces headphone theatre exploring immersive practices. Chapters 4 and 5 engage with chatbots. Chapter 6 goes in depth exploring Human and Artificial Intelligence interaction, whereas chapter 7 moves slightly towards music production and live art. The body of the thesis includes the work of Pipeline Theatre, Rimini Protokoll, Annie Dorsen, Begüm Erciyas, and Holly Herndon. The analysis is informed by posthumanism, feminism, and performance studies, starting from my own practice as sound designer and singer, looking at aesthetics of reproduction, audience engagement, and voice composition. This thesis has been designed to inspire and provoke practitioners and scholars to explore synthetic voices further, question predominant biases of binarism and acknowledge their importance in redefining technology