Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge

This paper provides a comprehensive analysis of the first shared task on End-to-End Natural Language Generation (NLG) and identifies avenues for future research based on the results. This shared task aimed to assess whether recent end-to-end NLG systems can generate more complex output by learning from datasets containing higher lexical richness, syntactic complexity and diverse discourse phenomena. Introducing novel automatic and human metrics, we compare 62 systems submitted by 17 institutions, covering a wide range of approaches, including machine learning architectures -- with the majority implementing sequence-to-sequence models (seq2seq) -- as well as systems based on grammatical rules and templates. Seq2seq-based systems have demonstrated a great potential for NLG in the challenge. We find that seq2seq systems generally score high in terms of word-overlap metrics and human evaluations of naturalness -- with the winning SLUG system (Juraska et al., 2018) being seq2seq-based. However, vanilla seq2seq models often fail to correctly express a given meaning representation if they lack a strong semantic control mechanism applied during decoding. Moreover, seq2seq models can be outperformed by hand-engineered systems in terms of overall quality, as well as complexity, length and diversity of outputs. This research has influenced, inspired and motivated a number of recent studies outwith the original competition, which we also summarise as part of this paper.Comment: Computer Speech and Language, final accepted manuscript (in press

Scalable and Quality-Aware Training Data Acquisition for Conversational Cognitive Services

Dialog Systems (or simply bots) have recently become a popular human-computer interface for performing user's tasks, by invoking the appropriate back-end APIs (Application Programming Interfaces) based on the user's request in natural language. Building task-oriented bots, which aim at performing real-world tasks (e.g., booking flights), has become feasible with the continuous advances in Natural Language Processing (NLP), Artificial Intelligence (AI), and the countless number of devices which allow third-party software systems to invoke their back-end APIs. Nonetheless, bot development technologies are still in their preliminary stages, with several unsolved theoretical and technical challenges stemming from the ambiguous nature of human languages. Given the richness of natural language, supervised models require a large number of user utterances paired with their corresponding tasks -- called intents. To build a bot, developers need to manually translate APIs to utterances (called canonical utterances) and paraphrase them to obtain a diverse set of utterances. Crowdsourcing has been widely used to obtain such datasets, by paraphrasing the initial utterances generated by the bot developers for each task. However, there are several unsolved issues. First, generating canonical utterances requires manual efforts, making bot development both expensive and hard to scale. Second, since crowd workers may be anonymous and are asked to provide open-ended text (paraphrases), crowdsourced paraphrases may be noisy and incorrect (not conveying the same intent as the given task). This thesis first surveys the state-of-the-art approaches for collecting large training utterances for task-oriented bots. Next, we conduct an empirical study to identify quality issues of crowdsourced utterances (e.g., grammatical errors, semantic completeness). Moreover, we propose novel approaches for identifying unqualified crowd workers and eliminating malicious workers from crowdsourcing tasks. Particularly, we propose a novel technique to promote the diversity of crowdsourced paraphrases by dynamically generating word suggestions while crowd workers are paraphrasing a particular utterance. Moreover, we propose a novel technique to automatically translate APIs to canonical utterances. Finally, we present our platform to automatically generate bots out of API specifications. We also conduct thorough experiments to validate the proposed techniques and models

Low-Resource Unsupervised NMT:Diagnosing the Problem and Providing a Linguistically Motivated Solution

Unsupervised Machine Translation hasbeen advancing our ability to translatewithout parallel data, but state-of-the-artmethods assume an abundance of mono-lingual data. This paper investigates thescenario where monolingual data is lim-ited as well, finding that current unsuper-vised methods suffer in performance un-der this stricter setting. We find that theperformance loss originates from the poorquality of the pretrained monolingual em-beddings, and we propose using linguis-tic information in the embedding train-ing scheme. To support this, we look attwo linguistic features that may help im-prove alignment quality: dependency in-formation and sub-word information. Us-ing dependency-based embeddings resultsin a complementary word representationwhich offers a boost in performance ofaround 1.5 BLEU points compared to stan-dardWORD2VECwhen monolingual datais limited to 1 million sentences per lan-guage. We also find that the inclusion ofsub-word information is crucial to improv-ing the quality of the embedding

Proceedings of the 19th Sound and Music Computing Conference

Proceedings of the 19th Sound and Music Computing Conference - June 5-12, 2022 - Saint-Étienne (France). https://smc22.grame.f

Critical Programming: Toward a Philosophy of Computing

Beliefs about the relationship between human beings and computing machines and their destinies have alternated from heroic counterparts to conspirators of automated genocide, from apocalyptic extinction events to evolutionary cyborg convergences. Many fear that people are losing key intellectual and social abilities as tasks are offloaded to the everywhere of the built environment, which is developing a mind of its own. If digital technologies have contributed to forming a dumbest generation and ushering in a robotic moment, we all have a stake in addressing this collective intelligence problem. While digital humanities continue to flourish and introduce new uses for computer technologies, the basic modes of philosophical inquiry remain in the grip of print media, and default philosophies of computing prevail, or experimental ones propagate false hopes. I cast this as-is situation as the post-postmodern network dividual cyborg, recognizing that the rational enlightenment of modernism and regressive subjectivity of postmodernism now operate in an empire of extended mind cybernetics combined with techno-capitalist networks forming societies of control. Recent critical theorists identify a justificatory scheme foregrounding participation in projects, valorizing social network linkages over heroic individualism, and commending flexibility and adaptability through life long learning over stable career paths. It seems to reify one possible, contingent configuration of global capitalism as if it was the reflection of a deterministic evolution of commingled technogenesis and synaptogenesis. To counter this trend I offer a theoretical framework to focus on the phenomenology of software and code, joining social critiques with textuality and media studies, the former proposing that theory be done through practice, and the latter seeking to understand their schematism of perceptibility by taking into account engineering techniques like time axis manipulation. The social construction of technology makes additional theoretical contributions dispelling closed world, deterministic historical narratives and requiring voices be given to the engineers and technologists that best know their subject area. This theoretical slate has been recently deployed to produce rich histories of computing, networking, and software, inform the nascent disciplines of software studies and code studies, as well as guide ethnographers of software development communities. I call my syncretism of these approaches the procedural rhetoric of diachrony in synchrony, recognizing that multiple explanatory layers operating in their individual temporal and physical orders of magnitude simultaneously undergird post-postmodern network phenomena. Its touchstone is that the human-machine situation is best contemplated by doing, which as a methodology for digital humanities research I call critical programming. Philosophers of computing explore working code places by designing, coding, and executing complex software projects as an integral part of their intellectual activity, reflecting on how developing theoretical understanding necessitates iterative development of code as it does other texts, and how resolving coding dilemmas may clarify or modify provisional theories as our minds struggle to intuit the alien temporalities of machine processes

Autopoietic-extended architecture: can buildings think?

To incorporate bioremedial functions into the performance of buildings and to balance generative architecture's dominant focus on computational programming and digital fabrication, this thesis first hybridizes theories of autopoiesis into extended cognition in order to research biological domains that include synthetic biology and biocomputation. Under the rubric of living technology I survey multidisciplinary fields to gather perspective for student design of bioremedial and/or metabolic components in generative architecture where generative not only denotes the use of computation but also includes biochemical, biomechanical, and metabolic functions. I trace computation and digital simulations back to Alan Turing's early 1950s Morphogenetic drawings, reaction-diffusion algorithms, and pioneering artificial intelligence (AI) in order to establish generative architecture's point of origin. I ask provocatively: Can buildings think? as a question echoing Turing's own "Can machines think?" Thereafter, I anticipate not only future bioperformative materials but also theories capable of underpinning strains of metabolic intelligences made possible via AI, synthetic biology, and living technology. I do not imply that metabolic architectural intelligence will be like human cognition. I suggest, rather, that new research and pedagogies involving the intelligence of bacteria, plants, synthetic biology, and algorithms define approaches that generative architecture should take in order to source new forms of autonomous life that will be deployable as corrective environmental interfaces. I call the research protocol autopoietic-extended design, theorizing it as an operating system (OS), a research methodology, and an app schematic for design studios and distance learning that makes use of in-field, e-, and m-learning technologies. A quest of this complexity requires scaffolding for coordinating theory-driven teaching with practice-oriented learning. Accordingly, I fuse Maturana and Varela's biological autopoiesis and its definitions of minimal biological life with Andy Clark's hypothesis of extended cognition and its cognition-to-environment linkages. I articulate a generative design strategy and student research method explained via architectural history interpreted from Louis Sullivan's 1924 pedagogical drawing system, Le Corbusier's Modernist pronouncements, and Greg Lynn's Animate Form. Thus, autopoietic-extended design organizes thinking about the generation of ideas for design prior to computational production and fabrication, necessitating a fresh relationship between nature/science/technology and design cognition. To systematize such a program requires the avoidance of simple binaries (mind/body, mind/nature) as well as the stationing of tool making, technology, and architecture within the ream of nature. Hence, I argue, in relation to extended phenotypes, plant-neurobiology, and recent genetic research: Consequently, autopoietic-extended design advances design protocols grounded in morphology, anatomy, cognition, biology, and technology in order to appropriate metabolic and intelligent properties for sensory/response duty in buildings. At m-learning levels smartphones, social media, and design apps source data from nature for students to mediate on-site research by extending 3D pedagogical reach into new university design programs. I intend the creation of a dialectical investigation of animal/human architecture and computational history augmented by theory relevant to current algorithmic design and fablab production. The autopoietic-extended design dialectic sets out ways to articulate opposition/differences outside the Cartesian either/or philosophy in order to prototype metabolic architecture, while dialectically maintaining: Buildings can think