On Building Generalizable Learning Agents

It has been a long-standing goal in Artificial Intelligence (AI) to build machines that can solve tasks that humans can. Thanks to the recent rapid progress in data-driven methods, which train agents to solve tasks by learning from massive training data, there have been many successes in applying such learning approaches to handle and even solve a number of extremely challenging tasks, including image classification, language generation, robotics control, and several multi-player games. The key factor for all these data-driven successes is that the trained agents can generalize to test scenarios that are unseen during training. This generalization capability is the foundation for building any practical AI system. This thesis studies generalization, the fundamental challenge in AI, and proposes solutions to improve the generalization performances of learning agents in a variety of problems. We start by providing a formal formulation of the generalization problem in the context of reinforcement learning and proposing 4 principles within this formulation to guide the design of training techniques for improved generalization. We validate the effectiveness of our proposed principles by considering 4 different domains, from simple to complex, and developing domain-specific techniques following these principles. Particularly, we begin with the simplest domain, i.e., path-finding on graphs (Part I), and then consider visual navigation in a 3D world (Part II) and competition in complex multi-agent games (Part III), and lastly tackle some natural language processing tasks (Part IV). Empirical evidences demonstrate that the proposed principles can generally lead to much improved generalization performances in a wide range of problems

Spectators’ aesthetic experiences of sound and movement in dance performance

In this paper we present a study of spectators’ aesthetic experiences of sound and movement in live dance performance. A multidisciplinary team comprising a choreographer, neuroscientists and qualitative researchers investigated the effects of different sound scores on dance spectators. What would be the impact of auditory stimulation on kinesthetic experience and/or aesthetic appreciation of the dance? What would be the effect of removing music altogether, so that spectators watched dance while hearing only the performers’ breathing and footfalls? We investigated audience experience through qualitative research, using post-performance focus groups, while a separately conducted functional brain imaging (fMRI) study measured the synchrony in brain activity across spectators when they watched dance with sound or breathing only. When audiences watched dance accompanied by music the fMRI data revealed evidence of greater intersubject synchronisation in a brain region consistent with complex auditory processing. The audience research found that some spectators derived pleasure from finding convergences between two complex stimuli (dance and music). The removal of music and the resulting audibility of the performers’ breathing had a significant impact on spectators’ aesthetic experience. The fMRI analysis showed increased synchronisation among observers, suggesting greater influence of the body when interpreting the dance stimuli. The audience research found evidence of similar corporeally focused experience. The paper discusses possible connections between the findings of our different approaches, and considers the implications of this study for interdisciplinary research collaborations between arts and sciences

Slisp: A Flexible Software Toolkit for Hybrid, Embedded and Distributed Applications

We describe Slisp (pronounced ‘Ess-Lisp’), a hybrid Lisp–C programming toolkit for the development of scriptable and distributed applications. Computationally expensive operations implemented as separate C-coded modules are selectively compiled into a small Xlisp interpreter, then called as Lisp functions in a Lisp-coded program. The resulting hybrid program may run in several modes: as a stand-alone executable, embedded in a different C program, as a networked server accessed from another Slisp client, or as a networked server accessed from a C-coded client. Five years of experience with Slisp, as well experience with other scripting languages such as Tcl and Perl, are summarized. These experiences suggest that Slisp will be most useful for mid-sized applications in which the kinds of scripting and embeddability features provided by Tcl and Perl can be extended in an efficient manner to larger applications, while maintaining a well-defined standard (Common Lisp) for these extensions. In addition, the generality of Lisp makes Lisp a good candidate for an application-level communication language in distributed environments

Directional adposition use in English, Swedish and Finnish

Directional adpositions such as to the left of describe where a Figure is in relation to a Ground. English and Swedish directional adpositions refer to the location of a Figure in relation to a Ground, whether both are static or in motion. In contrast, the Finnish directional adpositions edellä (in front of) and jäljessä (behind) solely describe the location of a moving Figure in relation to a moving Ground (Nikanne, 2003). When using directional adpositions, a frame of reference must be assumed for interpreting the meaning of directional adpositions. For example, the meaning of to the left of in English can be based on a relative (speaker or listener based) reference frame or an intrinsic (object based) reference frame (Levinson, 1996). When a Figure and a Ground are both in motion, it is possible for a Figure to be described as being behind or in front of the Ground, even if neither have intrinsic features. As shown by Walker (in preparation), there are good reasons to assume that in the latter case a motion based reference frame is involved. This means that if Finnish speakers would use edellä (in front of) and jäljessä (behind) more frequently in situations where both the Figure and Ground are in motion, a difference in reference frame use between Finnish on one hand and English and Swedish on the other could be expected. We asked native English, Swedish and Finnish speakers’ to select adpositions from a language specific list to describe the location of a Figure relative to a Ground when both were shown to be moving on a computer screen. We were interested in any differences between Finnish, English and Swedish speakers. All languages showed a predominant use of directional spatial adpositions referring to the lexical concepts TO THE LEFT OF, TO THE RIGHT OF, ABOVE and BELOW. There were no differences between the languages in directional adpositions use or reference frame use, including reference frame use based on motion. We conclude that despite differences in the grammars of the languages involved, and potential differences in reference frame system use, the three languages investigated encode Figure location in relation to Ground location in a similar way when both are in motion. Levinson, S. C. (1996). Frames of reference and Molyneux’s question: Crosslingiuistic evidence. In P. Bloom, M.A. Peterson, L. Nadel & M.F. Garrett (Eds.) Language and Space (pp.109-170). Massachusetts: MIT Press. Nikanne, U. (2003). How Finnish postpositions see the axis system. In E. van der Zee & J. Slack (Eds.), Representing direction in language and space. Oxford, UK: Oxford University Press. Walker, C. (in preparation). Motion encoding in language, the use of spatial locatives in a motion context. Unpublished doctoral dissertation, University of Lincoln, Lincoln. United Kingdo

Engineering opportunities for originality and invention: the importance of playful making as developmental method in practice-led design research

Donald Schön (1995) describes the development of design through making as a reflective ‘conversation with the materials of the situation’. In design practice and research this dynamic dialogue often originates from playful making processes built on intuition and embodied knowledge. Using ludic research methods in practice-led research allows one to work spontaneously and without pre-meditated purpose. It offers opportunities to break out of linear patterns of thought and established ways of working ingrained by institutional education and years of professional practice. But how is it possible to preserve this precious playfulness within institutional constraints that often prioritise intellectualised, rationalised and well-documented methodologies? To achieve balance and rigor it is necessary to devise systems to record and reflect upon both the pragmatic and the phenomenological aspects of the research without losing the spontaneity of embodied, playful and intuitive design practices. This paper investigates the use of methods such as ‘reflection-in-action’ and ‘active documentation’ that allows the later evaluation and analysis of playful research activities whilst maintaining the ability to play authentically at the point of making. It explores how such methods might best support the development of original practice-led design research that retains playful practices of making at its core

Perspectives on adaptive dynamical systems

Adaptivity is a dynamical feature that is omnipresent in nature, socio-economics, and technology. For example, adaptive couplings appear in various real-world systems, such as the power grid, social, and neural networks, and they form the backbone of closed-loop control strategies and machine learning algorithms. In this article, we provide an interdisciplinary perspective on adaptive systems. We reflect on the notion and terminology of adaptivity in different disciplines and discuss which role adaptivity plays for various fields. We highlight common open challenges and give perspectives on future research directions, looking to inspire interdisciplinary approaches

Poetics of Reception: a phenomenological aesthetics of bodies and technology in performance

This study examines the provocative claim by Performance Studies theorist Philip Auslander (1999) that there is no ontological distinction between live and mediatised forms because they participate in the same cultural economy. This claim has led to something of a stagnation of debate between, on the one hand, scholars who privilege the live over the mediatised and on the other those who extinguish the live in favour of mediatisation. Moving beyond the limitations of ontology, this project proposes and develops a phenomenological aesthetics in order to investigate the essential structures and modes of experienced phenomena from within audience. The phenomenological approach understands the complexity and dynamism of the relationship between bodies and technologies in performance, reorienting the investigation away from a rehearsal of established and unhelpful ontological positions. The methodology for the project draws primarily upon methods from the N orth-American tradition of practical phenomenology (Herbert Spiegelberg, Edward S. Casey, Don Ihde, and Anthony Steinbock), and the transcendental philosophy of Edmund Husserl. Through a series of specially designed workshops, in which audience participants are trained in phenomenological techniques of bracketing and attention, A Poetics of Reception tests the potential of practical phenomenology to break the ontological impasse set up by Auslander. The method elicits the grasping of experiences of embodiment, kinesthetic empathy, temporality, orientation, imagination and poetic language. Participants were trained and required to write their experiences of the interaction between bodies and performance technologies, creating texts that then underwent hermeneutic analysis. The results of this interpretation yielded six interactive encounters, and revealed the constituted structures and modes of the relational phenomena experienced in performance by the participants. This study’s methodology has both practical and philoso! phical i mplications, including its proposed use as an audience-based dramaturgy for digital performance, and a method of inquiry into the kinesthetic dimensions of aesthetic experience