Semantic HELM: A Human-Readable Memory for Reinforcement Learning

Reinforcement learning agents deployed in the real world often have to cope with partially observable environments. Therefore, most agents employ memory mechanisms to approximate the state of the environment. Recently, there have been impressive success stories in mastering partially observable environments, mostly in the realm of computer games like Dota 2, StarCraft II, or MineCraft. However, existing methods lack interpretability in the sense that it is not comprehensible for humans what the agent stores in its memory. In this regard, we propose a novel memory mechanism that represents past events in human language. Our method uses CLIP to associate visual inputs with language tokens. Then we feed these tokens to a pretrained language model that serves the agent as memory and provides it with a coherent and human-readable representation of the past. We train our memory mechanism on a set of partially observable environments and find that it excels on tasks that require a memory component, while mostly attaining performance on-par with strong baselines on tasks that do not. On a challenging continuous recognition task, where memorizing the past is crucial, our memory mechanism converges two orders of magnitude faster than prior methods. Since our memory mechanism is human-readable, we can peek at an agent's memory and check whether crucial pieces of information have been stored. This significantly enhances troubleshooting and paves the way toward more interpretable agents.Comment: To appear at NeurIPS 2023, 10 pages (+ references and appendix), Code: https://github.com/ml-jku/hel

The emergence of language as a function of brain-hemispheric feedback

This text posits the emergence of language as a function of brain-hemispheric feedback, where “emergence” refers to the generation of complex patterns from relatively simple interactions, “language” refers to an abstraction-based and representational-recombinatorial-recursive mapping-signaling system, “function” refers to an input-output relationship described by fractal algorithms, “brain-hemispheric” refers to complementary (approach-abstraction / avoidance-gestalt) cognitive modules, and “feedback” refers to self-regulation driven by neural inhibition and recruitment. The origin of language marks the dawn of human self-awareness and culture, and is thus a matter of fundamental and cross-disciplinary interest. This text is a synthesized research essay that constructs its argument by drawing diverse scholarly voices into a critical, cross-disciplinary intertextual narrative. While it does not report any original empirical findings, it harnesses those made by others to offer a tentative, partial solution—one that can later be altered and expanded—to a problem that has occupied thinkers for centuries. The research contained within this text is preceded by an introductory Section 1 that contextualizes the problem of the origin of language. Section 2 details the potential of evolutionary theory for addressing the problem, and the reasons for the century-long failure of linguistics to take advantage of that potential. Section 3 reviews the history of the discovery of brain lateralization, as well as its behavioral and structural characteristics. Section 4 discusses evolutionary evidence and mechanisms in terms of increasing adaptive complexity and intelligence, in general, and tool use, in particular. Section 5 combines chaos theory, brain science, and semiotics to propose that, after the neotenic acquisition of contingency-based abstraction, language emerged as a feedback interaction between the left-hemisphere abstract word and the right-hemisphere gestalt image. I conclude that the model proposed here might be a valuable tool for understanding, organizing, and relating data and ideas concerning human evolution, language, culture, and psychology. I recommend, of course, that I present this text to the scholarly community for criticism, and that I continue to gather and collate relevant data and ideas, in order to prepare its next iteration

Procedural Films: Algorithmic Affect in Research Media Art Practice

This thesis explores the political aesthetics of ‘procedural films’—media works that use generative algorithmic procedures and manifest as moving images. In contrast to long-held techno-positivist understandings of generative art, the thesis reframes procedural films as a critical media art practice aiming to understand the ‘procedure’ as an affective engine of moving image experience. It employs an interdisciplinary approach that borrows from materialist theories of media, experimental film, artificial life and computational culture, and draws on my practices as artist and curator. These processes of making, curating and experiencing serve as enacted research, as a scalable architecture of thinking through and thinking with the technical media. The thesis proposes a conceptual framework for exploring procedural films as techno-cultural artefacts, addressing the ‘apparatus’, the affective space-time of their viewing and their sociopolitical operation. It proposes that algorithmic autonomy brings an affective renegotiation of the traditional roles of the spectator and the moving image, instead seeing it as a complex entanglement of human and non-human agencies, computational temporalities and generative procedures. Furthermore, it addresses procedural mediation and automation as a part of the political aesthetics of media art, exploring the techno-capitalist commodification of attention, time and images. The thesis investigates two case studies—screensaver and game engine—as procedural apparatuses. It explores these media artefacts as sites of labour, design, affect and experience, addressing their techno-cultural construction, as well as their processes of liveness and emergence

Listening to the tale of the Old Man : conceptions of old age in radio works by Samuel Beckett and Wolfgang Hildesheimer

In this dissertation, I set out to examine the depiction and problematisation of old age in the radio art of Samuel Beckett and Wolfgang Hildesheimer. I lay the groundwork for the close reading of the radio works of these two writers by surveying both the technological and ontological status of mid-twentieth-century radio art and the prevailing associations, stereotypes, and definitions surrounding ageing. Drawing on a keen awareness of the specificity of radio as an artistic medium and on notions anchored in both chronometric time (such as disengagement theory and activity theory) and ‘lived’ time (such as the réveil mortel and the idea of old age as the radicalisation of the human condition), I discuss the deliberate and recurrent thematisation of old age in the radio art of Beckett and Hildesheimer, as well as other important and interrelated concepts such as embodiment, intersubjectivity, and meaning-making. Thus, the intriguing link (established by means of the parable) between the theme of old age, the philosophical concerns that animated Beckett and Hildesheimer, and the unique features of radio is shown to be not the mere fruit of chance but indeed the logical outcome of an unassailable theoretical symbiosis.Nesta dissertação, proponho-me a examinar a representação e problematização da velhice na arte radiofónica de Samuel Beckett e Wolfgang Hildesheimer. Os alicerces para uma leitura atenta das suas obras radiofónicas são estabelecidos através de uma análise do estado tecnológico e ontológico da arte radiofónica de meados do século XX e das associações, estereótipos e definições em torno do envelhecimento. Servindo-me de uma consciência clara da especificidade da rádio enquanto meio artístico e de noções ancoradas tanto no tempo cronométrico (tal como a disengagement theory e a activity theory) como no tempo ‘vivido’ (tal como o réveil mortel e a noção de velhice enquanto radicalização da condição humana), debruço-me sobre a tematização deliberada e recorrente da velhice na arte radiofónica de Beckett e Hildesheimer, assim como outros conceitos importantes e interrelacionados como a corporalidade, a intersubjectividade e a criação de sentido. Assim, a curiosa ligação (estabelecida através da parábola) entre o tema da velhice, as preocupações filosóficas que animaram Beckett e Hildesheimer, e as características únicas da rádio revela-se não como mero fruto do acaso mas sim como o resultado lógico de uma simbiose teórica indiscutível