Expressive social exchange between humans and robots

Thesis (Sc.D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2000.Includes bibliographical references (p. 253-264).Sociable humanoid robots are natural and intuitive for people to communicate with and to teach. We present recent advances in building an autonomous humanoid robot, Kismet, that can engage humans in expressive social interaction. We outline a set of design issues and a framework that we have found to be of particular importance for sociable robots. Having a human-in-the-loop places significant social constraints on how the robot aesthetically appears, how its sensors are configured, its quality of movement, and its behavior. Inspired by infant social development, psychology, ethology, and evolutionary perspectives, this work integrates theories and concepts from these diverse viewpoints to enable Kismet to enter into natural and intuitive social interaction with a human caregiver, reminiscent of parent-infant exchanges. Kismet perceives a variety of natural social cues from visual and auditory channels, and delivers social signals to people through gaze direction, facial expression, body posture, and vocalizations. We present the implementation of Kismet's social competencies and evaluate each with respect to: 1) the ability of naive subjects to read and interpret the robot's social cues, 2) the robot's ability to perceive and appropriately respond to naturally offered social cues, 3) the robot's ability to elicit interaction scenarios that afford rich learning potential, and 4) how this produces a rich, flexible, dynamic interaction that is physical, affective, and social. Numerous studies with naive human subjects are described that provide the data upon which we base our evaluations.by Cynthia L. Breazeal.Sc.D

Real-time generation and adaptation of social companion robot behaviors

Social robots will be part of our future homes. They will assist us in everyday tasks, entertain us, and provide helpful advice. However, the technology still faces challenges that must be overcome to equip the machine with social competencies and make it a socially intelligent and accepted housemate. An essential skill of every social robot is verbal and non-verbal communication. In contrast to voice assistants, smartphones, and smart home technology, which are already part of many people's lives today, social robots have an embodiment that raises expectations towards the machine. Their anthropomorphic or zoomorphic appearance suggests they can communicate naturally with speech, gestures, or facial expressions and understand corresponding human behaviors. In addition, robots also need to consider individual users' preferences: everybody is shaped by their culture, social norms, and life experiences, resulting in different expectations towards communication with a robot. However, robots do not have human intuition - they must be equipped with the corresponding algorithmic solutions to these problems. This thesis investigates the use of reinforcement learning to adapt the robot's verbal and non-verbal communication to the user's needs and preferences. Such non-functional adaptation of the robot's behaviors primarily aims to improve the user experience and the robot's perceived social intelligence. The literature has not yet provided a holistic view of the overall challenge: real-time adaptation requires control over the robot's multimodal behavior generation, an understanding of human feedback, and an algorithmic basis for machine learning. Thus, this thesis develops a conceptual framework for designing real-time non-functional social robot behavior adaptation with reinforcement learning. It provides a higher-level view from the system designer's perspective and guidance from the start to the end. It illustrates the process of modeling, simulating, and evaluating such adaptation processes. Specifically, it guides the integration of human feedback and social signals to equip the machine with social awareness. The conceptual framework is put into practice for several use cases, resulting in technical proofs of concept and research prototypes. They are evaluated in the lab and in in-situ studies. These approaches address typical activities in domestic environments, focussing on the robot's expression of personality, persona, politeness, and humor. Within this scope, the robot adapts its spoken utterances, prosody, and animations based on human explicit or implicit feedback.Soziale Roboter werden Teil unseres zukünftigen Zuhauses sein. Sie werden uns bei alltäglichen Aufgaben unterstützen, uns unterhalten und uns mit hilfreichen Ratschlägen versorgen. Noch gibt es allerdings technische Herausforderungen, die zunächst überwunden werden müssen, um die Maschine mit sozialen Kompetenzen auszustatten und zu einem sozial intelligenten und akzeptierten Mitbewohner zu machen. Eine wesentliche Fähigkeit eines jeden sozialen Roboters ist die verbale und nonverbale Kommunikation. Im Gegensatz zu Sprachassistenten, Smartphones und Smart-Home-Technologien, die bereits heute Teil des Lebens vieler Menschen sind, haben soziale Roboter eine Verkörperung, die Erwartungen an die Maschine weckt. Ihr anthropomorphes oder zoomorphes Aussehen legt nahe, dass sie in der Lage sind, auf natürliche Weise mit Sprache, Gestik oder Mimik zu kommunizieren, aber auch entsprechende menschliche Kommunikation zu verstehen. Darüber hinaus müssen Roboter auch die individuellen Vorlieben der Benutzer berücksichtigen. So ist jeder Mensch von seiner Kultur, sozialen Normen und eigenen Lebenserfahrungen geprägt, was zu unterschiedlichen Erwartungen an die Kommunikation mit einem Roboter führt. Roboter haben jedoch keine menschliche Intuition - sie müssen mit entsprechenden Algorithmen für diese Probleme ausgestattet werden. In dieser Arbeit wird der Einsatz von bestärkendem Lernen untersucht, um die verbale und nonverbale Kommunikation des Roboters an die Bedürfnisse und Vorlieben des Benutzers anzupassen. Eine solche nicht-funktionale Anpassung des Roboterverhaltens zielt in erster Linie darauf ab, das Benutzererlebnis und die wahrgenommene soziale Intelligenz des Roboters zu verbessern. Die Literatur bietet bisher keine ganzheitliche Sicht auf diese Herausforderung: Echtzeitanpassung erfordert die Kontrolle über die multimodale Verhaltenserzeugung des Roboters, ein Verständnis des menschlichen Feedbacks und eine algorithmische Basis für maschinelles Lernen. Daher wird in dieser Arbeit ein konzeptioneller Rahmen für die Gestaltung von nicht-funktionaler Anpassung der Kommunikation sozialer Roboter mit bestärkendem Lernen entwickelt. Er bietet eine übergeordnete Sichtweise aus der Perspektive des Systemdesigners und eine Anleitung vom Anfang bis zum Ende. Er veranschaulicht den Prozess der Modellierung, Simulation und Evaluierung solcher Anpassungsprozesse. Insbesondere wird auf die Integration von menschlichem Feedback und sozialen Signalen eingegangen, um die Maschine mit sozialem Bewusstsein auszustatten. Der konzeptionelle Rahmen wird für mehrere Anwendungsfälle in die Praxis umgesetzt, was zu technischen Konzeptnachweisen und Forschungsprototypen führt, die in Labor- und In-situ-Studien evaluiert werden. Diese Ansätze befassen sich mit typischen Aktivitäten in häuslichen Umgebungen, wobei der Schwerpunkt auf dem Ausdruck der Persönlichkeit, dem Persona, der Höflichkeit und dem Humor des Roboters liegt. In diesem Rahmen passt der Roboter seine Sprache, Prosodie, und Animationen auf Basis expliziten oder impliziten menschlichen Feedbacks an

Learning To Refuse: Pedagogy, Protest, And Lecture-Performance, 1964-1975

This study examines how artists in the US reimagined aesthetic practice through performances of refusal from 1964 to 1975. Attending to the emergent genre of the lecture-performance, I analyze pedagogical projects that articulate dissent through interventions into existing models of knowledge, asking: what is to be learned from saying no? These projects respond, in part, to artists’ encounters with university training. They redefine artistic activity through critical engagements with the labor of the information worker, a figure sartorially invoked by the bespectacled uniform of a professorial archetype. Artists deployed the lecture format to imagine how knowledge might be assembled otherwise: within counter-institutional frameworks, beyond authorized discourse, through embodied tactics of performativity, and toward socially transformative ends. They did so at a moment when artists’ academicization proceeded as an explicitly gendered project that privileged masculine-coded cognitive labor over and against modes of work coded as feminized craft. Jettisoning these divisions, the lecture-performance situates knowledge in the specificity of embodied agents. In this way, lecture-performance renegotiates the discursive practices that regulate bodies of knowledge and knowledgeable bodies. Placing these developments in conversation with the agitational speech of artist activism, my study focuses on affiliates of the 1970 Art Strike Against Racism, Sexism, War, and Repression. It tracks forms of pedagogy and protest across a range of media beyond the lecture-performance, including video lectures, pamphlets, and photographic series by Robert Morris, Adrian Piper, Faith Ringgold, and Andy Warhol. Its case studies toggle between artworks, performative speech acts, and direct action, arguing for the porousness of their categorical boundaries in this period. Redressing the claim that artists’ strikes, protests, and boycotts foreclose possibilities for productive engagement, I route practices of refusal toward their generative, dialogic capacities. Charting the convergence of movements in art and activism from 1964 to 1975, this study asks what we have to learn from statements of refusal delivered at the interstices of academic lecterns, political podiums, and sites of artistic display

25th Annual Computational Neuroscience Meeting: CNS-2016

Abstracts of the 25th Annual Computational Neuroscience Meeting: CNS-2016 Seogwipo City, Jeju-do, South Korea. 2–7 July 201

25th annual computational neuroscience meeting: CNS-2016

The same neuron may play different functional roles in the neural circuits to which it belongs. For example, neurons in the Tritonia pedal ganglia may participate in variable phases of the swim motor rhythms [1]. While such neuronal functional variability is likely to play a major role the delivery of the functionality of neural systems, it is difficult to study it in most nervous systems. We work on the pyloric rhythm network of the crustacean stomatogastric ganglion (STG) [2]. Typically network models of the STG treat neurons of the same functional type as a single model neuron (e.g. PD neurons), assuming the same conductance parameters for these neurons and implying their synchronous firing [3, 4]. However, simultaneous recording of PD neurons shows differences between the timings of spikes of these neurons. This may indicate functional variability of these neurons. Here we modelled separately the two PD neurons of the STG in a multi-neuron model of the pyloric network. Our neuron models comply with known correlations between conductance parameters of ionic currents. Our results reproduce the experimental finding of increasing spike time distance between spikes originating from the two model PD neurons during their synchronised burst phase. The PD neuron with the larger calcium conductance generates its spikes before the other PD neuron. Larger potassium conductance values in the follower neuron imply longer delays between spikes, see Fig. 17.Neuromodulators change the conductance parameters of neurons and maintain the ratios of these parameters [5]. Our results show that such changes may shift the individual contribution of two PD neurons to the PD-phase of the pyloric rhythm altering their functionality within this rhythm. Our work paves the way towards an accessible experimental and computational framework for the analysis of the mechanisms and impact of functional variability of neurons within the neural circuits to which they belong

IberSPEECH 2020: XI Jornadas en Tecnología del Habla and VII Iberian SLTech

IberSPEECH2020 is a two-day event, bringing together the best researchers and practitioners in speech and language technologies in Iberian languages to promote interaction and discussion. The organizing committee has planned a wide variety of scientific and social activities, including technical paper presentations, keynote lectures, presentation of projects, laboratories activities, recent PhD thesis, discussion panels, a round table, and awards to the best thesis and papers. The program of IberSPEECH2020 includes a total of 32 contributions that will be presented distributed among 5 oral sessions, a PhD session, and a projects session. To ensure the quality of all the contributions, each submitted paper was reviewed by three members of the scientific review committee. All the papers in the conference will be accessible through the International Speech Communication Association (ISCA) Online Archive. Paper selection was based on the scores and comments provided by the scientific review committee, which includes 73 researchers from different institutions (mainly from Spain and Portugal, but also from France, Germany, Brazil, Iran, Greece, Hungary, Czech Republic, Ucrania, Slovenia). Furthermore, it is confirmed to publish an extension of selected papers as a special issue of the Journal of Applied Sciences, “IberSPEECH 2020: Speech and Language Technologies for Iberian Languages”, published by MDPI with fully open access. In addition to regular paper sessions, the IberSPEECH2020 scientific program features the following activities: the ALBAYZIN evaluation challenge session.Red Española de Tecnologías del Habla. Universidad de Valladoli

Temporally scattered brain: neural mechanism apprehending the paradox of the discrete and continuous flow of consciousness

In dieser Dissertation werden offene Fragen über „Bewußtsein“ aus neurowissenschaftlicher, psychologischer und auch philosophischer Sicht erörtert. Es geht insbesondere darum, wie in traditionellen und modernen Ansätzen die Beziehung von zeitlicher Informations-Verarbeitung und Repräsentationen im Bewußtsein konzipiert werden. In einer historischen Aufarbeitung wird gezeigt, dass Zeit als mentale Kategorie in der psychologischen und neurowissenschaftlichen Forschung lange eher vernachlässigt worden ist, was sich erst jetzt zu ändern scheint. Wie wichtig Informations-Verarbeitung im Zeitbereich für ein besseres Verständnis von Bewußtseinsprozessen ist, wird herausgearbeitet, denn alle Verhaltensweisen und alles bewußte Erleben haben notwendigerweise eine zeitliche Charakteristik. Die grundsätzliche aber bisher nicht hinreichend beantwortete Frage ist, ob diese Charakteristik, die Informations-Verarbeitung in der Zeit, als kontinuierlich oder als diskret zu verstehen ist. Neue Befunde legen nahe, dass so genannte „Zeitfenster“ bestimmter Dauer notwendig sind, um neuronale Informations-Verarbeitung zu ermöglichen, damit Bewußtseinsprozesse überhaupt entstehen können. Wenn aber diskrete Zeitfenster notwendig sind, dann stellt sich die weitere Frage, wie es paradoxerweise zum subjektiven Eindruck einer zeitlichen Kontinuität zum Beispiel in der Wahrnehmung kommen kann. Das Erleben zeitlicher Kontinuität bezieht sich nicht nur auf Prozesse der Wahrnehmng und des Erlebens, sondern auch auf den viel weiteren Rahmen der Kontinuität personaler Identität. Es wird darauf hingewiesen, dass diese Kontinuität bei bestimmten Erkrankungen oder veränderten Bewußtseinszuständen verloren gehen kann. Ein derartiger Verlust legt nahe, dass es aktive Mechanismen auf neuronaler Ebene geben sollte, die personale Identität über die Zeit hinweg erzeugen. Es mußt gleichsam ein „Klebstoff“ vermutet werden, der zeitlich Diskretes in anschauliche Kontinuität verwandelt. Mechanismen, die hierfür in Frage kommen könnten, sind bisher nicht bekannt. Ein erster Versuch zur Aufklärung dieser Frage wird mit einem fMRT-Experiment gemacht, in dem visuelle und auditive Vorstellungen untersucht werden. Hier zeigt sich, dass in beiden Modalitäten gemeinsame neuronale Aktivierungen in bestimmten Hirnstrukturen zu beobachten sind, was möglicherweise einen ersten Hinweis auf die Erzeugung von anschaulicher Kontinuität geben könnte. Des weiteren wird am Ende der Arbeit auf die Bedeutung des Reafferenzprinzips hingewiesen, das vielleicht einen neuen Ansatz zum besseren Verständnis mancher Phänomene wie des Déjà Vu geben könnte, wenn man in dieses klassische Prinzip einen Zeitfaktor integriert, was bisher theoretisch nicht geschehen ist. Während der Fokus der Arbeit auf theoretischen Konzepten zu Zeit und Bewußtsein im psychologischen, neurowissenschaftlichen und philosophischen Kontext liegt, werden auch Bezüge zu den Künsten, der Dichtkunst und der Musik, offen gelegt. Dies soll darauf hinweisen, dass die Beziehung zwischen der „Zeit des Menschen“ und dem bewußten Erleben ein Menschheitsthema ist, das über den wissenschhaftlichen Rahmen hinaus weist.In this dissertation open questions about “consciousness” are discussed from a psychological, neuroscientific and also philosophical perspective. In particular, it is described how the relationship between temporal information processing and conscious representations is conceived in traditional and modern approaches. In a historical review it is shown that time as a mental category has been neglected for a long time in psychological and neuroscientific research, although this is changing recently. The importance of temporal information processing for a better understanding of conscious processes is analyzed because any behavior and all subjective experiences necessarily have a temporal characteristic. There is, however, a basic and still open question whether this characteristic, i.e., temporal information processing, is continuous or discrete. New research suggests that “time windows” of specific durations are necessary in neural information processing being the basis for conscious representations. If, however, discrete time windows are necessary the question arises how paradoxically the subjective impression of temporal continuity for instance in perception is possible. The impression of temporal continuity refers, however, not only to processes of perception and experience, but in a broader context also to the continuity of personal identity. It is indicated that this continuity can break down in certain diseases or in altered states of consciousness. Such losses suggest the existence of active mechanisms on the neural level which allow for the creation of personal identity across time. Some kind of “glue” has to be suspected that transforms what is temporally discrete into apparent continuity. Potential mechanisms for this transformation are still not known. A first attempt is made to answer this question with an fMRI-experiment in which visual and auditory images are analyzed. For both modalities’ common activations in certain brain regions are observed which possibly might be a first indication about a mechanism creating apparent continuity. At the end of the dissertation the importance of the reafference principle is stressed as it may provide a new perspective towards a better understanding of some phenomena like Déja Vu, if one includes in this classical principle a temporal factor which theoretically has not been done yet. Although the focus of the dissertation lies on theoretical concepts of time and consciousness within a psychological, neuroscientific and philosophical context, some links to the arts like poetry and music are made transparent. This shall indicate that the relationship between the “time of humans” and conscious experiences is a topic of humankind that goes beyond the scientific frame