Haptic communication between partner dancers and swing as a finite state machine

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2005.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Vita.Includes bibliographical references (p. 129-138).To see two expert partners, one leading and the other following, swing dance together is to watch a remarkable two-agent communication and control system in action. Even blindfolded, the follower can decode the leader's moves from haptic cues. The leader composes the dance from the vocabulary of known moves so as to complement the music he is dancing to. Systematically addressing questions about partner dance communication is of scientific interest and could improve human-robotic interaction, and imitating the leader's choreographic skill is an engineering problem with applications beyond the dance domain. Swing dance choreography is a finite state machine, with moves that transition between a small number of poses. Two automated choreographers are presented. One uses an optimization and randomization scheme to compose dances by a sequence of shortest path problems, with edge lengths measuring the dissimilarity of dance moves to each bar of music. The other solves a two-player zero-sum game between the choreographer and a judge. Choosing moves at random from among moves that are good enough is rational under the game model.(cont.) Further, experiments presenting conflicting musical environments to two partners demonstrate that although musical expression clearly guides the leader's choice of moves, the follower need not hear the same music to properly decode the leader's signals. Dancers embody gentle interaction, in which each participant extends the capabilities of the other, and their cooperation is facilitated by a shared understanding of the motions to be performed. To demonstrate that followers use their understanding of the move vocabulary to interact better with their leaders, an experiment paired a haptic robot leader with human followers in a haptically cued dance to a swing music soundtrack. The subjects' performance differed significantly between instances when the subjects could determine which move was being led and instances when the subjects could not determine what the next move would be. Also, two-person teams that cooperated haptically to perform cyclical aiming tasks showed improvements in the Fitts' law or Schmidt's law speed-accuracy tradeoff consistent with a novel endpoint compromise hypothesis about haptic collaboration.by Sommer Elizabeth Gentry.Ph.D

Material agency and performative dynamics in the practices of media art

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel UniversityThis dissertation identifies a strategy of artistic inquiry within contemporary media art practice. It applies the concept of material that acts in an agential capacity, generating performative acts. It argues that the emergent potentials of materials and their interconnectedness with the compositional layers of a work can facilitate modes of effecting change in the artistic system. Through the theoretical investigation of the production processes of physical structures and environments, the thesis focuses on the compositional dynamics within which materials actively perform. It examines how Lars Spuybroek’s architectural design method of Material Machines (2004), and both the tactile potential as well as tactical uses of materials as generators to the formtaking process, might describe an open and active artistic strategy for employing the experimental capacities of such materialization processes. Building on philosophical and conceptual arguments that trace concepts of agency (Bruno Latour’s Actant-Network theory) and enactment (Karen Barad’s concept of intra-acting), the thesis introduces the two installation works ANI_MATE (described as a performative pneumatic stage machine) and ON TRACK (described as a mechanic-robotic installation). These apply the introduced artistic strategies. The analyses of these two artworks traces the particular capacities of the materials involved (respectively, their elasticity or viscosity) to negotiate forces of physical movement, which effect the system to transiently or irreversibly transform. ANI_MATE is a machine that is artist-operated and that explores the relationship between liveanimation procedures and the transformability and flexibility of its material environment. In contrast, ON TRACK’s performative machine ecology removes human agency. The machines act autonomously, giving rise to chance in the artistic system and allowing agency to emerge from the dynamic interconnectivity between materials, parts, and processes, eventually producing an entropic scenario of spilling resources. The thesis concludes that, in the context of a post digital paradigm in-development, such artistic practice offers a new strategy for an emergent aesthetics within contemporary physical-digital performance

Development of a toolkit for component-based automation systems

From the earliest days of mass production in the automotive industry there has been a progressive move towards the use of flexible manufacturing systems that cater for product variants that meet market demands. In recent years this market has become more demanding with pressures from legislation, globalisation and increased customer expectations. This has lead to the current trends of mass customisation in production. In order to support this manufacturing systems are not only becoming more flexible† to cope with the increased product variants, but also more agile‡ such that they may respond more rapidly to market changes. Modularisation§ is widely used to increase the agility of automation systems, such that they may be more readily reconfigured¶. Also with globalisation into India and Asia semi-automatic machines (machines that interact with human operators) are more frequently used to reduce capital outlay and increase flexibility. There is an increasing need for tools and methodologies that support this in order to improve design robustness, reduce design time and gain a competitive edge in the market. The research presented in this thesis is built upon the work from COMPAG/COMPANION (COMponent- based Paradigm for AGile automation, and COmmon Model for PArtNers in automatION), and as part of the BDA (Business Driven Automation), SOCRADES (Service Oriented Cross-layer infrastructure for Distributed smart Embedded deviceS), and IMC-AESOP (ArchitecturE for Service- Oriented Process – monitoring and control) projects conducted at Loughborough University UK. This research details the design and implementation of a toolkit for building and simulating automation systems comprising components with behaviour described using Finite State Machines (FSM). The research focus is the development of the engineering toolkit that can support the automation system lifecycle from initial design through commissioning to maintenance and reconfiguration as well as the integration of a virtual human. This is achieved using a novel data structure that supports component definitions for control, simulation, maintenance and the novel integration of a virtual human into the automation system operation

Towards a framework for socially interactive robots

250 p.En las últimas décadas, la investigación en el campo de la robótica social ha crecido considerablemente. El desarrollo de diferentes tipos de robots y sus roles dentro de la sociedad se están expandiendo poco a poco. Los robots dotados de habilidades sociales pretenden ser utilizados para diferentes aplicaciones; por ejemplo, como profesores interactivos y asistentes educativos, para apoyar el manejo de la diabetes en niños, para ayudar a personas mayores con necesidades especiales, como actores interactivos en el teatro o incluso como asistentes en hoteles y centros comerciales.El equipo de investigación RSAIT ha estado trabajando en varias áreas de la robótica, en particular,en arquitecturas de control, exploración y navegación de robots, aprendizaje automático y visión por computador. El trabajo presentado en este trabajo de investigación tiene como objetivo añadir una nueva capa al desarrollo anterior, la capa de interacción humano-robot que se centra en las capacidades sociales que un robot debe mostrar al interactuar con personas, como expresar y percibir emociones, mostrar un alto nivel de diálogo, aprender modelos de otros agentes, establecer y mantener relaciones sociales, usar medios naturales de comunicación (mirada, gestos, etc.),mostrar personalidad y carácter distintivos y aprender competencias sociales.En esta tesis doctoral, tratamos de aportar nuestro grano de arena a las preguntas básicas que surgen cuando pensamos en robots sociales: (1) ¿Cómo nos comunicamos (u operamos) los humanos con los robots sociales?; y (2) ¿Cómo actúan los robots sociales con nosotros? En esa línea, el trabajo se ha desarrollado en dos fases: en la primera, nos hemos centrado en explorar desde un punto de vista práctico varias formas que los humanos utilizan para comunicarse con los robots de una maneranatural. En la segunda además, hemos investigado cómo los robots sociales deben actuar con el usuario.Con respecto a la primera fase, hemos desarrollado tres interfaces de usuario naturales que pretenden hacer que la interacción con los robots sociales sea más natural. Para probar tales interfaces se han desarrollado dos aplicaciones de diferente uso: robots guía y un sistema de controlde robot humanoides con fines de entretenimiento. Trabajar en esas aplicaciones nos ha permitido dotar a nuestros robots con algunas habilidades básicas, como la navegación, la comunicación entre robots y el reconocimiento de voz y las capacidades de comprensión.Por otro lado, en la segunda fase nos hemos centrado en la identificación y el desarrollo de los módulos básicos de comportamiento que este tipo de robots necesitan para ser socialmente creíbles y confiables mientras actúan como agentes sociales. Se ha desarrollado una arquitectura(framework) para robots socialmente interactivos que permite a los robots expresar diferentes tipos de emociones y mostrar un lenguaje corporal natural similar al humano según la tarea a realizar y lascondiciones ambientales.La validación de los diferentes estados de desarrollo de nuestros robots sociales se ha realizado mediante representaciones públicas. La exposición de nuestros robots al público en esas actuaciones se ha convertido en una herramienta esencial para medir cualitativamente la aceptación social de los prototipos que estamos desarrollando. De la misma manera que los robots necesitan un cuerpo físico para interactuar con el entorno y convertirse en inteligentes, los robots sociales necesitan participar socialmente en tareas reales para las que han sido desarrollados, para así poder mejorar su sociabilida

Messaging in the noosphere: theatre art, integrated media, and human-comupter interaction

Bibliography: p. 188-19

Space, people, networks:exploring the relationship between built structures and seamless wireless communication infrastructures

In this thesis, I investigate wireless communication from an architectural perspective. I am using design prototypes to explore possibilities for interaction and designing with wirelessness in mind. The public primarily regards wireless networking technology as a technical infrastructure that should provide a seamless flow of information across a network of base stations, access points and mobile devices. From this perspective, wireless infrastructure is evaluated in terms of network availability and speed, and is continuously optimised. Researchers explored some other perspectives on wireless communication technology: they used computational spatial analysis to measure signal propagation in space. Some ethnographic studies explored its effect on the use of public space. Wireless connectivity was also explored through the philosophical framework of radical empiricism. All this points to the fact that wireless network infrastructure is a complex topic, spanning multiple fields of expertise and interest (engineering, architecture, urban studies but also sociology and philosophy). It is rarely explored from a plural perspective, as each study typically focuses on the one aspect within its expertise. I propose a more complex view of wireless connectivity, encompassing these different perspectives through an intellectual framework that is based on the notion of architecturality. Architecturality, a property common to all architecture but exceeding the limits of built artefacts, is a measure of the effect something has on the experience of space. Through the lens of the built environment, I expose the complex transactions that take place between networks, people and space. In order to evaluate architecturality of wireless communication signals, I conducted a series of practical design experiments, involving people and interactive installations, and using data gathered from mobile devices and wireless access points. The design of these experiments relies on the principles described by human-computer interaction (HCI) researchers as seamful design. Seamful design reveals underlying structures and relationships behind what appears as a utilitarian infrastructure. The design experiments contribute to the discussion on the use of design artefacts in practice-based research methodologies, thus challenging the different agents of knowledge production and the superiority of established research traditions. The insights gained from this complex examination of wireless networks are important for architectural design, as a way to account more adequately for signal propagation through buildings. The experience of internalising wireless networks in the process of design engenders a designerâs sensitivity towards the presence of wireless communications in space. This sensitivity, similar to the one we have for the distribution of natural and artificial lighting, will be needed in the ever more challenging design of the built environment. The sensible designer can account for, and envision, more dynamic environments that are able to accommodate change and information in completely new ways

Simulating Humans: Computer Graphics, Animation, and Control

People are all around us. They inhabit our home, workplace, entertainment, and environment. Their presence and actions are noted or ignored, enjoyed or disdained, analyzed or prescribed. The very ubiquitousness of other people in our lives poses a tantalizing challenge to the computational modeler: people are at once the most common object of interest and yet the most structurally complex. Their everyday movements are amazingly uid yet demanding to reproduce, with actions driven not just mechanically by muscles and bones but also cognitively by beliefs and intentions. Our motor systems manage to learn how to make us move without leaving us the burden or pleasure of knowing how we did it. Likewise we learn how to describe the actions and behaviors of others without consciously struggling with the processes of perception, recognition, and language

Digital Theatre: A "Live" and Mediated Art Form Expanding Perceptions of Body, Place, and Community

This work discusses Digital Theatre, a type of performance which utilizes both "live" actors and co-present audiences along with digital media to create a hybrid art form revitalizing theatre for contemporary audiences. This work surveys a wide range of digital performances (with "live" and digital elements, limited interactivity/participation and spoken words) and identifies the group collectively as Digital Theatre, an art form with the flexibility and reach of digital data and the sense of community found in "live" theatre. I offer performance examples from Mark Reaney, David Saltz, Troika Ranch, Gertrude Stein Repertory Theatre, Flying Karamazov Brothers, Talking Birds, Yacov Sharir, Studio Z, George Coates Performance Group, and ArtGrid. (The technologies utilized in performances include: video-conferencing, media projection, MIDI control, motion capture, VR animation, and AI). Rather than looking at these productions as isolated events, I identify them as a movement and link the use of digital techniques to continuing theatrical tradition of utilizing new technologies on the stage. The work ties many of the aesthetic choices explored in theatrical past by the likes of Piscator, Svoboda, Craig, and in Bauhaus and Futurist movements. While it retains the essential qualities of public human connection and imaginative thought central to theatre, Digital Theatre can cause theatrical roles to merge as it extends the performer's body, expands our concept of place, and creates new models of global community

Gesture generation by imitation : from human behavior to computer character animation

This dissertation shows how to generate conversational gestures for an animated agent based on annotated text input. The central idea is to imitate the gestural behavior of human individuals. Using TV show recordings as empirical data, gestural key parameters are extracted for the generation of natural and individual gestures. For each of the three tasks in the generation pipeline a software was developed. The generic ANVIL annotation tool allows to transcribe gesture and speech in the empirical data. The NOVALIS module uses the annotations to compute individual gesture profiles with statistical methods. The NOVA generator creates gestures based on these profiles and heuristic rules, and outputs them in a linear script. In all, this work presents a complete work pipeline from collecting empirical data to obtaining an executable script and provides the necessary software, too.Die vorliegende Dissertation stellt einen Ansatz zur Generierung von Konversationsgesten für animierte Agenten aus annotatiertem Textinput vor. Zentrale Idee ist es, die Gestik menschlicher Individuen zu imitieren. Als empirisches Material dient eine Fernsehsendung, aus der Schlüsselparameter zur Generierung natürlicher und individueller Gesten extrahiert werden. Die Generierungsaufgabe wurde in drei Schritten mit eigens entwickelter Software gelöst. Das generische ANVIL-Annotationswerkzeug ermöglicht die Transkription von Gestik und Sprache in den empirischen Daten. Das NOVALIS-Modul berechnet aus den Annotationen individuelle Gestenprofile mit Hilfe statistischer Verfahren. Der NOVAGenerator erzeugt Gesten anhand dieser Profile und allgemeiner Heuristiken und gibt diese in Skriptform aus. Die Arbeit stellt somit einen vollständigen Arbeitspfad von empirischer Datenerhebung bis zum abspielfertigen Skript vor und liefert die entsprechenden Software-Werkzeuge dazu