Visual Decoding of Targets During Visual Search From Human Eye Fixations

What does human gaze reveal about a users' intents and to which extend can these intents be inferred or even visualized? Gaze was proposed as an implicit source of information to predict the target of visual search and, more recently, to predict the object class and attributes of the search target. In this work, we go one step further and investigate the feasibility of combining recent advances in encoding human gaze information using deep convolutional neural networks with the power of generative image models to visually decode, i.e. create a visual representation of, the search target. Such visual decoding is challenging for two reasons: 1) the search target only resides in the user's mind as a subjective visual pattern, and can most often not even be described verbally by the person, and 2) it is, as of yet, unclear if gaze fixations contain sufficient information for this task at all. We show, for the first time, that visual representations of search targets can indeed be decoded only from human gaze fixations. We propose to first encode fixations into a semantic representation and then decode this representation into an image. We evaluate our method on a recent gaze dataset of 14 participants searching for clothing in image collages and validate the model's predictions using two human studies. Our results show that 62% (Chance level = 10%) of the time users were able to select the categories of the decoded image right. In our second studies we show the importance of a local gaze encoding for decoding visual search targets of use

The development of a human-robot interface for industrial collaborative system

Industrial robots have been identified as one of the most effective solutions for optimising output and quality within many industries. However, there are a number of manufacturing applications involving complex tasks and inconstant components which prohibit the use of fully automated solutions in the foreseeable future. A breakthrough in robotic technologies and changes in safety legislations have supported the creation of robots that coexist and assist humans in industrial applications. It has been broadly recognised that human-robot collaborative systems would be a realistic solution as an advanced production system with wide range of applications and high economic impact. This type of system can utilise the best of both worlds, where the robot can perform simple tasks that require high repeatability while the human performs tasks that require judgement and dexterity of the human hands. Robots in such system will operate as “intelligent assistants”. In a collaborative working environment, robot and human share the same working area, and interact with each other. This level of interface will require effective ways of communication and collaboration to avoid unwanted conflicts. This project aims to create a user interface for industrial collaborative robot system through integration of current robotic technologies. The robotic system is designed for seamless collaboration with a human in close proximity. The system is capable to communicate with the human via the exchange of gestures, as well as visual signal which operators can observe and comprehend at a glance. The main objective of this PhD is to develop a Human-Robot Interface (HRI) for communication with an industrial collaborative robot during collaboration in proximity. The system is developed in conjunction with a small scale collaborative robot system which has been integrated using off-the-shelf components. The system should be capable of receiving input from the human user via an intuitive method as well as indicating its status to the user ii effectively. The HRI will be developed using a combination of hardware integrations and software developments. The software and the control framework were developed in a way that is applicable to other industrial robots in the future. The developed gesture command system is demonstrated on a heavy duty industrial robot

Hack the Experience: Tools for Artists from Cognitive Science

Hack The Experience will reframe your perspective on how your audience engages your work. This will happen as you learn how to control attention through spatial and time-based techniques that you can harness as you build immersive installations or as you think about how to best arrange your work in an exhibition. You’ll learn things about the senses and how they interface with attention so that you can build in visceral forms of interactivity, engage people’s empathetic responses, and frame their moods. This book is a dense bouillon-cube of techniques that you can adapt and apply to your personal practice, and it’s a book that will walk you step-by-step through skill sets from ethnography, cognitive science, and multi-modal metaphors. The core argument of this book is that art is a form of cognitive engineering and that the physical environment (or objects in the physical environment) can be shaped to maximize emotional and sensory experience. Many types of art will benefit from this handbook (because cognition is pervasive in our experience of art), but it is particularly relevant to immersive experiential works such as installations, participatory/interactive environments, performance art, curatorial practice, architecture and landscape architecture, complex durational works, and works requiring new models of documentation. These types of work benefit from the empirical findings of cognitive science because intentionally leveraging basic human cognition in artworks can give participants new ways of seeing the world that are cognitively relevant. This leveraging process provides a new layer in the construction of conceptually grounded works

For an echology of microbe-artworks : thinking in between art and science

Une entité scientifique, tout en ayant son propre devenir dans le domaine scientifique, s’étend aussi souvent à d’autres domaines d’activité. Parallèlement à la diffusion des découvertes scientifiques, elle peut susciter un intérêt artistique ou conceptuel. Les études sur le microbiome humain ont nourri un tel intérêt pour les microbes et ont encouragé de nombreux artistes à entrer dans un laboratoire de biologie et à produire des oeuvres artistiques avec et à travers les microbes. Ces oeuvres d’art établissent une relation étroite avec les découvertes scientifiques récentes, les procédures et les protocoles, et posent des questions philosophiques sur la vie et la mort, la nature, l’humanité, et les relations entre les êtres vivants. Cette thèse vise à examiner les processus sociaux, techniques, politiques et économiques qui traversent les sciences des microbes et à déterminer comment ils aboutissent dans les oeuvres d’Elaine Whittaker, Tarsh Bates, François-Joseph Lapointe, Günes-Helen Isitan, le collectif Interspecifics, Victoria Shennan, Saša Spačal, Sonja Bäumel, Raphael Kim et Kathy High. Lorsque nous trouvons un microbe dans un contexte particulier, que trouvons-nous d’autre avec lui ? Dans quelles conditions apparaît-il dans une oeuvre d’art et avec quels éléments l’oeuvre compose-t-elle pour produire des effets esthétiques ? Dans cette thèse, l’histoire des microbes considérée du point de vue des formes d’art les mobilisant (ou « microbe-oeuvres d’art » pour microbe-artworks) commence en fait avec des animalcules qui n’étaient pas encore des entités scientifiques à part entière, mais qui présentaient virtuellement les forces qui seraient réunies plus tard sous le terme scientifique de « microbe ». Dans un premier temps, les animalcules, nommés après des observations d’Antonie von Leeuwenhoek, ont suscité l’intérêt de philosophes comme Leibniz et Spinoza et intensifié la curiosité de peintres comme Johannes Vermeer pour les éléments microscopiques de la vision, initiant ainsi des voyages entre les champs scientifiques et artistiques. Cette étude propose de problématiser ces voyages à l’aide du concept d’« échologie », un terme oublié d’une thèse écrite dans les années 1970 par Jean Milet sur la sociologie de Gabriel Tarde. Mais les théories d’autres philosophes tels que Georges Canguilhem, Michel Foucault, Gilles Deleuze, Félix Guattari, Marie-José Mondzain, et Gilbert Simondon, et des penseurs contemporains tels que Thierry Bardini et Brian Massumi sont également mobilisées pour donner à ce terme toute sa cohérence. Selon l’échologie, les entités sont constituées des motifs (patterns) d’interférence et de résonance avec d’autres choses, qui précèdent leur représentation. Ainsi, une 5 entité donnée est un complexe de forces, et son apparition, le résultat de certaines techniques qui la mettent en relation avec d’autres complexes ne peut s’expliquer comme un effet associé à une seule cause, mais se donne comme un effet supplémentaire, un extra-effet ou un surplus qui laisse toujours une trace ou un résidu. D’un point de vue échologique, une microbe-oeuvre d’art s’opère comme une interface qui intègre des potentiels qui se rendent visibles à travers les traces en vertu de multiples processus recoupant les activités scientifiques et les stratégies artistiques. Chaque chapitre de la thèse est ainsi une étape dans un voyage conceptuel expérimental, révélant les dimensions des oeuvres d’art considérées au regard de l’analyse de ces traces. Au cours de ce voyage, les éléments des théories scientifiques concernées, des entretiens avec des artistes, des sorties sur des sites de pratique des arts biologiques, lors d’ateliers, de conférence et d’écoles d’été sont mobilisés comme facteurs contribuant à la construction des champs problématiques dans chaque chapitre. Les microbes considérés comme des objets de beauté apparaissent comme le résultat d’une transformation discursive des sciences biologiques. D’une conception pathogène des microbes aux approches écologiques, l’iconicité des microbes associés aux microbe-images, l’échologie des microbe-sons, le devenir-milieu de certaines microbe-oeuvres d’art, et enfin la question de l’individuation de la pensée, et l’éthique corrélée compris comme le problème de la valorisation des microbes dans des microbe-oeuvres d’art, le devenir-microbe découle de cette transformation discursive à travers le champ artistique.A scientific entity, while having its own becoming in the scientific field, often also spreads to other fields of activity, such as art and philosophy. Microbiome studies fed such an interest towards microbes and encouraged many artists to enter a biology laboratory and produce a work of art with and through microbes. These artworks establish a close relationship with recent scientific findings, procedures and protocols, and ask philosophical questions about life and death, nature, humanness, and the relationships between living beings. This thesis aims to examine the social, technical, political, and economic processes that go through the microbe sciences and determine how they come together in the artworks of Elaine Whittaker, Tarsh Bates, François-Joseph Lapointe, Günes-Helen Isitan, the collective Interspecifics, Victoria Shennan, Saša Spačal, Sonja Bäumel, Raphael Kim, and Kathy High. When we find a microbe in a particular context, what else do we find with it? Under which conditions does it appear in an artwork and which elements does the artwork compose with to produce aesthetic effects? In this thesis, the story of microbes is recounted from the perspective of microbe-artworks and starts with animalcules, the not yet full-fledged scientific entity which virtually present the forces that would be brought together under the scientific term “microbe”. At first, animalcules––named after Antonie van Leeuwenhoek’s observations, attracted the interest of philosophers such as Leibniz and Spinoza and intensified the curiosity of painters such as Johannes Vermeer towards the microscopic elements of seeing, hereby initiating journeys between scientific and artistic fields. This study proposes to problematize these journeys as an “echology”. Echology is a forgotten term first introduced in the ‘70s by Jean Milet in his thesis about the sociology of Gabriel Tarde. Here, the theories of other philosophers such as Georges Canguilhem, Michel Foucault, Gilles Deleuze, Félix Guattari, Marie-José Mondzain, Gilbert Simondon, and contemporary thinkers such as Thierry Bardini and Brian Massumi are mobilized in order to give this term its full consistency. According to echology, entities consist of patterns of interference and resonance with other things, which arise before their representation. Thus, a given entity is a complex of forces and its apparition the result of certain techniques which put it into relation with other complexes cannot be explained as an effect associated with a single cause but gives itself as an extra-effect or surplus that always leaves a remainder. From an echological perspective, a microbe- 7 artwork operates as an interface that incorporates potentials that make themselves visible through the remainders by virtue of multiple processes cutting across scientific activities and artistic strategies. Each chapter of the thesis is thus a way station in a conceptual journey of experimentation, revealing the dimensions of the artworks under consideration with respect to the analysis of these remainders. During this journey, elements of scientific theories, interviews with artists, field trips to sites of practice of the biological arts, related workshops and summer schools are mobilized as contributory factors of the construction of the problematic fields in each chapter. Microbes considered as objects of beauty hence appear as the result of discursive transformation of biological sciences. From earlier pathogenic conceptions of microbes to contemporary ecological approaches, the iconicity of microbes associated with microbe-images, echology of microbe-sounds, becoming-milieu of certain microbe-artworks, and finally, the question of individuation of thought and the correlated ethics understood as the problem of valuation of microbe-artworks, the becoming-microbe stems from this discursive transformation through the art field

SHELDON Smart habitat for the elderly.

An insightful document concerning active and assisted living under different perspectives: Furniture and habitat, ICT solutions and Healthcare

Advanced Augmentative and Alternative Communication System Based in Physiological Control

Dyskinetic Cerebral Palsy (DCP) is mainly characterized by alterations in muscle tone and involuntary movements. Therefore, these people present with difficulties in coordination and movement control, which makes walking difficult and affects their posture when seated. Additionally, their cognitive performance varies between being completely normal and severe mental retardation. People with DCP were selected as the objective of this thesis due to their multiple and complex limitations (speech problems and motor control) and because their capabilities have a great margin for improvement thanks to physiological control systems. Given their communication difficulties, some people with DCP have good motor con-trol and can communicate with written language. However, most have difficulty using Augmentative and Alternative Communication (AAC) systems. People with DCP gen-erally use concept boards to indicate the idea they want to communicate. However, most communication solutions available today are based on proprietary software that makes it difficult to customize the concept board and this type of control system. This is the motivation behind this thesis, with the aim of creating an interface with characteristics, able to be adapted to the user needs and limitations. Thus, this thesis proposes an Augmentative and Alternative Communication System for people with DCP based on physiological control. In addition, an innovative system for direct con-trol of concept boards with EMG is proposed. This control system is based on a physi-cal model that reproduces the muscular mechanical response (stiffness, inertia and viscosity). It allows for a selection of elements thanks to small pulses of EMG signal with sensors on a muscle with motor control. Its main advantage is the possibility of correcting errors during selection associated with uncontrolled muscle impulses, avoid-ing sustained muscle effort and thus reduced fatigue.La Parálisis Cerebral de tipo Discinésica (DCP) se caracteriza principalmente por las alteraciones del tono muscular y los movimientos involuntarios. Por ello, estos pacientes presentan dificultades en la coordinación y en el control de movimientos, lo cual les dificulta el caminar y afecta su postura cuando están sentados. Cabe resaltar que la capacidad cognitiva de las personas con DCP puede variar desde completamente normal, hasta un retraso mental severo. Las personas con DCP han sido seleccionadas como objetivo de esta tesis ya el margen de mejora de sus capacidades es amplio gracias a sistemas de control fisiológico, debido a sus múltiples y complejas limitaciones (problemas de habla y control motor). Debido a sus dificultades de comunicación, algunas personas con DCP se pueden comunicar con lenguaje escrito, siempre y cuando tenga un buen control motor. Sin embargo, la mayoría tienen dificultades para usar sistemas de Comunicación Aumentativos y Alternativos (AAC). De hecho, las personas con DCP utilizan generalmente tableros de conceptos para indicar la idea que quieren transmitir. Sin embargo, la mayoría las soluciones de comunicación disponibles en la actualidad están basadas en software propietario que hacen difícil la personalización del tablero de conceptos y el tipo de sistema de control. Es aquí donde surge esta tesis, con el objetivo de crear una interfaz con esas características, capaz de adaptarse a las necesidades y limitaciones del usuario. De esta forma, esta tesis propone un sistema de comunicación aumentativo y alternativo para personas con DCP basado en control fisiológico. Además, se propone un Sistema innovador de control directo sobre tableros de conceptos basado en EMG. Este Sistema de control se basa en un modelo físico que reproduce la respuesta mecánica muscular (basado en parámetros como Rigidez, Inercia y Viscosidad), permitiendo la selección de elementos gracias a pequeños pulsos de señal EMG con sensores sobre un músculo con control motor. Sus principales ventajas son la posibilidad de corregir errores durante la selección asociado a los impulsos musculares no controlados, evitar el esfuerzo muscular mantenido para alcanzar un nivel y reducir la fatiga.La Paràlisi Cerebral de tipus Discinèsica (DCP) es caracteritza principalment per les alteracions del to muscular i els moviments involuntaris. Per açò, aquests pacients presenten dificultats en la coordinació i en el control de moviments, la qual cosa els dificulta el caminar i afecta la seua postura quan estan asseguts. Cal ressaltar que la capacitat cognitiva de les persones amb DCP pot variar des de completament normal, fins a un retard mental sever. Les persones amb DCP han sigut seleccionades com a objectiu d'aquesta tesi ja el marge de millora de les seues capacitats és ampli gràcies a sistemes de control fisiològic, a causa dels seus múltiples i complexes limitacions (problemes de parla i control motor). A causa de les seues dificultats de comunicació, algunes persones amb DCP es poden comunicar amb llenguatge escrit, sempre que tinga un bon control motor. No obstant açò, la majoria tenen dificultats per a usar sistemes de Comunicació Augmentatius i Alternatius (AAC). De fet, les persones amb DCP utilitzen generalment taulers de conceptes per a indicar la idea que volen transmetre. No obstant açò, la majoria les solucions de comunicació disponibles en l'actualitat estan basades en programari propietari que fan difícil la personalització del tauler de conceptes i el tipus de sistema de control. És ací on sorgeix aquesta tesi, amb l'objectiu de crear una interfície amb aqueixes característiques, capaç d'adaptar-se a les necessitats i limitacions de l'usuari. D'aquesta forma, aquesta tesi proposa un sistema de comunicació augmentatiu i alternatiu per a persones amb DCP basat en control fisiològic. A més, es proposa un sistema innovador de control directe sobre taulers de conceptes basat en EMG. Aquest sistema de control es basa en un model físic que reprodueix la resposta mecànica muscular (basat en paràmetres com a Rigidesa, Inèrcia i Viscositat), permetent la selecció d'elements gràcies a xicotets polsos de senyal EMG amb sensors sobre un múscul amb control motor. Els seus principals avantatges són la possibilitat de corregir errors durant la selecció associat als impulsos musculars no controlats, evitar l'esforç muscular mantingut per a aconseguir un nivell i reduir la fatiga.Díaz Pineda, JA. (2017). Advanced Augmentative and Alternative Communication System Based in Physiological Control [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/90418TESI

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Situated memory for architectural space

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2003.Includes bibliographical references (p. 76-78).The immediacy and transportability of digital images has changed the spatial and temporal relationship between the viewer and the image. The malleability and large volume of these images affords us the ability to set up new such relationships. This thesis introduces a system that creates an asynchronous channel of connection and interaction by allowing two people or two groups of people to simultaneously inhabit a temporally neutral space. Construed as an elastic collective memory, the system intelligently documents audio and visual activities in a social space. This data is dynamically recomposed and manifested in the present as an interactive display environment that composites the past with the present, collapsing the temporal gap between them.by Parul Shailesh Vora.S.M