Predicting human behavior in smart environments: theory and application to gaze prediction

Predicting human behavior is desirable in many application scenarios in smart environments. The existing models for eye movements do not take contextual factors into account. This addressed in this thesis using a systematic machine-learning approach, where user profiles for eye movements behaviors are learned from data. In addition, a theoretical innovation is presented, which goes beyond pure data analysis. The thesis proposed the modeling of eye movements as a Markov Decision Processes. It uses Inverse Reinforcement Learning paradigm to infer the user eye movements behaviors

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion

Artificial Intelligence in the Creative Industries: A Review

This paper reviews the current state of the art in Artificial Intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically Machine Learning (ML) algorithms, is provided including Convolutional Neural Network (CNNs), Generative Adversarial Networks (GANs), Recurrent Neural Networks (RNNs) and Deep Reinforcement Learning (DRL). We categorise creative applications into five groups related to how AI technologies are used: i) content creation, ii) information analysis, iii) content enhancement and post production workflows, iv) information extraction and enhancement, and v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, machine learning-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of machine learning in domains with fewer constraints, where AI is the `creator', remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human centric -- where it is designed to augment, rather than replace, human creativity

Active Observers in a 3D World: Human Visual Behaviours for Active Vision

Human-like performance in computational vision systems is yet to be achieved. In fact, human-like visuospatial behaviours are not well understood a crucial capability for any robotic system whose role is to be a real assistant. This dissertation examines human visual behaviours involved in solving a well-known visual task; The Same-Different Task. It is used as a probe to explore the space of active human observation during visual problem-solving. It asks a simple question: are two objects the same?. To study this question, we created a set of novel objects with known complexity to push the boundaries of the human visual system. We wanted to examine these behaviours as opposed to the static, 2D, display-driven experiments done to date. We thus needed to develop a complete infrastructure for an experimental investigation using 3D objects and active, free, human observers. We have built a novel, psychophysical experimental setup that allows for precise and synchronized gaze and head-pose tracking to analyze subjects performing the task. To the best of our knowledge, no other system provides the same characteristics. We have collected detailed, first-of-its-kind data of humans performing a visuospatial task in hundreds of experiments. We present an in-depth analysis of different metrics of humans solving this task, who demonstrated up to 100% accuracy for specific settings and that no trial used less than six fixations. We provide a complexity analysis that reveals human performance in solving this task is about O(n), where n is the size of the object. Furthermore, we discovered that our subjects used many different visuospatial strategies and showed that they are deployed dynamically. Strikingly, no learning effect was observed that affected the accuracy. With this extensive and unique data set, we addressed its computational counterpart. We used reinforcement learning to learn the three-dimensional same-different task and discovered crucial limitations which only were overcome if the task was simplified to the point of trivialization. Lastly, we formalized a set of suggestions to inform the enhancement of existing machine learning methods based on our findings from the human experiments and multiple tests we performed with modern machine learning methods

Tactile Perception And Visuotactile Integration For Robotic Exploration

As the close perceptual sibling of vision, the sense of touch has historically received less than deserved attention in both human psychology and robotics. In robotics, this may be attributed to at least two reasons. First, it suffers from the vicious cycle of immature sensor technology, which causes industry demand to be low, and then there is even less incentive to make existing sensors in research labs easy to manufacture and marketable. Second, the situation stems from a fear of making contact with the environment, avoided in every way so that visually perceived states do not change before a carefully estimated and ballistically executed physical interaction. Fortunately, the latter viewpoint is starting to change. Work in interactive perception and contact-rich manipulation are on the rise. Good reasons are steering the manipulation and locomotion communities’ attention towards deliberate physical interaction with the environment prior to, during, and after a task. We approach the problem of perception prior to manipulation, using the sense of touch, for the purpose of understanding the surroundings of an autonomous robot. The overwhelming majority of work in perception for manipulation is based on vision. While vision is a fast and global modality, it is insufficient as the sole modality, especially in environments where the ambient light or the objects therein do not lend themselves to vision, such as in darkness, smoky or dusty rooms in search and rescue, underwater, transparent and reflective objects, and retrieving items inside a bag. Even in normal lighting conditions, during a manipulation task, the target object and fingers are usually occluded from view by the gripper. Moreover, vision-based grasp planners, typically trained in simulation, often make errors that cannot be foreseen until contact. As a step towards addressing these problems, we present first a global shape-based feature descriptor for object recognition using non-prehensile tactile probing alone. Then, we investigate in making the tactile modality, local and slow by nature, more efficient for the task by predicting the most cost-effective moves using active exploration. To combine the local and physical advantages of touch and the fast and global advantages of vision, we propose and evaluate a learning-based method for visuotactile integration for grasping

Active Observers in a 3D World: Human Visual Behaviours for Active Vision

Human-like performance in computational vision systems is yet to be achieved. In fact, human-like visuospatial behaviours are not well understood – a crucial capability for any robotic system whose role is to be a real assistant. This dissertation examines human visual behaviours involved in solving a well-known visual task; The Same-Different Task. It is used as a probe to explore the space of active human observation during visual problem-solving. It asks a simple question: “are two objects the same?”. To study this question, we created a set of novel objects with known complexity to push the boundaries of the human visual system. We wanted to examine these behaviours as opposed to the static, 2D, display-driven experiments done to date. We thus needed to develop a complete infrastructure for an experimental investigation using 3D objects and active, free, human observers. We have built a novel, psychophysical experimental setup that allows for precise and synchronized gaze and head-pose tracking to analyze subjects performing the task. To the best of our knowledge, no other system provides the same characteristics. We have collected detailed, first-of-its-kind data of humans performing a visuospatial task in hundreds of experiments. We present an in-depth analysis of different metrics of humans solving this task, who demonstrated up to 100% accuracy for specific settings and that no trial used less than six fixations. We provide a complexity analysis that reveals human performance in solving this task is about O(n), where n is the size of the object. Furthermore, we discovered that our subjects used many different visuospatial strategies and showed that they are deployed dynamically. Strikingly, no learning effect was observed that affected the accuracy. With this extensive and unique data set, we addressed its computational counterpart. We used reinforcement learning to learn the three-dimensional same-different task and discovered crucial limitations which only were overcome if the task was simplified to the point of trivialization. Lastly, we formalized a set of suggestions to inform the enhancement of existing machine learning methods based on our findings from the human experiments and multiple tests we performed with modern machine learning methods

Self-supervised learning techniques for monitoring industrial spaces

Dissertação de mestrado em Matemática e ComputaçãoEste documento é uma Dissertação de Mestrado com o título ”Self-Supervised Learning Techniques for Monitoring Industrial Spaces”e foi realizada e ambiente empresarial na empresa Neadvance - Machine Vision S.A. em conjunto com a Universidade do Minho. Esta dissertação surge de um grande projeto que consiste no desenvolvimento de uma plataforma de monitorização de operações específicas num espaço industrial, denominada SMARTICS (Plataforma tecnoló gica para monitorização inteligente de espaços industriais abertos). Este projeto continha uma componente de investigação para explorar um paradigma de aprendizagem diferente e os seus métodos - self-supervised learning, que foi o foco e principal contributo deste trabalho. O supervised learning atingiu um limite, pois exige anotações caras e dispendiosas. Em problemas reais, como em espaços industriais nem sempre é possível adquirir um grande número de imagens. O self-supervised learning ajuda nesses problemas, ex traindo informações dos próprios dados e alcançando bom desempenho em conjuntos de dados de grande escala. Este trabalho fornece uma revisão geral da literatura sobre a estrutura de self-supervised learning e alguns métodos. Também aplica um método para resolver uma tarefa de classificação para se assemelhar a um problema em um espaço industrial.This document is a Master’s Thesis with the title ”Self-Supervised Learning Techniques for Monitoring Industrial Spaces” and was carried out in a business environment at Neadvance - Machine Vision S.A. together with the University of Minho. This dissertation arises from a major project that consists of developing a platform to monitor specific operations in an industrial space, named SMARTICS (Plataforma tecnológica para monitorização inteligente de espaços industriais abertos). This project contained a research component to explore a different learning paradigm and its methods - self-supervised learning, which was the focus and main contribution of this work. Supervised learning has reached a bottleneck as they require expensive and time-consuming annotations. In real problems, such as in industrial spaces it is not always possible to require a large number of images. Self-supervised learning helps these issues by extracting information from the data itself and has achieved good performance in large-scale datasets. This work provides a comprehensive literature review of the self supervised learning framework and some methods. It also applies a method to solve a classification task to resemble a problem in an industrial space and evaluate its performance

Neural and Behavioral Mechanisms of Interval Timing in the Striatum

To guide behavior and learn from its consequences, the brain must represent time over many scales. Yet, the neural signals used to encode time in the seconds to minute range are not known. The striatum is the major input area of the basal ganglia; it plays important roles in learning, motor function and normal timing behavior in the range of seconds to minutes. We investigated how striatal population activity might encode time. To do so, we recorded the electrical activity from striatal neurons in rats performing the serial fixed interval task, a dynamic version of the fixed Interval schedule of reinforcement. The animals performed in conformity with proportional timing, but did not strictly conform to scalar timing predictions, which might reflect a parallel strategy to optimize the adaptation to changes in temporal contingencies and consequently to improve reward rate over the session. Regarding the neural activity, we found that neurons fired at delays spanning tens of seconds and that this pattern of responding reflected the interaction between time and the animals’ ongoing sensorimotor state. Surprisingly, cells rescaled responses in time when intervals changed, indicating that striatal populations encoded relative time. Moreover, time estimates decoded from activity predicted trial-bytrial timing behavior as animals adjusted to new intervals, and disrupting striatal function with local infusion of muscimol led to a decrease in timing performance. Because of practical limitations in testing for sufficiency a biological system, we ran a simple simulation of the task; we have shown that neural responses similar to those we observe are conceptually sufficient to produce temporally adaptive behavior. Furthermore, we attempted to explain temporal processes on the basis of ongoing behavior by decoding temporal estimates from high-speed videos of the animals performing the task; we could not explain the temporal report solely on basis of ongoing behavior. These results suggest that striatal activity forms a scalable population firing rate code for time, providing timing signals that animals use to guide their actions