Information dynamics: patterns of expectation and surprise in the perception of music

This is a postprint of an article submitted for consideration in Connection Science © 2009 [copyright Taylor & Francis]; Connection Science is available online at:http://www.tandfonline.com/openurl?genre=article&issn=0954-0091&volume=21&issue=2-3&spage=8

Learning the Preferences of Ignorant, Inconsistent Agents

An important use of machine learning is to learn what people value. What posts or photos should a user be shown? Which jobs or activities would a person find rewarding? In each case, observations of people's past choices can inform our inferences about their likes and preferences. If we assume that choices are approximately optimal according to some utility function, we can treat preference inference as Bayesian inverse planning. That is, given a prior on utility functions and some observed choices, we invert an optimal decision-making process to infer a posterior distribution on utility functions. However, people often deviate from approximate optimality. They have false beliefs, their planning is sub-optimal, and their choices may be temporally inconsistent due to hyperbolic discounting and other biases. We demonstrate how to incorporate these deviations into algorithms for preference inference by constructing generative models of planning for agents who are subject to false beliefs and time inconsistency. We explore the inferences these models make about preferences, beliefs, and biases. We present a behavioral experiment in which human subjects perform preference inference given the same observations of choices as our model. Results show that human subjects (like our model) explain choices in terms of systematic deviations from optimal behavior and suggest that they take such deviations into account when inferring preferences.Comment: AAAI 201

Nominalist Heuristics and Economic Theory

This paper introduces a new theoretic entity, a nominalist heuristic, defined as a focus on prominent numbers, indices or ratios. Abstractions used in the evaluation stage of decision making typically involve nominalist heuristics that are incompatible with expected utility theory which excludes the evaluation stage, and are also incompatible with prospect theory which assumes that, while the evaluation procedure can involve systematic mistakes, the overall decision situation is nevertheless sufficiently simple: 1) for economists and psychologists to identify what is a mistake, and 2) to be compatible with maximisation. But in the typical complex situation giving rise to nominalist heuristics neither 1) nor 2) hold, and therefore what is required is a fundamentally different class of models that allow for the progressive anticipated changes in knowledge ahead faced under risk and uncertainty, namely models under the umbrella of SKAT, the Stages of Knowledge Ahead Theory. A sequel paper. Pope et al 2009b, shows field and laboratory evidence of heuristics in the form of prominent numbers entering exchange rate determination.nominalism, money illusion, heuristic, unpredictability, experiment, SKAT the Stages of Knowledge Ahead Theory, prominent numbers, prominent indices, prominent ratios, equality, historical benchmarks, complexity, decision costs, evaluation

An original framework for understanding human actions and body language by using deep neural networks

The evolution of both fields of Computer Vision (CV) and Artificial Neural Networks (ANNs) has allowed the development of efficient automatic systems for the analysis of people's behaviour. By studying hand movements it is possible to recognize gestures, often used by people to communicate information in a non-verbal way. These gestures can also be used to control or interact with devices without physically touching them. In particular, sign language and semaphoric hand gestures are the two foremost areas of interest due to their importance in Human-Human Communication (HHC) and Human-Computer Interaction (HCI), respectively. While the processing of body movements play a key role in the action recognition and affective computing fields. The former is essential to understand how people act in an environment, while the latter tries to interpret people's emotions based on their poses and movements; both are essential tasks in many computer vision applications, including event recognition, and video surveillance. In this Ph.D. thesis, an original framework for understanding Actions and body language is presented. The framework is composed of three main modules: in the first one, a Long Short Term Memory Recurrent Neural Networks (LSTM-RNNs) based method for the Recognition of Sign Language and Semaphoric Hand Gestures is proposed; the second module presents a solution based on 2D skeleton and two-branch stacked LSTM-RNNs for action recognition in video sequences; finally, in the last module, a solution for basic non-acted emotion recognition by using 3D skeleton and Deep Neural Networks (DNNs) is provided. The performances of RNN-LSTMs are explored in depth, due to their ability to model the long term contextual information of temporal sequences, making them suitable for analysing body movements. All the modules were tested by using challenging datasets, well known in the state of the art, showing remarkable results compared to the current literature methods

Cortico-hippocampal activations for high entropy visual stimulus: an fMRI perspective

We perceive the environment around us in order to act upon it. To gain the desirable outcome effectively, we not only need the incoming information to be processed efficiently but we also need to know how reliable this information is. How this uncertainty is extracted from the visual input and how is it represented in the brain are still open questions. The hippocampus reacts to different measures of uncertainty. Because it is strongly connected to different cortical and subcortical regions, the hippocampus has the resources to communicate such information to other brain regions involved in visual processing and other cognitive processes. In this thesis, we investigate the aspects of uncertainty to which the hippocampus reacts. Is it the uncertainty in the ongoing recognition attempt of a temporally unfolding stimulus or is it the low-level spatiotemporal entropy? To answer this question, we used a dynamic visual stimulus with varying spatial and spatiotemporal entropy. We used well-structured virtual tunnel videos and the corresponding phase-scrambled videos with matching local luminance and contrast per frame. We also included pixel scrambled videos with high spatial and spatiotemporal entropy in our stimulus set. Brain responses (fMRI images) from the participants were recorded while they watched these videos and performed an engaging but cognitively independent task. Using the General Linear Model (GLM), we modeled the brain responses corresponding to different video types and found that the early visual cortex and the hippocampus had a stronger response to videos with higher spatiotemporal entropy. Using independent component analysis, we further investigated which underlying networks were recruited in processing high entropy visual information. We also discovered how these networks might influence each other. We found two cortico-hippocampal networks involved in processing our stimulus videos. While one of them represented a general primary visual processing network, the other was activated strongly by the high entropy videos and deactivated by the well-structured virtual tunnel videos. We also found a hierarchy in the processing stream with information flowing from less stimulus-specific to more stimulus-specific networks

Coherent Multi-Sentence Video Description with Variable Level of Detail

Humans can easily describe what they see in a coherent way and at varying level of detail. However, existing approaches for automatic video description are mainly focused on single sentence generation and produce descriptions at a fixed level of detail. In this paper, we address both of these limitations: for a variable level of detail we produce coherent multi-sentence descriptions of complex videos. We follow a two-step approach where we first learn to predict a semantic representation (SR) from video and then generate natural language descriptions from the SR. To produce consistent multi-sentence descriptions, we model across-sentence consistency at the level of the SR by enforcing a consistent topic. We also contribute both to the visual recognition of objects proposing a hand-centric approach as well as to the robust generation of sentences using a word lattice. Human judges rate our multi-sentence descriptions as more readable, correct, and relevant than related work. To understand the difference between more detailed and shorter descriptions, we collect and analyze a video description corpus of three levels of detail.Comment: 10 page