Information-theoretic Sensorimotor Foundations of Fitts' Law

© 2019 ACM. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published is accessible via https://doi.org/10.1145/3290607.3313053We propose a novel, biologically plausible cost/fitness function for sensorimotor control, formalized with the information-theoretic principle of empowerment, a task-independent universal utility. Empowerment captures uncertainty in the perception-action loop of different nature (e.g. noise, delays, etc.) in a single quantity. We present the formalism in a Fitts' law type goal-directed arm movement task and suggest that empowerment is one potential underlying determinant of movement trajectory planning in the presence of signal-dependent sensorimotor noise. Simulation results demonstrate the temporal relation of empowerment and various plausible control strategies for this specific task

Information parsimony in collaborative interaction

We investigate the information processing cost associated with performing a collaborative dyadic task at a specific utility level. We build our approach on the Relevant Information formalism, which combines Shannon's Information Theory and Markov Decision Processes, for modelling a dyadic interaction scenario in which two agents with independent controllers move an object together with fully redundant control. Results show that increasing dyad's collaboration decreases the information intake and vice versa, antagonistic behavior puts a strain on the information bandwidth capacity. The key role of the particular embodiment of the environment in this trade-off is demonstrated in a series of simulations with informationally parsimonious optimal controllers.Peer reviewedFinal Published versio

Empowerment as a metric for Optimization in HCI

We propose a novel metric for optimizing human-computer interfaces, based on the information-theoretic capacity of empowerment, a task-independent universal utility measure. Empowerment measures, for agent-environment systems with stochastic transitions, how much influence, which can be sensed by the agent sensors, an agent has on its environment. It captures the uncertainty in human-machine systems arising from different sources (i.e. noise, delays, errors, etc.) as a single quantity. We suggest the potential empowerment has as an objective optimality criterion in user interface design optimization, contributing to the more solid theoretical foundations of HCI.Peer reviewedFinal Accepted Versio

Model of Coordination Flow in Remote Collaborative Interaction

© 2015 IEEEWe present an information-theoretic approach for modelling coordination in human-human interaction and measuring coordination flows in a remote collaborative tracking task. Building on Shannon's mutual information, coordination flow measures, for stochastic collaborative systems, how much influence, the environment has on the joint control of collaborating parties. We demonstrate the application of the approach on interactive human data recorded in a user study and reveal the amount of effort required for creating rigorous models. Our initial results suggest the potential coordination flow has - as an objective, task-independent measure - in supporting designers of human collaborative systems and in providing better theoretical foundations for the science of Human-Computer Interaction

An information-theoretic account of human–computer interaction

This thesis presents a theoretical framework for the study of interactive systems, using methods from information theory, machine learning and control theory. The framework builds on the information-theoretic capacities of empowerment, relevant information and mutual information, which I adapt and apply to the domain of human-computer interaction. Three user studies exploring dynamic interactive scenarios - one car-tracking and two collaborative target-acquisition experiments - provide empirical data for the development of probabilistic models, used in the characterisation of specific aspects of human performance, such as the level of control, the quality of decision-making, and the level of engagement in interpersonal coordination. Human control models are extended to accommodate for the inherent lags, characteristic for human-computer and human-human interaction, in a principled way. Optimal controllers, describing particular patterns of human behaviour, are built on these theoretical models, providing evidence for specific limits of human performance through simulations. The thesis describes the potential of empowerment, as a generic task-independent measure of control, to characterise the uncertainty in human-machine interfaces. This work builds an important bridge between theory and experiments, and suggests that the proposed information-theoretic concepts could provide analytical tools for supporting the design and evaluation of interactive systems, by elucidating novel aspects of human performance complementing standard measures. The thesis provides proof of concept examples for the application of such information-theoretic measures, and demonstrates how they can be treated naturally side-by-side along traditional metrics used in HCI research. It emphasises the acquisition cost of accurate theoretical models, necessary to ensure the reliability of such measures

Interpersonal sensorimotor contingencies: Information-theoretic relevance of subjective experience

Peer reviewedFinal Published versio

A visual attention mechanism for autonomous robots controlled by sensorimotor contingencies

Alexander Maye, Dari Trendafilov, Daniel Polani, Andreas Engel, ‘A visual attention mechanism for autonomous robots controlled by sensorimotor contingencies’, paper presented at the International Conference on Intelligent Robots and Systems (IROS) 2015 Workshop on Sensorimotor Contingencies for Robotics, Hamburg, Germany, 2 October, 2015.Robot control architectures that are based on learning the dependencies between robot's actions and the resulting change in sensory input face the fundamental problem that for high-dimensional action and/or sensor spaces, the number of these sensorimotor dependencies can become huge. In this article we present a scenario of a robot that learns to avoid collisions with stationary objects from image-based motion flow and a collision detector. Following an information-theoretic approach, we demonstrate that the robot can infer image regions that facilitate the prediction of imminent collisions. This allows restricting the computation to the domain in the input space that is relevant for the given task, which enables learning sensorimotor contingencies in robots with high-dimensional sensor spaces.Peer reviewedFinal Accepted Versio

The role of the environment in collective perception:A generic complexity measure

We propose a novel generic information-theoretic framework for characterizing the task difficulty in the Collective Perception paradigm. Our formalism builds on the notion of Empowerment - a task-independent, universal and generic utility function, which characterizes the level of perceivable control an embodied agent has over its environment. Series of simulations with an empowerment model of the collective perception scenario revealed a significant correlation between the levels of empowerment and the accuracy demonstrated by a set of standard collective decision-making strategies and a recent state-of-the-art neural network controller on nine benchmark patterns, used previously for assessing swarm performance. The results elucidate the key role of both the agent embodiment and the environmental pattern in characterising task difficulty, and justify the application of empowerment to analytically assess this role, which could help predict swarm performance and support the development of more efficient decision-making strategies

Cross-Entropy Regularization with Mutual Information in Training CNNs

We examine the learning behavior of a shallow and a deep convolutional neural network performing classification tasks on subsets of two databases. Our investigation focuses on the label, the input, and the prediction layer, and we compute the mutual information between these layers epoch-wise using Rényi’s matrix-based entropy functional. We evaluate the data processing inequality to interpret the learning behavior in a consistent information-theoretic framework. Our primary goals are to 1) clarify the relation between the two training objectives of minimizing the cross-entropy and maximizing the mutual information between the label and the prediction layer, 2) gradually switch from the first to the second training objective, and 3) interpret the impact of the latter transition. One of the main contributions is the proposed novel method for regularizing the cross-entropy objective and assessing the neural network’s learning activity