Attentive monitoring of multiple video streams driven by a Bayesian foraging strategy

In this paper we shall consider the problem of deploying attention to subsets of the video streams for collating the most relevant data and information of interest related to a given task. We formalize this monitoring problem as a foraging problem. We propose a probabilistic framework to model observer's attentive behavior as the behavior of a forager. The forager, moment to moment, focuses its attention on the most informative stream/camera, detects interesting objects or activities, or switches to a more profitable stream. The approach proposed here is suitable to be exploited for multi-stream video summarization. Meanwhile, it can serve as a preliminary step for more sophisticated video surveillance, e.g. activity and behavior analysis. Experimental results achieved on the UCR Videoweb Activities Dataset, a publicly available dataset, are presented to illustrate the utility of the proposed technique.Comment: Accepted to IEEE Transactions on Image Processin

Take another little piece of my heart: a note on bridging cognition and emotions

Science urges philosophy to be more empirical and philosophy urges science to be more reflective. This markedly occurred along the “discovery of the artificial” (CORDESCHI 2002): in the early days of Cybernetics and Artificial Intelligence (AI) researchers aimed at making machines more cognizant while setting up a framework to better understand human intelligence. By and large, those genuine goals still hold today, whereas AI has become more concerned with specific aspects of intelligence, such as (machine) learning, reasoning, vision, and action. As a matter of fact, the field suffers from a chasm between two formerly integrated aspects. One is the engineering endeavour involving the development of tools, e.g., autonomous systems for driving cars as well as software for semantic information retrieval. The other is the philosophical debate that tries to answer questions concerning the nature of intelligence. Bridging these two levels can indeed be crucial in developing a deeper understanding of minds. An opportunity might be offered by the cogent theme of emotions. Traditionally, computer science, psychological and philosophical research have been compelled to investigate mental processes that do not involve mood, emotions and feelings, in spite of Simon’s early caveat (SIMON 1967) that a general theory of cognition must incorporate the influences of emotion. Given recent neurobiological findings and technological advances, the time is ripe to seriously weigh this promising, albeit controversial, opportunity

Problems with Saliency Maps

Despite the popularity that saliency models have gained in the computer vision community, they are most often conceived, exploited and benchmarked without taking heed of a number of problems and subtle issues they bring about. When saliency maps are used as proxies for the likelihood of fixating a location in a viewed scene, one such issue is the temporal dimension of visual attention deployment. Through a simple simulation it is shown how neglecting this dimension leads to results that at best cast shadows on the predictive performance of a model and its assessment via benchmarking procedures

How to look next? A data-driven approach for scanpath prediction

By and large, current visual attention models mostly rely, when considering static stimuli, on the following procedure. Given an image, a saliency map is computed, which, in turn, might serve the purpose of predicting a sequence of gaze shifts, namely a scanpath instantiating the dynamics of visual attention deployment. The temporal pattern of attention unfolding is thus confined to the scanpath generation stage, whilst salience is conceived as a static map, at best conflating a number of factors (bottom-up information, top-down, spatial biases, etc.). In this note we propose a novel sequential scheme that consists of a three-stage processing relying on a center-bias model, a context/layout model, and an object-based model, respectively. Each stage contributes, at different times, to the sequential sampling of the final scanpath. We compare the method against classic scanpath generation that exploits state-of-the-art static saliency model. Results show that accounting for the structure of the temporal unfolding leads to gaze dynamics close to human gaze behaviour

Backwards is the way forward: feedback in the cortical hierarchy predicts the expected future

Clark offers a powerful description of the brain as a prediction machine, which offers progress on two distinct levels. First, on an abstract conceptual level, it provides a unifying framework for perception, action, and cognition (including subdivisions such as attention, expectation, and imagination). Second, hierarchical prediction offers progress on a concrete descriptive level for testing and constraining conceptual elements and mechanisms of predictive coding models (estimation of predictions, prediction errors, and internal models)

The application of ocean front metrics for understanding habitat selection by marine predators

Marine predators such as seabirds, cetaceans, turtles, pinnipeds, sharks and large teleost fish are essential components of healthy, biologically diverse marine ecosystems. However, intense anthropogenic pressure on the global ocean is causing rapid and widespread change, and many predator populations are in decline. Conservation solutions are urgently required, yet only recently have we begun to comprehend how these animals interact with the vast and dynamic oceans that they inhabit. A better understanding of the mechanisms that underlie habitat selection at sea is critical to our knowledge of marine ecosystem functioning, and to ecologically-sensitive marine spatial planning. The collection of studies presented in this thesis aims to elucidate the influence of biophysical coupling at oceanographic fronts – physical interfaces at the transitions between water masses – on habitat selection by marine predators. High-resolution composite front mapping via Earth Observation remote sensing is used to provide oceanographic context to several biologging datasets describing the movements and behaviours of animals at sea. A series of species-habitat models reveal the influence of mesoscale (10s to 100s of kilometres) thermal and chlorophyll-a fronts on habitat selection by taxonomically diverse species inhabiting contrasting ocean regions; northern gannets (Morus bassanus; Celtic Sea), basking sharks (Cetorhinus maximus; north-east Atlantic), loggerhead turtles (Caretta caretta; Canary Current), and grey-headed albatrosses (Thalassarche chrysostoma; Southern Ocean). Original aspects of this work include an exploration of quantitative approaches to understanding habitat selection using remotely-sensed front metrics; and explicit investigation of how the biophysical properties of fronts and species-specific foraging ecology interact to influence associations. Main findings indicate that front metrics, particularly seasonal indices, are useful predictors of habitat preference across taxa. Moreover, frontal persistence and spatiotemporal predictability appear to mediate the use of front-associated foraging habitats, both in shelf seas and in the open oceans. These findings have implications for marine spatial planning and the design of protected area networks, and may prove useful in the development of tools supporting spatially dynamic ocean management

The role of phonology in visual word recognition: evidence from Chinese

Posters - Letter/Word Processing V: abstract no. 5024The hypothesis of bidirectional coupling of orthography and phonology predicts that phonology plays a role in visual word recognition, as observed in the effects of feedforward and feedback spelling to sound consistency on lexical decision. However, because orthography and phonology are closely related in alphabetic languages (homophones in alphabetic languages are usually orthographically similar), it is difficult to exclude an influence of orthography on phonological effects in visual word recognition. Chinese languages contain many written homophones that are orthographically dissimilar, allowing a test of the claim that phonological effects can be independent of orthographic similarity. We report a study of visual word recognition in Chinese based on a mega-analysis of lexical decision performance with 500 characters. The results from multiple regression analyses, after controlling for orthographic frequency, stroke number, and radical frequency, showed main effects of feedforward and feedback consistency, as well as interactions between these variables and phonological frequency and number of homophones. Implications of these results for resonance models of visual word recognition are discussed.postprin

Interactive effects of orthography and semantics in Chinese picture naming

Posters - Language Production/Writing: abstract no. 4035Picture-naming performance in English and Dutch is enhanced by presentation of a word that is similar in form to the picture name. However, it is unclear whether facilitation has an orthographic or a phonological locus. We investigated the loci of the facilitation effect in Cantonese Chinese speakers by manipulating—at three SOAs (2100, 0, and 1100 msec)—semantic, orthographic, and phonological similarity. We identified an effect of orthographic facilitation that was independent of and larger than phonological facilitation across all SOAs. Semantic interference was also found at SOAs of 2100 and 0 msec. Critically, an interaction of semantics and orthography was observed at an SOA of 1100 msec. This interaction suggests that independent effects of orthographic facilitation on picture naming are located either at the level of semantic processing or at the lemma level and are not due to the activation of picture name segments at the level of phonological retrieval.postprin