GazeDPM: Early Integration of Gaze Information in Deformable Part Models

An increasing number of works explore collaborative human-computer systems in which human gaze is used to enhance computer vision systems. For object detection these efforts were so far restricted to late integration approaches that have inherent limitations, such as increased precision without increase in recall. We propose an early integration approach in a deformable part model, which constitutes a joint formulation over gaze and visual data. We show that our GazeDPM method improves over the state-of-the-art DPM baseline by 4% and a recent method for gaze-supported object detection by 3% on the public POET dataset. Our approach additionally provides introspection of the learnt models, can reveal salient image structures, and allows us to investigate the interplay between gaze attracting and repelling areas, the importance of view-specific models, as well as viewers' personal biases in gaze patterns. We finally study important practical aspects of our approach, such as the impact of using saliency maps instead of real fixations, the impact of the number of fixations, as well as robustness to gaze estimation error

Visual Decoding of Targets During Visual Search From Human Eye Fixations

What does human gaze reveal about a users' intents and to which extend can these intents be inferred or even visualized? Gaze was proposed as an implicit source of information to predict the target of visual search and, more recently, to predict the object class and attributes of the search target. In this work, we go one step further and investigate the feasibility of combining recent advances in encoding human gaze information using deep convolutional neural networks with the power of generative image models to visually decode, i.e. create a visual representation of, the search target. Such visual decoding is challenging for two reasons: 1) the search target only resides in the user's mind as a subjective visual pattern, and can most often not even be described verbally by the person, and 2) it is, as of yet, unclear if gaze fixations contain sufficient information for this task at all. We show, for the first time, that visual representations of search targets can indeed be decoded only from human gaze fixations. We propose to first encode fixations into a semantic representation and then decode this representation into an image. We evaluate our method on a recent gaze dataset of 14 participants searching for clothing in image collages and validate the model's predictions using two human studies. Our results show that 62% (Chance level = 10%) of the time users were able to select the categories of the decoded image right. In our second studies we show the importance of a local gaze encoding for decoding visual search targets of use

Appearance-Based Gaze Estimation in the Wild

Appearance-based gaze estimation is believed to work well in real-world settings, but existing datasets have been collected under controlled laboratory conditions and methods have been not evaluated across multiple datasets. In this work we study appearance-based gaze estimation in the wild. We present the MPIIGaze dataset that contains 213,659 images we collected from 15 participants during natural everyday laptop use over more than three months. Our dataset is significantly more variable than existing ones with respect to appearance and illumination. We also present a method for in-the-wild appearance-based gaze estimation using multimodal convolutional neural networks that significantly outperforms state-of-the art methods in the most challenging cross-dataset evaluation. We present an extensive evaluation of several state-of-the-art image-based gaze estimation algorithms on three current datasets, including our own. This evaluation provides clear insights and allows us to identify key research challenges of gaze estimation in the wild

Prediction of Search Targets From Fixations in Open-World Settings

Previous work on predicting the target of visual search from human fixations only considered closed-world settings in which training labels are available and predictions are performed for a known set of potential targets. In this work we go beyond the state of the art by studying search target prediction in an open-world setting in which we no longer assume that we have fixation data to train for the search targets. We present a dataset containing fixation data of 18 users searching for natural images from three image categories within synthesised image collages of about 80 images. In a closed-world baseline experiment we show that we can predict the correct target image out of a candidate set of five images. We then present a new problem formulation for search target prediction in the open-world setting that is based on learning compatibilities between fixations and potential targets

Contextual Media Retrieval Using Natural Language Queries

The widespread integration of cameras in hand-held and head-worn devices as well as the ability to share content online enables a large and diverse visual capture of the world that millions of users build up collectively every day. We envision these images as well as associated meta information, such as GPS coordinates and timestamps, to form a collective visual memory that can be queried while automatically taking the ever-changing context of mobile users into account. As a first step towards this vision, in this work we present Xplore-M-Ego: a novel media retrieval system that allows users to query a dynamic database of images and videos using spatio-temporal natural language queries. We evaluate our system using a new dataset of real user queries as well as through a usability study. One key finding is that there is a considerable amount of inter-user variability, for example in the resolution of spatial relations in natural language utterances. We show that our retrieval system can cope with this variability using personalisation through an online learning-based retrieval formulation.Comment: 8 pages, 9 figures, 1 tabl

Molecular gas in blue compact dwarf galaxies

Blue compact dwarf galaxies (BCDGs) are currently undergoing strong bursts of star formation. Nevertheless, only a few of them have been clearly detected in CO, which is thought to trace the "fuel" of star formation: H_2. In this paper, we present a deep search for CO J=1-->0 and J=2-->1 emission lines in a sample of 8 BCDGs and two companions. Only 2 of them (Haro 2 and UM 465) are detected. For the other galaxies we have obtained more stringent upper limits on the CO luminosity than published values. We could not confirm the previously reported ``detection'' of CO for the galaxies UM 456 and UM 462. We analyze a possible relation between metallicity, CO luminosity, and absolute blue magnitude of the galaxies. We use previously determined relations between X = N(H_2)/I_CO and the metallicity to derive molecular cloud masses or upper limits for them. With these ``global'' X_CO values we find that for those galaxies which we detect in CO, the molecular gas mass is similar to the HI mass, whereas for the non-detections, the upper limits on the molecular gas masses are significantly lower than the HI mass. Using an LVG (Large Velocity Gradient) model we show that X_CO depends not only on metallicity, but also on other physical parameters such as volume density and kinetic temperature, which rises the question on the validity of ``global'' X_CO factors.Comment: 9 pages, 6 figures, to be published on MNRA

The d-separation criterion in Categorical Probability

The d-separation criterion detects the compatibility of a joint probability distribution with a directed acyclic graph through certain conditional independences. In this work, we study this problem in the context of categorical probability theory by introducing a categorical definition of causal models, a categorical notion of d-separation, and proving an abstract version of the d-separation criterion. This approach has two main benefits. First, categorical d-separation is a very intuitive criterion based on topological connectedness. Second, our results apply in measure-theoretic probability (with standard Borel spaces), and therefore provide a clean proof of the equivalence of local and global Markov properties with causal compatibility for continuous and mixed variables.Comment: 34 page

Interaction between the Arabidopsis thaliana heat shock transcription factor HSF1 and the TATA binding protein TBP

AbstractThe heat shock factor (HSF1) is the central regulator of the heat stress (hs) response and is required for stimulating the transcription of the hs genes and consequently the expression of heat shock proteins. To promote the polymerase II-dependent transcription of the hs genes, HSF has to communicate with the basal transcription machinery. Here, we report that the Arabidopsis thaliana HSF1 interacts directly with TBP, the general TATA box binding transcription factor, as shown by affinity chromatography and electrophoretic mobility shift analyses in vitro. An in vivo interaction between AtHSF1 and AtTBP1 was suggested by results employing the yeast two-hybrid system

Contributions to the evidence base for reducing the impact of influenza in primary care

Influenza can have a significant impact on both individual and societal level. This can be reduced by vaccination and usage of point-of-care tests in specific settings like the Emergency Department. De general physician plays a key role in implementing influenza vaccination. The attitude of the Dutch general physician towards influenza vaccination is generally in favour of vaccination, which is reflected by a majority of the GPs that is vaccinated against the flu and advises practice personnel to get vaccinated. This thesis found indications of a possible relevant effect of influenza vaccination on mortality in the younger elderly. Also, after extensive ethical and methodological considerations, it appears unlikely that a randomized controlled trial evaluating the long-term effect of influenza vaccination on mortality will ever be conducted. This thesis stresses the potency of long-term follow-up of intervention studies and the importance of considering ethical and methodological aspects of research when it comes to developing new influenza vaccination trials