Measuring gaze and pupil in the real world: object-based attention,3D eye tracking and applications

This dissertation contains studies on visual attention, as measured by gaze orientation, and the use of mobile eye-tracking and pupillometry in applications. It combines the development of methods for mobile eye-tracking (studies II and III) with experimental studies on gaze guidance and pupillary responses in patients (studies IV and VI) and healthy observers (studies I and V). Object based attention / Study I What is the main factor of fixation guidance in natural scenes? Low-level features or objects? We developed a fixation-predicting model, which regards preferred viewing locations (PVL) per object and combines these distributions over the entirety of existing objects in the scene. Object-based fixation predictions for natural scene viewing perform at par with the best early salience model, that are based on low-level features. However, when stimuli are manipulated so that low-level features and objects are dissociated, the greater prediction power of saliency models diminishes. Thus, we dare to claim, that highly developed saliency models implicitly obtain object-hood and that fixation selection is mainly influenced by objects and much less by low-level features. Consequently, attention guidance in natural scenes is object-based. 3D tracking / Study II The second study focussed on improving calibration procedures for eye-in-head positions with a mobile eye-tracker.We used a mobile eye-tracker prototype, the EyeSeeCam with a high video-oculography (VOG) sampling rate and the technical gadget to follow the users gaze direction instantaneously with a rotatable camera. For a better accuracy in eye-positioning, we explored a refinement in the implementation of the eye-in-head calibration that yields a measure for fixation distance, which led to a mobile eye-tracker 3D calibration. Additionally, by developing the analytical mechanics for parametrically reorienting the gaze-centred camera, the 3D calibration could be applied to reliably record gaze-centred videos. Such videos are suitable as stimuli for investigating gaze-behaviour during object manipulation or object recognition in real worlds point-of-view (PoV) perspective. In fact, the 3D calibration produces a higher accuracy in positioning the gaze-centred camera over the whole 3D visual range. Study III, eye-tracking methods With a further development on the EyeSeeCam we achieved to record gaze-in-world data, by superposing eye-in-head and head-in-world coordinates. This novel approach uses a combination of few absolute head-positions extracted manually from the PoV video and of relative head-shifts integrated over angular velocities and translational accelerations, both given by an inertia measurement unit (IMU) synchronized to the VOG data. Gaze-in-world data consist of room-referenced gaze directions and their origins within the environment. They easily allow to assign fixation targets by using a 3D model of the measuring environment – a strong rationalisation regarding fixation analysis. Applications Study III Daylight is an important perceptual factor for visual comfort, but can also create discomfort glare situations during office work, so we developed to measure its behavioural influences. We achieve to compare luminance distributions and fixations in a real-world setting, by also recording indoor luminance variations time-resolved using luminance maps of a scenery spanning over a 3pi sr. Luminance evaluations in the workplace environment yield a well controlled categorisation of different lighting conditions and a localisation as well as a brightness measure of glare sources.We used common tasks like reading, typing on a computer, a phone call and thinking about a subject. The 3D model gives the possibility to test for gaze distribution shifts in the presence of glare patches and for variations between lighting conditions. Here, a low contrast lighting condition with no sun inside and a high contrast lighting condition with direct sunlight inside were compared. When the participants are not engaged in any visually focused task and the presence of the task support is minimal, the dominant view directions are inclined towards the view outside the window under the low contrast lighting conditions, but this tendency is less apparent and sways more towards the inside of the room under the high contrast lighting condition. This result implicates an avoidance of glare sources in gaze behaviour. In a second more extensive series of experiments, the participants’ subjective assessments of the lighting conditions will be included. Thus, the influence of glare can be analysed in more detail and tested whether visual discomfort judgements are correlated in differences in gaze-behaviour. Study IV The advanced eye-tracker calibration found application in several following projects and included in this dissertation is an investigation with patients suffering either from idiopathic Parkinson’s disease or from progressive supranuclear palsy (PSP) syndrome. PSP’s key symptom is the decreased ability to carry out vertical saccades and thus the main diagnostic feature for differentiating between the two forms of Parkinson’s syndrome. By measuring ocular movements during a rapid (< 20s) procedure with a standardized fixation protocol, we could successfully differentiate pre-diagnosed patients between idiopathic Parkinson’s disease and PSP, thus between PSP patients and HCs too. In PSP patients, the EyeSeeCam detected prominent impairment of both saccade velocity and amplitude. Furthermore, we show the benefits of a mobile eye-tracking device for application in clinical practice. Study V Decision-making is one of the basic cognitive processes of human behaviours and thus, also evokes a pupil dilation. Since this dilation reflects a marker for the temporal occurrence of the decision, we wondered whether individuals can read decisions from another’s pupil and thus become a mentalist. For this purpose, a modified version of the rock-paper-scissors childhood game was played with 3 prototypical opponents, while their eyes were video taped. These videos served as stimuli for further persons, who competed in rock-paper-scissors. Our results show, that reading decisions from a competitor’s pupil can be achieved and players can raise their winning probability significantly above chance. This ability does not require training but the instruction, that the time of maximum pupil dilation was indicative of the opponent’s choice. Therefore we conclude, that people could use the pupil to detect cognitive decisions in another individual, if they get explicit knowledge of the pupil’s utility. Study VI For patients with severe motor disabilities, a robust mean of communication is a crucial factor for well-being. Locked-in-Syndrome (LiS) patients suffer from quadriplegia and lack the ability of articulating their voice, though their consciousness is fully intact. While classic and incomplete LiS allows at least voluntary vertical eye movements or blinks to be used for communication, total LiS patients are not able to perform such movements. What remains, are involuntarily evoked muscle reactions, like it is the case with the pupillary response. The pupil dilation reflects enhanced cognitive or emotional processing, which we successfully observed in LiS patients. Furthermore, we created a communication system based on yes-no questions combined with the task of solving arithmetic problems during matching answer intervals, that yet invokes the most solid pupil dilation usable on a trial-by-trial basis for decoding yes or no as answers. Applied to HCs and patients with various severe motor disabilities, we provide the proof of principle that pupil responses allow communication for all tested HCs and 4/7 typical LiS patients. Résumé Together, the methods established within this thesis are promising advances in measuring visual attention allocation with 3D eye-tracking in real world and in the use of pupillometry as on-line measurement of cognitive processes. The two most outstanding findings are the possibility to communicate with complete LiS patients and further a conclusive evidence that objects are the primary unit of fixation selection in natural scenes

Validation of mobile eye-tracking as novel and efficient means for differentiating progressive supranuclear palsy from Parkinson's disease

Background: The decreased ability to carry out vertical saccades is a key symptom of Progressive Supranuclear Palsy (PSP). Objective measurement devices can help to reliably detect subtle eye movement disturbances to improve sensitivity and specificity of the clinical diagnosis. The present study aims at transferring findings from restricted stationary video-oculography (VOG) to a wearable head-mounted device, which can be readily applied in clinical practice. Methods: We investigated the eye movements in 10 possible or probable PSP patients, 11 Parkinson's disease (PD) patients, and 10 age-matched healthy controls (HCs) using a mobile, gaze-driven video camera setup (EyeSeeCam). Ocular movements were analyzed during a standardized fixation protocol and in an unrestricted real-life scenario while walking along a corridor. Results: The EyeSeeCam detected prominent impairment of both saccade velocity and amplitude in PSP patients, differentiating them from PD and HCs. Differences were particularly evident for saccades in the vertical plane, and stronger for saccades than for other eye movements. Differences were more pronounced during the standardized protocol than in the real-life scenario. Conclusions: Combined analysis of saccade velocity and saccade amplitude during the fixation protocol with the EyeSeeCam provides a simple, rapid (<20 s), and reliable tool to differentiate clinically established PSP patients from PD and HCs. As such, our findings prepare the ground for using wearable eye-tracking in patients with uncertain diagnoses

Gaze-Driven approach for estimating luminance values in the field of view for discomfort assessments

A gaze-driven methodology for discomfort glare was developed and applied for glare evaluation. A series of user assessments were performed in an office-like test laboratory under various lighting conditions. The participants’ gaze responses were recorded by means of mobile eye tracking while monitoring photometric quantities relevant to visual comfort using HDR luminance imaging. The integration of the luminance images coupled with eye-tracking methods enabled us to use gaze-centred luminance distribution to have an accurate estimate of the light received at the eye. Using a novel gaze-driven approach, a unique database was created as a basis to investigate the gaze direction dependencies of visual comfort. Here we compare the proposed gaze-driven approach with two other approaches based on fixed-gaze assumptions: gaze fixating on the task area, and gaze shifted 45 ° towards the window area. The results show that there is a significant difference between luminance distributions driven by gaze and those based on fixed-gaze assumptions, indicating a potentially important impact on glare assessment results as well

Uncovering relationships between view direction patterns and glare perception in a daylit workspace

This paper presents the results of an experimental study that aims to provide objective insights as to how luminance distribution in an office setting modulates our view direction (VD) in a daylit workspace while performing office tasks. Using the office-like test facility at Fraunhofer ISE (Freiburg, Germany) to create a range of controlled daylighting conditions, and a wearable mobile eye-tracker to measure eye and head orientation, we assessed VD distributions for subjects performing a standardized sequence of typical office tasks relative to two different daylight conditions: low contrast condition with no direct sunlight as compared to high contrast condition with direct sunlight coming into the room. Our results show that while the participants look more outside the window during a non-cognitive and non-visual office task, this effect is lower under the high contrast lighting conditions. Moreover, the focus of the VDs is on the task area when the participants are performing a task involving visual and cognitive activities

Investigation of gaze patterns in daylit workplaces: using eye-tracking methods to objectify view direction as a function of lighting conditions

Despite numerous efforts in developing glare indices through human assessment studies, predicting visual comfort in indoor environments still poses important challenges in design. A major limitation in discomfort glare indices is that they all ignore its dependencies on view direction. In this study we adopted eye-tracking methods in a series of human assessment experiments in order to record actual visual response when experiencing discomfort glare. We set up an experiment where the view directions distributions were monitored as the participants were working in a side-lit office with three different task-supports - monitor, paper and phone - on a standardized office task sequence. The participants were allocated randomly to two groups where they were exposed to two different views from the window. The results show that the “view outside the window” is the main determinant of view direction bias whenever the participant is not focused on any cognitive or visual office task procedure

Combining wearable eye-tracking with 4π light-field measurements: towards controlling all bottom-up and top-down factors driving overt attention during real-world tasks

For improvement of office space design, we intend to capture the full (4π) light-field of an office space, while measuring gaze, head direction, body position, blink rate, and pupil size along with task performance and subjective well-being during a variety of office tasks. Besides the immediate application aspects this will allow for the first time to have full control over task and visual input in a fully unconstrained real-world setting. In the study reported here, 52 participants performed office tasks that varied in the tools used (phone, computer, paper ) as well as in their mental load – input, output, reflection and interaction – and were recorded under various experimentally controlled lighting conditions and outside views. We analyze gaze allocation during these tasks, with a particular emphasis on the distinct roles of eye and head, as well as on the effects of discomfort glare

Estimating Trends in the Proportion of Transmitted and Acquired HIV Drug Resistance in a Long Term Observational Cohort in Germany

Objective: We assessed trends in the proportion of transmitted (TDR) and acquired (ADR) HIV drug resistance and associated mutations between 2001 and 2011 in the German ClinSurv-HIV Drug Resistance Study. Method: The German ClinSurv-HIV Drug Resistance Study is a subset of the German ClinSurv-HIV Cohort. For the ClinSurv-HIV Drug Resistance Study all available sequences isolated from patients in five study centres of the long term observational ClinSurv-HIV Cohort were included. TDR was estimated using the first viral sequence of antiretroviral treatment (ART) naive patients. One HIV sequence/patient/year of ART experienced patients was considered to estimate the proportion of ADR. Trends in the proportion of HIV drug resistance were calculated by logistic regression. Results: 9,528 patients were included into the analysis. HIV-sequences of antiretroviral naive and treatment experienced patients were available from 34% (3,267/9,528) of patients. The proportion of TDR over time was stable at 10.4% (95% CI 9.1-11.8; p (for trend)=0.6; 2001-2011). The proportion of ADR among all treated patients was 16%, whereas it was high among those with available HIV genotypic resistance test (64%; 1,310/2,049 sequences; 95% CI 62-66) but declined significantly over time (OR 0.8; 95% CI 0.77-0.83; p (for trend)<0.001; 2001-2011). Viral load monitoring subsequent to resistance testing was performed in the majority of treated patients (96%) and most of them (67%) were treated successfully. Conclusions: The proportion of TDR was stable in this study population. ADR declined significantly over time. This decline might have been influenced by broader resistance testing, resistance test guided therapy and the availability of more therapeutic options and not by a decline in the proportion of TDR within the study population

Nutritional upgrading for omnivorous carpenter ants by the endosymbiont Blochmannia

<p>Abstract</p> <p>Background</p> <p>Carpenter ants (genus <it>Camponotus</it>) are considered to be omnivores. Nonetheless, the genome sequence of <it>Blochmannia floridanus</it>, the obligate intracellular endosymbiont of <it>Camponotus floridanus</it>, suggests a function in nutritional upgrading of host resources by the bacterium. Thus, the strongly reduced genome of the endosymbiont retains genes for all subunits of a functional urease, as well as those for biosynthetic pathways for all but one (arginine) of the amino acids essential to the host.</p> <p>Results</p> <p>Nutritional upgrading by <it>Blochmannia </it>was tested in 90-day feeding experiments with brood-raising in worker-groups on chemically defined diets with and without essential amino acids and treated or not with antibiotics. Control groups were fed with cockroaches, honey water and Bhatkar agar. Worker-groups were provided with brood collected from the queenright mother-colonies (45 eggs and 45 first instar larvae each). Brood production did not differ significantly between groups of symbiotic workers on diets with and without essential amino acids. However, aposymbiotic worker groups raised significantly less brood on a diet lacking essential amino acids. Reduced brood production by aposymbiotic workers was compensated when those groups were provided with essential amino acids in their diet. Decrease of endosymbionts due to treatment with antibiotic was monitored by qRT-PCR and FISH after the 90-day experimental period. Urease function was confirmed by feeding experiments using ¹⁵N-labelled urea. GC-MS analysis of ¹⁵N-enrichment of free amino acids in workers revealed significant labelling of the non-essential amino acids alanine, glycine, aspartic acid, and glutamic acid, as well as of the essential amino acids methionine and phenylalanine.</p> <p>Conclusion</p> <p>Our results show that endosymbiotic <it>Blochmannia </it>nutritionally upgrade the diet of <it>C. floridanus </it>hosts to provide essential amino acids, and that it may also play a role in nitrogen recycling via its functional urease. <it>Blochmannia </it>may confer a significant fitness advantage via nutritional upgrading by enhancing competitive ability of <it>Camponotus </it>with other ant species lacking such an endosymbiont. Domestication of the endosymbiont may have facilitated the evolutionary success of the genus <it>Camponotus</it>.</p