The roles of visual parallax and edge attraction in the foraging behaviour of the butterfly Papilio xuthus.

Several examples of insects using visual motion to measure distance have been documented, from locusts peering to gauge the proximity of prey, to honeybees performing visual odometry en route between the hive and a flower patch. However, whether the use of parallax information is confined to specialised behaviours like these or represents a more general purpose sensory capability, is an open question. We investigate this issue in the foraging swallowtail butterfly Papilio xuthus, which we trained to associate a target presented on a monitor with a food reward. We then tracked the animal\u27s flight in real-time, allowing us to manipulate the size and/or position of the target in a closed-loop manner to create the illusion that it is situated either above or below the monitor surface. Butterflies are less attracted to (i.e. slower to approach) targets that appear, based on motion parallax, to be more distant. Furthermore, we found that the number of abortive descent manoeuvres performed prior to the first successful target approach varies according to the depth of the virtual target, with expansion and parallax cues having effects of opposing polarity. However, we found no evidence that Papilio modulate the kinematic parameters of their descents according to the apparent distance of the target. Thus, we argue that motion parallax is used to identify a proximal target object, but that the subsequent process of approaching it is based on stabilising its edge in the 2D space of the retina, without estimating its distance

Visual discomfort whilst viewing 3D stereoscopic stimuli

3D stereoscopic technology intensifies and heightens the viewer s experience by adding an extra dimension to the viewing of visual content. However, with expansion of this technology to the commercial market concerns have been expressed about the potential negative effects on the visual system, producing viewer discomfort. The visual stimulus provided by a 3D stereoscopic display differs from that of the real world, and so it is important to understand whether these differences may pose a health hazard. The aim of this thesis is to investigate the effect of 3D stereoscopic stimulation on visual discomfort. To that end, four experimental studies were conducted. In the first study two hypotheses were tested. The first hypothesis was that the viewing of 3D stereoscopic stimuli, which are located geometrically beyond the screen on which the images are displayed, would induce adaptation changes in the resting position of the eyes (exophoric heterophoria changes). The second hypothesis was that participants whose heterophoria changed as a consequence of adaptation during the viewing of the stereoscopic stimuli would experience less visual discomfort than those people whose heterophoria did not adapt. In the experiment an increase of visual discomfort change in the 3D condition in comparison with the 2D condition was found. Also, there were statistically significant changes in heterophoria under 3D conditions as compared with 2D conditions. However, there was appreciable variability in the magnitude of this adaptation among individuals, and no correlation between the amount of heterophoria change and visual discomfort change was observed. In the second experiment the two hypotheses tested were based on the vergence-accommodation mismatch theory, and the visual-vestibular mismatch theory. The vergence-accommodation mismatch theory predicts that a greater mismatch between the stimuli to accommodation and to vergence would produce greater symptoms in visual discomfort when viewing in 3D conditions than when viewing in 2D conditions. An increase of visual discomfort change in the 3D condition in comparison with the 2D condition was indeed found; however the magnitude of visual discomfort reported did not correlate with the mismatch present during the watching of 3D stereoscopic stimuli. The visual-vestibular mismatch theory predicts that viewing a stimulus stereoscopically will produce a greater sense of vection than viewing it in 2D. This will increase the conflict between the signals from the visual and vestibular systems, producing greater VIMS (Visually- Induced Motion Sickness) symptoms. Participants did indeed report an increase in motion sickness symptoms in the 3D condition. Furthermore, participants with closer seating positions reported more VIMS than participants sitting farther away whilst viewing 3D stimuli. This suggests that the amount of visual field stimulated during 3D presentation affects VIMS, and is an important factor in terms of viewing comfort. In the study more younger viewers (21 to 39 years old) than older viewers (40 years old and older) reported a greater change in visual discomfort during the 3D condition than the 2D condition. This suggests that the visual system s response to a stimulus, rather than the stimulus itself, is a reason for discomfort. No influence of gender on viewing comfort was found. In the next experiment participants fusion capability, as measured by their fusional reserves, was examined to determine whether this component has an impact on reported discomfort during the watching of movies in the 3D condition versus the 2D condition. It was hypothesised that participants with limited fusional range would experience more visual discomfort than participants with a wide fusion range. The hypothesis was confirmed but only in the case of convergent and not divergent eye movement. This observation illustrates that participants capability to convergence has a significant impact on visual comfort. The aim of the last experiment was to examine responses of the accommodation system to changes in 3D stimulus position and to determine whether discrepancies in these responses (i.e. accommodation overshoot, accommodation undershoot) could account for visual discomfort experienced during 3D stereoscopic viewing. It was found that accommodation discrepancy was larger for perceived forwards movement than for perceived backwards movement. The discrepancy was slightly higher in the group susceptible to visual discomfort than in the group not susceptible to visual discomfort, but this difference was not statistically significant. When considering the research findings as a whole it was apparent that not all participants experienced more discomfort whilst watching 3D stereoscopic stimuli than whilst watching 2D stimuli. More visual discomfort in the 3D condition than in the 2D condition was reported by 35% of the participants, whilst 24% of the participants reported more headaches and 17% of the participants reported more VIMS. The research indicates that multiple causative factors have an impact on reported symptoms. The analysis of the data suggests that discomfort experienced by people during 3D stereoscopic stimulation may reveal binocular vision problems. This observation suggests that 3D technology could be used as a screening method to diagnose un-treated binocular vision disorder. Additionally, this work shows that 3D stereoscopic technology can be easily adopted to binocular vision measurement. The conclusion of this thesis is that many people do not suffer adverse symptoms when viewing 3D stereoscopic displays, but that if adverse symptoms are present they can be caused either by the conflict in the stimulus, or by the heightened experience of self-motion which leads to Visually-Induced Motion Sickness (VIMS)

Characterising the neck motor system of the blowfly

Flying insects use visual, mechanosensory, and proprioceptive information to control their movements, both when on the ground and when airborne. Exploiting visual information for motor control is significantly simplified if the eyes remain aligned with the external horizon. In fast flying insects, head rotations relative to the body enable gaze stabilisation during highspeed manoeuvres or externally caused attitude changes due to turbulent air. Previous behavioural studies into gaze stabilisation suffered from the dynamic properties of the supplying sensor systems and those of the neck motor system being convolved. Specifically, stabilisation of the head in Dipteran flies responding to induced thorax roll involves feed forward information from the mechanosensory halteres, as well as feedback information from the visual systems. To fully understand the functional design of the blowfly gaze stabilisation system as a whole, the neck motor system needs to be investigated independently. Through X-ray micro-computed tomography (μCT), high resolution 3D data has become available, and using staining techniques developed in collaboration with the Natural History Museum London, detailed anatomical data can be extracted. This resulted in a full 3- dimensional anatomical representation of the 21 neck muscle pairs and neighbouring cuticula structures which comprise the blowfly neck motor system. Currently, on the work presented in my PhD thesis, μCT data are being used to infer function from structure by creating a biomechanical model of the neck motor system. This effort aims to determine the specific function of each muscle individually, and is likely to inform the design of artificial gaze stabilisation systems. Any such design would incorporate both sensory and motor systems as well as the control architecture converting sensor signals into motor commands under the given physical constraints of the system as a whole.Open Acces

Real-time analysis of video signals

Many practical and experimental systems employing image processing techniques have been built by other workers for various applications. Most of these systems are computer-based and very few operate in a real time environment. The objective of this work is to build a microprocessor-based system for video image processing. The system is used in conjunction with an on-line TV camera and processing is carried out in real time. The enormous storage requirement of digitized TV signals and the real time constraint suggest that some simplification of the data must take place prior to any viable processing. Data reduction is attained through the representation of objects by their edges, an approach often adopted for feature extraction in pattern recognition systems. A new technique for edge detection by applying comparison criteria to differentials at adjacent pixels of the video image is developed and implemented as a preprocessing hardware unit. A circuit for the generation of the co-ordinates of edge points is constructed to free the processing computer of this task, allowing it more time for on-line analysis of video signals. Besides the edge detector and co-ordinate generator the hardware built consists of a microprocessor system based on a Texas Instruments T.US 9900 device, a first-in-first-out buffer store and interface circuitry to a TV camera and display devices. All hardware modules and their power supplies are assembled in one unit to provide a standalone instrument. The problem chosen for investigation is analysis of motion in a visual scene. Aspects of motion studied concern the tracking of moving objects with simple geometric shapes and description of their motion. More emphasis is paid to the analysis of human eye movements and measurement of its point-of-regard which has many practical applications in the fields of physiology and psychology. This study provides a basis for the design of a processing unit attached to an oculometer to replace bulky minicomputer-based eye motion analysis systems. Programs are written for storage, analysis and display of results in real time

A vision system for mobile maritime surveillance platforms

Mobile surveillance systems play an important role to minimise security and safety threats in high-risk or hazardous environments. Providing a mobile marine surveillance platform with situational awareness of its environment is important for mission success. An essential part of situational awareness is the ability to detect and subsequently track potential target objects.Typically, the exact type of target objects is unknown, hence detection is addressed as a problem of finding parts of an image that stand out in relation to their surrounding regions or are atypical to the domain. Contrary to existing saliency methods, this thesis proposes the use of a domain specific visual attention approach for detecting potential regions of interest in maritime imagery. For this, low-level features that are indicative of maritime targets are identified. These features are then evaluated with respect to their local, regional, and global significance. Together with a domain specific background segmentation technique, the features are combined in a Bayesian classifier to direct visual attention to potential target objects.The maritime environment introduces challenges to the camera system: gusts, wind, swell, or waves can cause the platform to move drastically and unpredictably. Pan-tilt-zoom cameras that are often utilised for surveillance tasks can adjusting their orientation to provide a stable view onto the target. However, in rough maritime environments this requires high-speed and precise inputs. In contrast, omnidirectional cameras provide a full spherical view, which allows the acquisition and tracking of multiple targets at the same time. However, the target itself only occupies a small fraction of the overall view. This thesis proposes a novel, target-centric approach for image stabilisation. A virtual camera is extracted from the omnidirectional view for each target and is adjusted based on the measurements of an inertial measurement unit and an image feature tracker. The combination of these two techniques in a probabilistic framework allows for stabilisation of rotational and translational ego-motion. Furthermore, it has the specific advantage of being robust to loosely calibrated and synchronised hardware since the fusion of tracking and stabilisation means that tracking uncertainty can be used to compensate for errors in calibration and synchronisation. This then completely eliminates the need for tedious calibration phases and the adverse effects of assembly slippage over time.Finally, this thesis combines the visual attention and omnidirectional stabilisation frameworks and proposes a multi view tracking system that is capable of detecting potential target objects in the maritime domain. Although the visual attention framework performed well on the benchmark datasets, the evaluation on real-world maritime imagery produced a high number of false positives. An investigation reveals that the problem is that benchmark data sets are unconsciously being influenced by human shot selection, which greatly simplifies the problem of visual attention. Despite the number of false positives, the tracking approach itself is robust even if a high number of false positives are tracked

Perceptual models for high-refresh-rate rendering

Rendering realistic images requires substantial computational power. With new high-refresh-rate displays as well as the renaissance of virtual reality (VR) and augmented reality (AR), one cannot expect that GPU performance will scale fast enough to meet the requirements of immersive photo-realistic rendering with current rendering techniques. In this dissertation, I follow the dual of the well-known computer vision approach: vision is inverse graphics: to improve graphical algorithms, I consider the operation of the human visual system. I propose to model and exploit the limitations of the visual system in the context of novel high-refresh-rate displays; specifically, I focus on spatio-temporal perception, a topic that has received remarkably less attention than spatial-only perception so far. I present three main contributions. First, I demonstrate the validity of the perceptual approach by presenting a conceptually simple rendering technique motivated by our eyes' limited sensitivity to high spatio-temporal change which reduces the rendering load and transmission requirement of current-generation VR headsets without introducing perceivable visual artefacts. Second, I present two visual models related to motion perception: (a) a metric for detecting flicker; and (b) a comprehensive visual model to predict perceived motion quality on monitors with arbitrary refresh rates and monitor resolutions. Third, I propose an adaptive rendering algorithm that utilises the proposed models. All algorithms operate on physical colorimetric units (instead of display-referenced pixel values), for which I provide the appropriate display measurements and models. All proposed algorithms and visual models are calibrated and validated with psychophysical experiments

An end-to-end review of gaze estimation and its interactive applications on handheld mobile devices

In recent years we have witnessed an increasing number of interactive systems on handheld mobile devices which utilise gaze as a single or complementary interaction modality. This trend is driven by the enhanced computational power of these devices, higher resolution and capacity of their cameras, and improved gaze estimation accuracy obtained from advanced machine learning techniques, especially in deep learning. As the literature is fast progressing, there is a pressing need to review the state of the art, delineate the boundary, and identify the key research challenges and opportunities in gaze estimation and interaction. This paper aims to serve this purpose by presenting an end-to-end holistic view in this area, from gaze capturing sensors, to gaze estimation workflows, to deep learning techniques, and to gaze interactive applications.PostprintPeer reviewe