Event-based Vision: A Survey

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world

Sensing and perception: Connectionist approaches to subcognitive computing

New approaches to machine sensing and perception are presented. The motivation for crossdisciplinary studies of perception in terms of AI and neurosciences is suggested. The question of computing architecture granularity as related to global/local computation underlying perceptual function is considered and examples of two environments are given. Finally, the examples of using one of the environments, UCLA PUNNS, to study neural architectures for visual function are presented

Seeing Things As We Do: Ecological Psychology And The Normativity Of Visual Perception

In virtue of what is perception successful? In philosophy and psychology, we sometimes assume that visual accuracy amounts to a correspondence between percepts and subject-independent, physical properties. In this dissertation, I argue that we should reject this assumption in favor of norms grounded in the action-guiding nature of perception. Recent theories of perception purport to cast off the intellectualist baggage of twentieth-century thinking, and to address perception in its own distinctive terms. I show that these approaches are unified in aiming to reduce spatial aspects of the percept to subject-independent geometrical facts about the object-perceiver relation. In doing so, these views remain guilty of an unwarranted assimilation of perception to cognition. Perceptual constancy, the capacity to encounter a relatively stable world of object properties despite variation in sensory stimulation, is measured using a metric that has percept-physical property correspondence at one extreme, and retinal match at the other. Advocates of the correspondence norm freely redeploy this metric as gauging accuracy in perception, so that the closer a percept comes to invariantly matching the distal property, the closer it comes to veridically presenting the environment. Yet, correspondence views are committed to widespread misperception that cannot be accounted for in terms of evolutionary complexity. I distinguish between descriptive and normative enterprises in cognitive science, and suggest that we reinterpret the constancy metric as an empirically useful, descriptive quantificational tool—one that does not straightforwardly entail normative facts. With the correspondence norm undercut, I develop a more viable framework for understanding accuracy, one that draws on James Gibson’s ecological theory. Accordingly, accuracy is best understood pragmatically, in ecological terms such as usefulness. Partial constancy is often sufficient for an organism to act effectively in its environment, a result that suggests surprising consequences for what is seen in perception. In color ontology, there is some theoretical attention to descriptive facts about constancy. However, because of a worry about stipulating perceiver and context standards, theorists continue to reject ecological approaches to color. I resolve the worry by appealing to pluralism about scientific objects. The resulting framework is ecologically sensible, empirically useful, and deeply interdisciplinary

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

The Land Experiments in Colour Vision - Colour as a Physical, Phenomenological and Synthetic Object

This thesis analyses the historical and intellectual context of Edwin Land’s experiments in colour vision. I argue that the colour vision research program and retinex theory developed by Land and his colleagues provided a satisfying synthesis of two divergent schools in the history of colour science. The first chapter of this thesis establishes the existence of the “physical” school of colour science. The defining feature of this school was the belief in the colour atomism hypothesis. This is the idea that the colour perceived at a point in the visual field is completely determined by the physical properties of the light rays entering the retina at that point. In other words, there is a one-to-one correspondence between the physical properties of light rays and colour sensation at a point in the visual field. The second chapter establishes the existence of the “phenomenological” school of colour science. The defining feature of this school was the discovery of colour phenomena which could not be accounted for by the colour atomism hypothesis. Among these phenomena were “coloured shadows”, “simultaneous colour contrast”, and “colour constancy”. The third chapter shows how Land’s colour vision research program and retinex theory reconciled these two schools. Land and his colleagues demonstrated that the colour atomism hypothesis is a special case, valid only for points of light. The colour phenomena studied by the “phenomenological” school could be predicted by a computational model – retinex theory – which accounted for colour as it is perceived over a wide visual field, rather than simply at single points. In this process, Land and colleagues built up a new understanding of colour vision as a practical utility evolved for the organism, designed to achieve colour constancy

Flowers through insect eyes: the contribution of pollinator vision to the evolution of flower colour

PhDFlowers’ colours are an essential element of their ability to attract visits from pollinators. However, the colours as they appear to human observers can differ substantially from their appearance to insect pollinators, and so it is essential to consider pollinator vision in any study of the ecology of flower colour. In this thesis I describe how I have overseen the development of an online database to provide accurate information on floral spectral reflectance measured without human observational bias. This resource allows a more accurate consideration of flower colours in future studies, and permits investigations of flower colours within and across habitats. Using the records in this database, I analysed flowers from two European habitats for spatial or temporal changes, modelling the colours according to insect visual perception. I discovered that the insect-colour composition of the plant communities does not change either along an altitudinal gradient or throughout the year. These novel and ecologically-relevant analyses contradict previous observational studies, but support the theory of a pollination “market” in which flowers compete for pollinator visitation. I then describe my experimental investigations into the visual capabilities of two pollinators and how this may relate to what colours of flowers they visit. Firstly I study the foraging behaviour of bees under spatially inconsistent illumination and how this impacts on their choice behaviour. I revealed patchy light can have measurable effects on bee foraging behaviour: they intentionally choose familiar over unfamiliar illumination, which may impact on the flowers they visit in complex natural environments. Secondly, I detail the new evidence for a red-sensitive photoreceptor in South African monkey beetles, a major pollinator in a habitat containing many longwavelength- reflecting flowers, which are not classically “attractive” to bees. Throughout this thesis, I explore how pollinator vision has shaped the evolution of flower colours in different contexts.Biotechnology and Biological Science Research Council. Royal Botanical Gardens Kew (BBS/S/L-2005/12155A

A Review of Evolution, Behavior, and Vision with an Experimental Evolution Study on Color Vision in Drosophila melanogaster

The first chapter of this thesis is to take a piece by piece look at the factors that contributed to the experimental evolution study that will be discussed in Chapter 2. Behavior, how that can affect experimental studies, and how biases can affect sensory systems and preference in subject species. Specifically visual sensory systems are described in detail, from the possible evolutionary histories, to major components that contribute to eye structure, form, and/or abilities. We discuss how to define color vision, and what are the prerequisites for color vision in species

An Investigation of the Operational and Design Characteristics of Circadian Lighting Systems - Report

This report investigates the operation of circadian lighting systems to gain an understanding of the main design and control characteristics and to promote different objectives for use. Research is guided by asking how color intensity, color temperature and their temporal characteristics are related to the circadian response, and how this knowledge can be utilized when designing and operating lighting systems for indoor environments. This report consists of an extensive literature review and case study application cut short by the impact of the novel coronavirus. The case study takes place in an office space housed on the UNMC campus featuring an installed circadian lighting system capable of changing color temperature and intensity independently. The results of the literature review lead to the understanding of biological impacts of suggested operational patterns for the lighting system. Specifically, the interaction between human physical characteristics as they relate to the current lighting technologies has helped to develop the rational for use of these systems. These biological impacts ultimately aim towards improved occupant attention and well-being in the space. Future investigation and implementation are encouraged to continue advanced analysis of occupant response to varied patterns of operation for this circadian lighting system

The Brightness of Colour

Background: The perception of brightness depends on spatial context: the same stimulus can appear light or dark depending on what surrounds it. A less well-known but equally important contextual phenomenon is that the colour of a stimulus can also alter its brightness. Specifically, stimuli that are more saturated (i.e. purer in colour) appear brighter than stimuli that are less saturated at the same luminance. Similarly, stimuli that are red or blue appear brighter than equiluminant yellow and green stimuli. This non-linear relationship between stimulus intensity and brightness, called the Helmholtz-Kohlrausch (HK) effect, was first described in the nineteenth century but has never been explained. Here, we take advantage of the relative simplicity of this 'illusion' to explain it and contextual effects more generally, by using a simple Bayesian ideal observer model of the human visual ecology. We also use fMRI brain scans to identify the neural correlates of brightness without changing the spatial context of the stimulus, which has complicated the interpretation of related fMRI studies.Results: Rather than modelling human vision directly, we use a Bayesian ideal observer to model human visual ecology. We show that the HK effect is a result of encoding the non-linear statistical relationship between retinal images and natural scenes that would have been experienced by the human visual system in the past. We further show that the complexity of this relationship is due to the response functions of the cone photoreceptors, which themselves are thought to represent an efficient solution to encoding the statistics of images. Finally, we show that the locus of the response to the relationship between images and scenes lies in the primary visual cortex (V1), if not earlier in the visual system, since the brightness of colours (as opposed to their luminance) accords with activity in V1 as measured with fMRI.Conclusions: The data suggest that perceptions of brightness represent a robust visual response to the likely sources of stimuli, as determined, in this instance, by the known statistical relationship between scenes and their retinal responses. While the responses of the early visual system (receptors in this case) may represent specifically the statistics of images, post receptor responses are more likely represent the statistical relationship between images and scenes. A corollary of this suggestion is that the visual cortex is adapted to relate the retinal image to behaviour given the statistics of its past interactions with the sources of retinal images: the visual cortex is adapted to the signals it receives from the eyes, and not directly to the world beyond