Inceptive Event Time-Surfaces for Object Classification Using Neuromorphic Cameras

This paper presents a novel fusion of low-level approaches for dimensionality reduction into an effective approach for high-level objects in neuromorphic camera data called Inceptive Event Time-Surfaces (IETS). IETSs overcome several limitations of conventional time-surfaces by increasing robustness to noise, promoting spatial consistency, and improving the temporal localization of (moving) edges. Combining IETS with transfer learning improves state-of-the-art performance on the challenging problem of object classification utilizing event camera data

Video synthesis from Intensity and Event Frames

Event cameras, neuromorphic devices that naturally respond to brightness changes, have multiple advantages with respect to traditional cameras. However, the difficulty of applying traditional computer vision algorithms on event data limits their usability. Therefore, in this paper we investigate the use of a deep learning-based architecture that combines an initial grayscale frame and a series of event data to estimate the following intensity frames. In particular, a fully-convolutional encoder-decoder network is employed and evaluated for the frame synthesis task on an automotive event-based dataset. Performance obtained with pixel-wise metrics confirms the quality of the images synthesized by the proposed architecture

Asynchronous Corner Tracking Algorithm based on Lifetime of Events for DAVIS Cameras

Event cameras, i.e., the Dynamic and Active-pixel Vision Sensor (DAVIS) ones, capture the intensity changes in the scene and generates a stream of events in an asynchronous fashion. The output rate of such cameras can reach up to 10 million events per second in high dynamic environments. DAVIS cameras use novel vision sensors that mimic human eyes. Their attractive attributes, such as high output rate, High Dynamic Range (HDR), and high pixel bandwidth, make them an ideal solution for applications that require high-frequency tracking. Moreover, applications that operate in challenging lighting scenarios can exploit the high HDR of event cameras, i.e., 140 dB compared to 60 dB of traditional cameras. In this paper, a novel asynchronous corner tracking method is proposed that uses both events and intensity images captured by a DAVIS camera. The Harris algorithm is used to extract features, i.e., frame-corners from keyframes, i.e., intensity images. Afterward, a matching algorithm is used to extract event-corners from the stream of events. Events are solely used to perform asynchronous tracking until the next keyframe is captured. Neighboring events, within a window size of 5x5 pixels around the event-corner, are used to calculate the velocity and direction of extracted event-corners by fitting the 2D planar using a randomized Hough transform algorithm. Experimental evaluation showed that our approach is able to update the location of the extracted corners up to 100 times during the blind time of traditional cameras, i.e., between two consecutive intensity images.Comment: Accepted to 15th International Symposium on Visual Computing (ISVC2020

Asynchronous, Photometric Feature Tracking using Events and Frames

We present a method that leverages the complementarity of event cameras and standard cameras to track visual features with low-latency. Event cameras are novel sensors that output pixel-level brightness changes, called "events". They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the same scene pattern can produce different events depending on the motion direction, establishing event correspondences across time is challenging. By contrast, standard cameras provide intensity measurements (frames) that do not depend on motion direction. Our method extracts features on frames and subsequently tracks them asynchronously using events, thereby exploiting the best of both types of data: the frames provide a photometric representation that does not depend on motion direction and the events provide low-latency updates. In contrast to previous works, which are based on heuristics, this is the first principled method that uses raw intensity measurements directly, based on a generative event model within a maximum-likelihood framework. As a result, our method produces feature tracks that are both more accurate (subpixel accuracy) and longer than the state of the art, across a wide variety of scenes.Comment: 22 pages, 15 figures, Video: https://youtu.be/A7UfeUnG6c

Remote Sensing of Ploidy Level in Quaking Aspen (Populus Tremuloides Michx.)

Ploidy level in plants may influence ecological functioning, demography and response to climate change. However, measuring ploidy level typically requires intensive cell or molecular methods. We map ploidy level variation in quaking aspen, a dominant North American tree species that can be diploid or triploid and that grows in spatially extensive clones. We identify the predictors and spatial scale of ploidy level variation using a combination of genetic and ground‐based and airborne remote sensing methods. We show that ground‐based leaf spectra and airborne canopy spectra can both classify aspen by ploidy level with a precision‐recall harmonic mean of 0.75–0.95 and Cohen\u27s kappa of c. 0.6–0.9. Ground‐based bark spectra cannot classify ploidy level better than chance. We also found that diploids are more common on higher elevation and steeper sites in a network of forest plots in Colorado, and that ploidy level distribution varies at subkilometer spatial scales. Synthesis. Our proof‐of‐concept study shows that remote sensing of ploidy level could become feasible in this tree species. Mapping ploidy level across landscapes could provide insights into the genetic basis of species\u27 responses to climate change

Semi-Dense 3D Reconstruction with a Stereo Event Camera

Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision. This paper presents a solution to the problem of 3D reconstruction from data captured by a stereo event-camera rig moving in a static scene, such as in the context of stereo Simultaneous Localization and Mapping. The proposed method consists of the optimization of an energy function designed to exploit small-baseline spatio-temporal consistency of events triggered across both stereo image planes. To improve the density of the reconstruction and to reduce the uncertainty of the estimation, a probabilistic depth-fusion strategy is also developed. The resulting method has no special requirements on either the motion of the stereo event-camera rig or on prior knowledge about the scene. Experiments demonstrate our method can deal with both texture-rich scenes as well as sparse scenes, outperforming state-of-the-art stereo methods based on event data image representations.Comment: 19 pages, 8 figures, Video: https://youtu.be/Qrnpj2FD1e

Diffusion-weighted magnetic resonance imaging detection of basal forebrain cholinergic degeneration in a mouse model

Loss of basal forebrain cholinergic neurons is an early and key feature of Alzheimer's disease, and magnetic resonance imaging (MRI) volumetric measurement of the basal forebrain has recently gained attention as a potential diagnostic tool for this condition. The aim of this study was to determine whether loss of basal forebrain cholinergic neurons underpins changes which can be detected through diffusion MRI using diffusion tensor imaging (DTI) and probabilistic tractography in a mouse model. To cause selective basal forebrain cholinergic degeneration, the toxin saporin conjugated to a p75 neurotrophin receptor antibody (mu-p75-SAP) was used. This resulted in similar to 25% loss of the basal forebrain cholinergic neurons and significant loss of terminal cholinergic projections in the hippocampus, as determined by histology. To test whether lesion of cholinergic neurons caused basal forebrain, hippocampal, or whole brain atrophy, we performed manual segmentation analysis, which revealed no significant atrophy in lesioned animals compared to controls (Rb-IgG-SAP). However, analysis by DTI of the basal forebrain area revealed a significant increase in fractional anisotropy (FA; + 7.7%), mean diffusivity (MD; + 6.1%), axial diffusivity (AD; + 8.5%) and radial diffusivity (RD; +4.0%) in lesioned mice compared to control animals. These parameters strongly inversely correlated with the number of choline acetyl transferase-positive neurons, with FA showing the greatest association (r(2) = 0.72), followed by MD (r(2) = 0.64), AD (r(2) = 0.64) and RD (r(2) = 0.61). Moreover, probabilistic tractography analysis of the septo-hippocampal tracts originating from the basal forebrain revealed an increase in streamline MD (+5.1%) and RD (+4.3%) in lesioned mice. This study illustrates that moderate loss of basal forebrain cholinergic neurons (representing only a minor proportion of all septo-hippocampal axons) can be detected by measuring either DTI parameters of the basal forebrain nuclei or tractography parameters of the basal forebrain tracts. These findings provide increased support for using DTI and probabilistic tractography as non-invasive tools for diagnosing and/or monitoring the progression of conditions affecting the integrity of the basal forebrain cholinergic system in humans, including Alzheimer's disease. Crown Copyright (C) 2012 Published by Elsevier Inc. All rights reserved

Identification of Cellular Infiltrates during Early Stages of Brain Inflammation with Magnetic Resonance Microscopy

A comprehensive view of brain inflammation during the pathogenesis of autoimmune encephalomyelitis can be achieved with the aid of high resolution non-invasive imaging techniques such as microscopic magnetic resonance imaging (μMRI). In this study we demonstrate the benefits of cryogenically-cooled RF coils to produce μMRI in vivo, with sufficient detail to reveal brain pathology in the experimental autoimmune encephalomyelitis (EAE) model. We could visualize inflammatory infiltrates in detail within various regions of the brain, already at an early phase of EAE. Importantly, this pathology could be seen clearly even without the use of contrast agents, and showed excellent correspondence with conventional histology. The cryogenically-cooled coil enabled the acquisition of high resolution images within short scan times: an important practical consideration in conducting animal experiments. The detail of the cellular infiltrates visualized by in vivo μMRI allows the opportunity to follow neuroinflammatory processes even during the early stages of disease progression. Thus μMRI will not only complement conventional histological examination but will also enable longitudinal studies on the kinetics and dynamics of immune cell infiltration

Lymphatic Clearance of the Brain: Perivascular, Paravascular and Significance for Neurodegenerative Diseases

The lymphatic clearance pathways of the brain are different compared to the other organs of the body and have been the subject of heated debates. Drainage of brain extracellular fluids, particularly interstitial fluid (ISF) and cerebrospinal fluid (CSF), is not only important for volume regulation, but also for removal of waste products such as amyloid beta (A?). CSF plays a special role in clinical medicine, as it is available for analysis of biomarkers for Alzheimer’s disease. Despite the lack of a complete anatomical and physiological picture of the communications between the subarachnoid space (SAS) and the brain parenchyma, it is often assumed that A? is cleared from the cerebral ISF into the CSF. Recent work suggests that clearance of the brain mainly occurs during sleep, with a specific role for peri- and para-vascular spaces as drainage pathways from the brain parenchyma. However, the direction of flow, the anatomical structures involved and the driving forces remain elusive, with partially conflicting data in literature. The presence of A? in the glia limitans in Alzheimer’s disease suggests a direct communication of ISF with CSF. Nonetheless, there is also the well-described pathology of cerebral amyloid angiopathy associated with the failure of perivascular drainage of A?. Herein, we review the role of the vasculature and the impact of vascular pathology on the peri- and para-vascular clearance pathways of the brain. The different views on the possible routes for ISF drainage of the brain are discussed in the context of pathological significance