PCA-RECT: An Energy-efficient Object Detection Approach for Event Cameras

We present the first purely event-based, energy-efficient approach for object detection and categorization using an event camera. Compared to traditional frame-based cameras, choosing event cameras results in high temporal resolution (order of microseconds), low power consumption (few hundred mW) and wide dynamic range (120 dB) as attractive properties. However, event-based object recognition systems are far behind their frame-based counterparts in terms of accuracy. To this end, this paper presents an event-based feature extraction method devised by accumulating local activity across the image frame and then applying principal component analysis (PCA) to the normalized neighborhood region. Subsequently, we propose a backtracking-free k-d tree mechanism for efficient feature matching by taking advantage of the low-dimensionality of the feature representation. Additionally, the proposed k-d tree mechanism allows for feature selection to obtain a lower-dimensional dictionary representation when hardware resources are limited to implement dimensionality reduction. Consequently, the proposed system can be realized on a field-programmable gate array (FPGA) device leading to high performance over resource ratio. The proposed system is tested on real-world event-based datasets for object categorization, showing superior classification performance and relevance to state-of-the-art algorithms. Additionally, we verified the object detection method and real-time FPGA performance in lab settings under non-controlled illumination conditions with limited training data and ground truth annotations.Comment: Accepted in ACCV 2018 Workshops, to appea

Video synthesis from Intensity and Event Frames

Event cameras, neuromorphic devices that naturally respond to brightness changes, have multiple advantages with respect to traditional cameras. However, the difficulty of applying traditional computer vision algorithms on event data limits their usability. Therefore, in this paper we investigate the use of a deep learning-based architecture that combines an initial grayscale frame and a series of event data to estimate the following intensity frames. In particular, a fully-convolutional encoder-decoder network is employed and evaluated for the frame synthesis task on an automotive event-based dataset. Performance obtained with pixel-wise metrics confirms the quality of the images synthesized by the proposed architecture

Semi-Dense 3D Reconstruction with a Stereo Event Camera

Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision. This paper presents a solution to the problem of 3D reconstruction from data captured by a stereo event-camera rig moving in a static scene, such as in the context of stereo Simultaneous Localization and Mapping. The proposed method consists of the optimization of an energy function designed to exploit small-baseline spatio-temporal consistency of events triggered across both stereo image planes. To improve the density of the reconstruction and to reduce the uncertainty of the estimation, a probabilistic depth-fusion strategy is also developed. The resulting method has no special requirements on either the motion of the stereo event-camera rig or on prior knowledge about the scene. Experiments demonstrate our method can deal with both texture-rich scenes as well as sparse scenes, outperforming state-of-the-art stereo methods based on event data image representations.Comment: 19 pages, 8 figures, Video: https://youtu.be/Qrnpj2FD1e

Event-based Asynchronous Sparse Convolutional Networks

Event cameras are bio-inspired sensors that respond to per-pixel brightness changes in the form of asynchronous and sparse "events". Recently, pattern recognition algorithms, such as learning-based methods, have made significant progress with event cameras by converting events into synchronous dense, image-like representations and applying traditional machine learning methods developed for standard cameras. However, these approaches discard the spatial and temporal sparsity inherent in event data at the cost of higher computational complexity and latency. In this work, we present a general framework for converting models trained on synchronous image-like event representations into asynchronous models with identical output, thus directly leveraging the intrinsic asynchronous and sparse nature of the event data. We show both theoretically and experimentally that this drastically reduces the computational complexity and latency of high-capacity, synchronous neural networks without sacrificing accuracy. In addition, our framework has several desirable characteristics: (i) it exploits spatio-temporal sparsity of events explicitly, (ii) it is agnostic to the event representation, network architecture, and task, and (iii) it does not require any train-time change, since it is compatible with the standard neural networks' training process. We thoroughly validate the proposed framework on two computer vision tasks: object detection and object recognition. In these tasks, we reduce the computational complexity up to 20 times with respect to high-latency neural networks. At the same time, we outperform state-of-the-art asynchronous approaches up to 24% in prediction accuracy

Tyrosine Kinase Syk Non-Enzymatic Inhibitors and Potential Anti-Allergic Drug-Like Compounds Discovered by Virtual and In Vitro Screening

In the past decade, the spleen tyrosine kinase (Syk) has shown a high potential for the discovery of new treatments for inflammatory and autoimmune disorders. Pharmacological inhibitors of Syk catalytic site bearing therapeutic potential have been developed, with however limited specificity towards Syk. To address this topic, we opted for the design of drug-like compounds that could impede the interaction of Syk with its cellular partners while maintaining an active kinase protein. To achieve this challenging task, we used the powerful potential of intracellular antibodies for the modulation of cellular functions in vivo, combined to structure-based in silico screening. In our previous studies, we reported the anti-allergic properties of the intracellular antibody G4G11. With the aim of finding functional mimics of G4G11, we developed an Antibody Displacement Assay and we isolated the drug-like compound C-13, with promising in vivo anti-allergic activity. The likely binding cavity of this compound is located at the close vicinity of G4G11 epitope, far away from the catalytic site of Syk. Here we report the virtual screen of a collection of 500,000 molecules against this new cavity, which led to the isolation of 1000 compounds subsequently evaluated for their in vitro inhibitory effects using the Antibody Displacement Assay. Eighty five compounds were selected and evaluated for their ability to inhibit the liberation of allergic mediators from mast cells. Among them, 10 compounds inhibited degranulation with IC50 values ≤10 µM. The most bioactive compounds combine biological activity, significant inhibition of antibody binding and strong affinity for Syk. Moreover, these molecules show a good potential for oral bioavailability and are not kinase catalytic site inhibitors. These bioactive compounds could be used as starting points for the development of new classes of non-enzymatic inhibitors of Syk and for drug discovery endeavour in the field of inflammation related disorders

Asynchronous, Photometric Feature Tracking using Events and Frames

We present a method that leverages the complementarity of event cameras and standard cameras to track visual features with low-latency. Event cameras are novel sensors that output pixel-level brightness changes, called "events". They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the same scene pattern can produce different events depending on the motion direction, establishing event correspondences across time is challenging. By contrast, standard cameras provide intensity measurements (frames) that do not depend on motion direction. Our method extracts features on frames and subsequently tracks them asynchronously using events, thereby exploiting the best of both types of data: the frames provide a photometric representation that does not depend on motion direction and the events provide low-latency updates. In contrast to previous works, which are based on heuristics, this is the first principled method that uses raw intensity measurements directly, based on a generative event model within a maximum-likelihood framework. As a result, our method produces feature tracks that are both more accurate (subpixel accuracy) and longer than the state of the art, across a wide variety of scenes.Comment: 22 pages, 15 figures, Video: https://youtu.be/A7UfeUnG6c

Neuromodulated synaptic plasticity on the SpiNNaker neuromorphic system

SpiNNaker is a digital neuromorphic architecture, designed specifically for the low power simulation of large-scale spiking neural networks at speeds close to biological real-time. Unlike other neuromorphic systems, SpiNNaker allows users to develop their own neuron and synapse models as well as specify arbitrary connectivity. As a result SpiNNaker has proved to be a powerful tool for studying different neuron models as well as synaptic plasticity—believed to be one of the main mechanisms behind learning and memory in the brain. A number of Spike-Timing-Dependent-Plasticity(STDP) rules have already been implemented on SpiNNaker and have been shown to be capable of solving various learning tasks in real-time. However, while STDP is an important biological theory of learning, it is a form of Hebbian or unsupervised learning and therefore does not explain behaviors that depend on feedback from the environment. Instead, learning rules based on neuromodulated STDP (three-factor learning rules) have been shown to be capable of solving reinforcement learning tasks in a biologically plausible manner. In this paper we demonstrate for the first time how a model of three-factor STDP, with the third-factor representing spikes from dopaminergic neurons, can be implemented on the SpiNNaker neuromorphic system. Using this learning rule we first show how reward and punishment signals can be delivered to a single synapse before going on to demonstrate it in a larger network which solves the credit assignment problem in a Pavlovian conditioning experiment. Because of its extra complexity, we find that our three-factor learning rule requires approximately 2× as much processing time as the existing SpiNNaker STDP learning rules. However, we show that it is still possible to run our Pavlovian conditioning model with up to 1 × 104 neurons in real-time, opening up new research opportunities for modeling behavioral learning on SpiNNaker

The genetic interaction network of CCW12, a Saccharomyces cerevisiae gene required for cell wall integrity during budding and formation of mating projections

<p>Abstract</p> <p>Background</p> <p>Mannoproteins construct the outer cover of the fungal cell wall. The covalently linked cell wall protein Ccw12p is an abundant mannoprotein. It is considered as crucial structural cell wall component since in baker's yeast the lack of <it>CCW12 </it>results in severe cell wall damage and reduced mating efficiency.</p> <p>Results</p> <p>In order to explore the function of <it>CCW12</it>, we performed a Synthetic Genetic Analysis (SGA) and identified genes that are essential in the absence of <it>CCW12</it>. The resulting interaction network identified 21 genes involved in cell wall integrity, chitin synthesis, cell polarity, vesicular transport and endocytosis. Among those are <it>PFD1</it>, <it>WHI3</it>, <it>SRN2</it>, <it>PAC10</it>, <it>FEN1 </it>and <it>YDR417C</it>, which have not been related to cell wall integrity before. We correlated our results with genetic interaction networks of genes involved in glucan and chitin synthesis. A core of genes essential to maintain cell integrity in response to cell wall stress was identified. In addition, we performed a large-scale transcriptional analysis and compared the transcriptional changes observed in mutant <it>ccw12</it>Δ with transcriptomes from studies investigating responses to constitutive or acute cell wall damage. We identified a set of genes that are highly induced in the majority of the mutants/conditions and are directly related to the cell wall integrity pathway and cell wall compensatory responses. Among those are <it>BCK1</it>, <it>CHS3</it>, <it>EDE1</it>, <it>PFD1</it>, <it>SLT2 </it>and <it>SLA1 </it>that were also identified in the SGA. In contrast, a specific feature of mutant <it>ccw12</it>Δ is the transcriptional repression of genes involved in mating. Physiological experiments substantiate this finding. Further, we demonstrate that Ccw12p is present at the cell periphery and highly concentrated at the presumptive budding site, around the bud, at the septum and at the tip of the mating projection.</p> <p>Conclusions</p> <p>The combination of high throughput screenings, phenotypic analyses and localization studies provides new insight into the function of Ccw12p. A compensatory response, culminating in cell wall remodelling and transport/recycling pathways is required to buffer the loss of <it>CCW12</it>. Moreover, the enrichment of Ccw12p in bud, septum and mating projection is consistent with a role of Ccw12p in preserving cell wall integrity at sites of active growth.</p> <p>The microarray data produced in this analysis have been submitted to NCBI GEO database and GSE22649 record was assigned.</p

Parametric POMDPs for planning in continuous state spaces

This thesis is concerned with planning and acting under uncertainty in partially-observable continuous domains. In particular, it focusses on the problem of mobile robot navigation given a known map. The dominant paradigm for robot localisation is to use Bayesian estimation to maintain a probability distribution over possible robot poses. In contrast, control algorithms often base their decisions on the assumption that a single state, such as the mode of this distribution, is correct. In scenarios involving significant uncertainty, this can lead to serious control errors. It is generally agreed that the reliability of navigation in uncertain environments would be greatly improved by the ability to consider the entire distribution when acting, rather than the single most likely state. The framework adopted in this thesis for modelling navigation problems mathematically is the Partially Observable Markov Decision Process (POMDP). An exact solution to a POMDP problem provides the optimal balance between reward-seeking behaviour and information-seeking behaviour, in the presence of sensor and actuation noise. Unfortunately, previous exact and approximate solution methods have had difficulty scaling to real applications. The contribution of this thesis is the formulation of an approach to planning in the space of continuous parameterised approximations to probability distributions. Theoretical and practical results are presented which show that, when compared with similar methods from the literature, this approach is capable of scaling to larger and more realistic problems. In order to apply the solution algorithm to real-world problems, a number of novel improvements are proposed. Specifically, Monte Carlo methods are employed to estimate distributions over future parameterised beliefs, improving planning accuracy without a loss of efficiency. Conditional independence assumptions are exploited to simplify the problem, reducing computational requirements. Scalability is further increased by focussing computation on likely beliefs, using metric indexing structures for efficient function approximation. Local online planning is incorporated to assist global offline planning, allowing the precision of the latter to be decreased without adversely affecting solution quality. Finally, the algorithm is implemented and demonstrated during real-time control of a mobile robot in a challenging navigation task. We argue that this task is substantially more challenging and realistic than previous problems to which POMDP solution methods have been applied. Results show that POMDP planning, which considers the evolution of the entire probability distribution over robot poses, produces significantly more robust behaviour when compared with a heuristic planner which considers only the most likely states and outcomes

Participation of Candida albicans transcription factor Rlm1 in cell wall biogenesis and virulence

Candida albicans cell wall is important for growth and interaction with the environment. RLM1 is one of the putative transcription factors involved in the cell wall integrity pathway, which plays an important role in the maintenance of the cell wall integrity. In this work we investigated the involvement of RLM1 in the cell wall biogenesis and in virulence. Newly constructed C. albicans Δ/Δrlm1 mutants showed typical cell wall weakening phenotypes, such as hypersensitivity to Congo Red, Calcofluor White, and caspofungin (phenotype reverted in the presence of sorbitol), confirming the involvement of RLM1 in the cell wall integrity. Additionally, the cell wall of C. albicans Δ/Δrlm1 showed a significant increase in chitin (213%) and reduction in mannans (60%), in comparison with the wild-type, results that are consistent with cell wall remodelling. Microarray analysis in the absence of any stress showed that deletion of RLM1 in C. albicans significantly down-regulated genes involved in carbohydrate catabolism such as DAK2, GLK4, NHT1 and TPS1, up-regulated genes involved in the utilization of alternative carbon sources, like AGP2, SOU1, SAP6, CIT1 or GAL4, and genes involved in cell adhesion like ECE1, ALS1, ALS3, HWP1 or RBT1. In agreement with the microarray results adhesion assays showed an increased amount of adhering cells and total biomass in the mutant strain, in comparison with the wild-type. C. albicans mutant Δ/Δrlm1 strain was also found to be less virulent than the wild-type and complemented strains in the murine model of disseminated candidiasis. Overall, we showed that in the absence of RLM1 the modifications in the cell wall composition alter yeast interaction with the environment, with consequences in adhesion ability and virulence. The gene expression findings suggest that this gene participates in the cell wall biogenesis, with the mutant rearranging its metabolic pathways to allow the use of alternative carbon sources.This work was supported by CBMA (Centre of Molecular and Environmental Biology) through the FCT (Fundacao para a Ciencia e Tecnologia) project PEst-C/BIA/UI4050/2011. Yolanda Delgado-Silva was supported by an ALbAN scholarship (No E07D400922PE), and Alexandra Correia by SFRH/BD/31354/2006 fellowship. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript