Demystifying Event-based Sensor Biasing to Optimize Signal to Noise for Space Domain Awareness

Neuromorphic dynamic vision sensors (DVS), often called event-based sensors (EBS), are a novel class of cameras that have recently shown potential to make a significant impact in the SDA community. Their biologically-inspired design simultaneously achieves high temporal resolution, wide dynamic range, low power consumption and sparse data output, making them an ideal fit for space applications. Although initial results for SDA are promising, they typically exhibit elevated noise rates in dim conditions and have thus far failed to outperform conventional cameras in terms of limiting visual magnitude and sensitivity with high telescope scan rates. A hurdle for widespread adoption is a lack of general guidance regarding optimal camera biases (settings) for SDA. Prior studies either serve as proof of concept or focus on algorithm development; however, to date, none have provided detailed guidance on biasing EBS to optimize signal to noise ratio (SNR) for SDA tasks. The goal of this paper is to narrow the knowledge gap between EBS pixel biasing and resulting performance to optimize their capabilities for SDA. To accomplish this, we adopt a bottom-up approach, revisiting the pixel architecture to consider physics-based performance limitations. In an EBS, each pixel responds autonomously, generating "events" in response to local brightness changes within its field of view (FOV), and outputs a sparse representation of the visual scene where each event is encoded by a pixel address (x,y), a microsecond resolution timestamp (t), and a single bit polarity value (p) indicating either an increase or decrease in brightness by a defined threshold. In most camera models, behavior is fine-tuned by adjusting roughly a half-dozen biases, including threshold levels (sensitivity), bandwidth (speed of the front-end photoreceptor), and refractory period (dead-time between events in a given pixel). These parameters make EBS cameras adaptable for varied applications, but many degrees of freedom presents a challenge for optimization. Researchers unfamiliar with the technology can be overwhelmed by the myriad of biasing options and must either rely on a prescribed set of biases or manually adjust them to achieve desired performance; the latter is not typically recommended for non-experts due to 2nd-order effects such as excessive noise rates. Manufacturer default biases are considered optimized for a broad range of applications, but recent studies have demonstrated non-conventional bias techniques can significantly reduce background noise in dim conditions while still retaining signal, suggesting that SDA capabilities could be improved by a more sophisticated biasing strategy. By conducting a detailed study of how sensitivity, response speed, and noise rates scale with varied bias configurations, we aim to approach an optimal SNR bias configuration and demonstrate the maximal capabilities of current generation COTS EBS cameras for SDA. To systematically analyze and benchmark performance against a calibrated and repeatable stimulus, we developed a custom SDA test-bench to simulate stars/satellites as sub-pixel point source targets of variable speed and brightness. The set-up includes an integrating light box to provide a calibrated flat-field illumination source, a custom 170 mm radius anodized aluminum disk with precision drilled holes of diameters ranging from 100 to 250 microns, and a digitally programmable motor capable of precise speed control from ~0.1 to 800 RPM. The disk is backlit by the flat-field illumination source and connected to the motor shaft, and a 7 x 10 cm region is viewed through a Fujinon 1:1.8/7-70mm CS mount lens at a distance of 50 cm. The FOV and zoom are chosen such that the dimension of the largest holes is still sub-pixel in diameter when in focus. Even with the ability to rapidly collect measurements with this setup, the overall parameter space is still too large to fully explore without any a-priori knowledge about how the sensor responds to signal and noise, and how this depends on biases. As a result, we consider fundamental pixel behaviors to devise an efficient test strategy. We first consider strategies to limit noise rates, as these can overwhelm sensor readout when the background is dark. In prior work, this was presumably accomplished by either reducing the bandwidth biases or increasing threshold biases, but these approaches inherently limit signal. Instead of this naive approach, we draw inspiration from two recent studies: the first demonstrated an optimal balance between two bandwidth related biases accessible in some camera prototypes, and the second relies on a key observation about the statistical distribution of noise events to devise two additional biasing techniques to enhance SNR by allowing either lower thresholds or broader bandwidth settings. Using these techniques as a starting point, we examine the performance the DAVIS346 EBS. We first report baseline performance using manufacturer default biases. To quantify performance, we measure sensitivity (dimmest point source detected) and bandwidth (fastest point source detected). Next, we tune bias settings with specific detection goals (i.e. maximum velocity and/or minimum brightness) and analyze the results. Finally, we apply newly developed low-noise bias techniques and attempt to identify general principles that can be applied universally to any EBS camera to improve performance in SDA tasks. This paper provides a baseline for understanding EBS performance characteristics and will significantly lower the entry barrier for new researchers in the field of event-based SDA. More importantly, it adds insight for optimizing EBS behavior for SDA tasks and demonstrates the absolute performance limits of current generation cameras for detecting calibrated point source targets against a dark background. Finally, this study will enable follow-on work including the development of customized denoising, detection, and tracking algorithms that consider signal response and noise statistics as a function of the selected camera and bias configuration

Transitions Induced by the Discreteness of Molecules in a Small Autocatalytic System

Autocatalytic reaction system with a small number of molecules is studied numerically by stochastic particle simulations. A novel state due to fluctuation and discreteness in molecular numbers is found, characterized as extinction of molecule species alternately in the autocatalytic reaction loop. Phase transition to this state with the change of the system size and flow is studied, while a single-molecule switch of the molecule distributions is reported. Relevance of the results to intracellular processes are briefly discussed.Comment: 5 pages, 4 figure

Knot localization in adsorbing polymer rings

We study by Monte Carlo simulations a model of knotted polymer ring adsorbing onto an impenetrable, attractive wall. The polymer is described by a self-avoiding polygon (SAP) on the cubic lattice. We find that the adsorption transition temperature, the crossover exponent $\phi$ and the metric exponent $\nu$, are the same as in the model where the topology of the ring is unrestricted. By measuring the average length of the knotted portion of the ring we are able to show that adsorbed knots are localized. This knot localization transition is triggered by the adsorption transition but is accompanied by a less sharp variation of the exponent related to the degree of localization. Indeed, for a whole interval below the adsorption transition, one can not exclude a contiuous variation with temperature of this exponent. Deep into the adsorbed phase we are able to verify that knot localization is strong and well described in terms of the flat knot model.Comment: 27 pages, 10 figures. Submitter to Phys. Rev.

Mapping between dissipative and Hamiltonian systems

Theoretical studies of nonequilibrium systems are complicated by the lack of a general framework. In this work we first show that a transformation introduced by Ao recently (J. Phys. A {\bf 37}, L25 (2004)) is related to previous works of Graham (Z. Physik B {\bf 26}, 397 (1977)) and Eyink {\it et al.} (J. Stat. Phys. {\bf 83}, 385 (1996)), which can also be viewed as the generalized application of the Helmholtz theorem in vector calculus. We then show that systems described by ordinary stochastic differential equations with white noise can be mapped to thermostated Hamiltonian systems. A steady-state of a dissipative system corresponds to the equilibrium state of the corresponding Hamiltonian system. These results provides a solid theoretical ground for corresponding studies on nonequilibrium dynamics, especially on nonequilibrium steady state. The mapping permits the application of established techniques and results for Hamiltonian systems to dissipative non-Hamiltonian systems, those for thermodynamic equilibrium states to nonequilibrium steady states. We discuss several implications of the present work.Comment: 18 pages, no figure. final version for publication on J. Phys. A: Math & Theo

Functional and Biogenetical Heterogeneity of the Inner Membrane of Rat-Liver Mitochondria

Rat liver mitochondria were fragmented by a combined technique of swelling, shrinking, and sonication. Fragments of inner membrane were separated by density gradient centrifugation. They differed in several respects: electronmicroscopic appearance, phospholipid and cytochrome contents, electrophoretic behaviour of proteins and enzymatic activities. Three types of inner membrane fractions were isolated. The first type is characterized by a high activity of metal chelatase, low activities of succinate-cytochrome c reductase and of glycerolphosphate dehydrogenase, as well as by a high phospholipid content and low contents of cytochromes aa3 and b. The second type displays maximal activities of glycerolphosphate dehydrogenase and metal chelatase, but contains relatively little cytochromes and has low succinate-cytochrome c reductase activity. The third type exhibits highest succinate-cytochrome c reductase activity, a high metal chelatase activity and highest cytochrome contents. However, this fraction was low in both glycerolphosphate dehydrogenase activity and phospholipid content. This fraction was also richest in the following enzyme activities: cytochrome oxidase, oligomycin-sensitive ATPase, proline oxidase, 3-hydroxybutyrate dehydrogenase and rotenone-sensitive NADH-cytochrome c reductase. Amino acid incorporation in vitro and in vivo in the presence of cycloheximide occurs predominantly into inner membrane fractions from the second type. These data suggest that the inner membrane is composed of differently organized parts, and that polypeptides synthesized by mitochondrial ribosomes are integrated into specific parts of the inner membrane

Live demonstration: Gesture-Based remote control using stereo pair of dynamic vision sensors

This demonstration shows a natural gesture interface for console entertainment devices using as input a stereo pair of dynamic vision sensors. The event-based processing of the sparse sensor output allows fluid interaction at a laptop processor load of less than 3%

Confession Session: Learning from Others Mistakes

Accepted versio

Semi-Dense 3D Reconstruction with a Stereo Event Camera

Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision. This paper presents a solution to the problem of 3D reconstruction from data captured by a stereo event-camera rig moving in a static scene, such as in the context of stereo Simultaneous Localization and Mapping. The proposed method consists of the optimization of an energy function designed to exploit small-baseline spatio-temporal consistency of events triggered across both stereo image planes. To improve the density of the reconstruction and to reduce the uncertainty of the estimation, a probabilistic depth-fusion strategy is also developed. The resulting method has no special requirements on either the motion of the stereo event-camera rig or on prior knowledge about the scene. Experiments demonstrate our method can deal with both texture-rich scenes as well as sparse scenes, outperforming state-of-the-art stereo methods based on event data image representations.Comment: 19 pages, 8 figures, Video: https://youtu.be/Qrnpj2FD1e

Stochastic Dynamical Structure (SDS) of Nonequilibrium Processes in the Absence of Detailed Balance. IV: Emerging of Stochastic Dynamical Equalities and Steady State Thermodynamics from Darwinian Dynamics

This is the fourth paper, the last one, on solution to the problem of absence of detailed balance in nonequilibrium processes. It is an approach based on another known universal dynamics: The evolutionary dynamics first conceived by Darwin and Wallace, referring to as Darwinian dynamics in the present paper, has been found to be universally valid in biology; The statistical mechanics and thermodynamics, while enormously successful in physics, have been in an awkward situation of wanting a consistent dynamical understanding; Here we present from a formal point of view an exploration of the connection between thermodynamics and Darwinian dynamics and a few related topics. We first show that the stochasticity in Darwinian dynamics implies the existence temperature, hence the canonical distribution of Boltzmann-Gibbs type. In term of relative entropy the Second Law of thermodynamics is dynamically demonstrated without detailed balance condition, and is valid regardless of size of the system. In particular, the dynamical component responsible for breaking detailed balance condition does not contribute to the change of the relative entropy. Two types of stochastic dynamical equalities of current interest are explicitly discussed in the present approach: One is based on Feynman-Kac formula and another is a generalization of Einstein relation. Both are directly accessible to experimental tests. Our demonstration indicates that Darwinian dynamics represents logically a simple and straightforward starting point for statistical mechanics and thermodynamics and is complementary to and consistent with conservative dynamics that dominates the physical sciences. Present exploration suggests the existence of a unified stochastic dynamical framework both near and far from equilibrium.Comment: latex, 49 page