Event-based Vision: A Survey

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world

Transient Response Improvement For Multi-phase Voltage Regulators

Next generation microprocessor (Vcore) requirements for high current slew rates and fast transient response together with low output voltage have posed great challenges on voltage regulator (VR) design . Since the debut of Intel 80X86 series, CPUs have greatly improved in performance with a dramatic increase on power consumption. According to the latest Intel VR11 design guidelines , the operational current may ramp up to 140A with typical voltages in the 1.1V to 1.4V range, while the slew rate of the transient current can be as high as 1.9A/ns [1, 2]. Meanwhile, the transient-response requirements are becoming stringer and stringer. This dissertation presents several topics on how to improve transient response for multi-phase voltage regulators. The Adaptive Modulation Control (AMC) is a type of non-linear control method which has proven to be effective in achieving high bandwidth designs as well as stabilizing the control loop during large load transients. It adaptively adjusts control bandwidth by changing the modulation gain, depending on different load conditions. With the AMC, a multiphase voltage regulator can be designed with an aggressively high bandwidth. When in heavy load transients where the loop could be potentially unstable, the bandwidth is lowered. Therefore, the AMC provides an optimal means for robust high-bandwidth design with excellent transient performance. The Error Amplifier Voltage Positioning (EAVP) is proposed to improve transient response by removing undesired spikes and dips after initial transient response. The EAVP works only in a short period of time during transient events without modifying the power stage and changing the control loop gain. It facilitates the error amplifier voltage recovering during transient events, achieving a fast settling time without impact on the whole control loop. Coupled inductors are an emerging topology for computing power supplies as VRs with coupled inductors show dynamic and steady-state advantages over traditional VRs. This dissertation first covers the coupling mechanism in terms of both electrical and reluctance modeling. Since the magnetizing inductance plays an important role in the coupled-inductor operation, a unified State-Space Averaging model is then built for a two-phase coupled-inductor voltage regulator. The DC solutions of the phase currents are derived in order to show the impact of the magnetizing inductance on phase current balancing. A small signal model is obtained based on the state-space-averaging model. The effects of magnetizing inductance on dynamic performance are presented. The limitations of conventional DCR current-sensing for coupled inductors are addressed. Traditional inductor DCR current sensing topology and prior arts fail to extract phase currents for coupled inductors. Two new DCR current sensing topologies for coupled inductors are presented in this dissertation. By implementation of simple RC networks, the proposed topologies can preserve the coupling effect between phases. As a result, accurate phase inductor currents and total current can be sensed, resulting in excellent current and voltage regulation. While coupled-inductor topologies are showing advantages in transient response and are becoming industry practices, they are suffering from low steady-state operating efficiency. Motivated by the challenging transient and efficiency requirements, this dissertation proposes a Full Bridge Coupled Inductor (FBCI) scheme which is able to improve transient response as well as savor high efficiency at (a) steady state. The FBCI can change the circuit configuration under different operational conditions. Its flexible topology is able to optimize both transient response and steady-state efficiency. The flexible core configuration makes implementation easy and clear of IP issues. A novel design methodology for planar magnetics based on numerical analysis of electromagnetic fields is offered and successfully applied to the design of low-voltage high power density dc-dc converters. The design methodology features intense use of FEM simulation. The design issues of planar magnetics, including loss mechanism in copper and core, winding design on PCB, core selections, winding arrangements and so on are first reviewed. After that, FEM simulators are introduced to numerically compute the core loss and winding loss. Consequently, a software platform for magnetics design is established, and optimized magnetics can then be achieved. Dynamic voltage scaling (DVS) technology is a common industry practice in optimizing power consumption of microprocessors by dynamically altering the supply voltage under different operational modes, while maintaining the performance requirements. During DVS operation, it is desirable to position the output voltage to a new level commanded by the microprocessor (CPU) with minimum delay. However, voltage deviation and slow settling time usually exist due to large output capacitance and compensation delay in voltage regulators. Although optimal DVS can be achieved by modifying the output capacitance and compensation, this method is limited by constraints from stringent static and dynamic requirements. In this dissertation, the effects of output capacitance and compensation network on DVS operation are discussed in detail. An active compensator scheme is then proposed to ensure smooth transition of the output voltage without change of power stage and compensation during DVS. Simulation and experimental results are included to demonstrate the effectiveness of the proposed scheme

Embedded Systems Energy Characterization using non-Intrusive Instrumentation

Research Report RR2006-37, LIP - ENS LyonThis research report presents a non intrusive methodology for building embedded systems energy consumption models. The method is based on measurement on real hardware in order to get a quantitative approach that takes into account the full architecture. Based on these measurements, data are grouped into class of instructions and events. These classes can then be reused in software simulators and in high-level source code transformation cost functions for optimizing compilers. The computed power model is much more simpler than previous power models while being accurate at the platform level. The methodology is illustrated using experimental results made on an ARM Integrator platform for which an accurate and full system energy model is build

Ultra-Low Power IoT Smart Visual Sensing Devices for Always-ON Applications

This work presents the design of a Smart Ultra-Low Power visual sensor architecture that couples together an ultra-low power event-based image sensor with a parallel and power-optimized digital architecture for data processing. By means of mixed-signal circuits, the imager generates a stream of address events after the extraction and binarization of spatial gradients. When targeting monitoring applications, the sensing and processing energy costs can be reduced by two orders of magnitude thanks to either the mixed-signal imaging technology, the event-based data compression and the use of event-driven computing approaches. From a system-level point of view, a context-aware power management scheme is enabled by means of a power-optimized sensor peripheral block, that requests the processor activation only when a relevant information is detected within the focal plane of the imager. When targeting a smart visual node for triggering purpose, the event-driven approach brings a 10x power reduction with respect to other presented visual systems, while leading to comparable results in terms of detection accuracy. To further enhance the recognition capabilities of the smart camera system, this work introduces the concept of event-based binarized neural networks. By coupling together the theory of binarized neural networks and focal-plane processing, a 17.8% energy reduction is demonstrated on a real-world data classification with a performance drop of 3% with respect to a baseline system featuring commercial visual sensors and a Binary Neural Network engine. Moreover, if coupling the BNN engine with the event-driven triggering detection flow, the average power consumption can be as low as the sleep power of 0.3mW in case of infrequent events, which is 8x lower than a smart camera system featuring a commercial RGB imager

System-Level Energy-Aware Design of Cyber-Physical Systems

In this technical report we present the work conducted during the first part of the PhD thesis “System-Level Energy-Aware Design of Cyber-Physical Systems”. We present the application of modelling techniques and methodologies to study energy consumption during the design and implementation of cyber-physical systems. This study is made from the electro-mechanical and computation angle. Additionally we present a setup that allows the combination of abstract models with hardware and software preliminary realizations. This allows a stepwise model to implementation transformation and improved model accuracy. Some of these techniques have been applied to the case study e-Stocking and others have been studied with more simple experimental setups.In addition to the scientific content, we also present a description of the envisioned future work and the plans that will lead to completion of this PhD thesis by April 2015

PROPOSED MIDDLEWARE SOLUTION FOR RESOURCE-CONSTRAINED DISTRIBUTED EMBEDDED NETWORKS

The explosion in processing power of embedded systems has enabled distributed embedded networks to perform more complicated tasks. Middleware are sets of encapsulations of common and network/operating system-specific functionality into generic, reusable frameworks to manage such distributed networks. This thesis will survey and categorize popular middleware implementations into three adapted layers: host-infrastructure, distribution, and common services. This thesis will then apply a quantitative approach to grading and proposing a single middleware solution from all layers for two target platforms: CubeSats and autonomous unmanned aerial vehicles (UAVs). CubeSats are 10x10x10cm nanosatellites that are popular university-level space missions, and impose power and volume constraints. Autonomous UAVs are similarly-popular hobbyist-level vehicles that exhibit similar power and volume constraints. The MAVLink middleware from the host-infrastructure layer is proposed as the middleware to manage the distributed embedded networks powering these platforms in future projects. Finally, this thesis presents a performance analysis on MAVLink managing the ARM Cortex-M 32-bit processors that power the target platforms

Design, analysis and implementation of voltage sensor for power-constrained systems

PhD ThesisThanks to an extensive effort by the global research community, the electronic technology has significantly matured over the last decade. This technology has enabled certain operations which humans could not otherwise easily perform. For instance, electronic systems can be used to perform sensing, monitoring and even control operations in environments such as outer space, underground, under the sea or even inside the human body. The main difficulty for electronics operating in these environments is access to a reliable and permanent source of energy. Using batteries as the immediate solution for this problem has helped to provide energy for limited periods of time; however, regular maintenance and replacement are required. Consequently, battery solutions fail wherever replacing them is not possible or operation for long periods is needed. For such cases, researchers have proposed harvesting ambient energy and converting it into an electrical form. An important issue with energy harvesters is that their operation and output power depend critically on the amount of energy they receive and because ambient energy often tends to be sporadic in nature, energy harvesters cannot produce stable or fixed levels of power all of the time. Therefore, electronic devices powered in this way must be capable of adapting their operation to the energy status of the harvester. To achieve this, information on the energy available for use is needed. This can be provided by a sensor capable of measuring voltage. However, stable and fixed voltage and time references are a prerequisite of most traditional voltage measurement devices, but these generally do not exist in energy harvesting environments. A further challenge is that such a sensor also needs to be powered by the energy harvester’s unstable voltage. In this thesis, the design of a reference-free voltage sensor, which can operate with a varying voltage source, is provided based on the capture of a portion of the total energy which is directly related to II the energy being sensed. This energy is then used to power a computation which quantifies captured energy over time, with the information directly generated as digital code. The sensor was fabricated in the 180 nm technology node and successfully tested by performing voltage measurements over the range 1.8 V to 0.8 V

Online Timing Slack Measurement and its Application in Field-Programmable Gate Arrays

Reliability, power consumption and timing performance are key concerns for today's integrated circuits. Measurement techniques capable of quantifying the timing characteristics of a circuit, while it is operating, facilitate a range of benefits. Delay variation due to environmental and operational conditions, and degradation can be monitored by tracking changes in timing performance. Using the measurements in a closed-loop to control power supply voltage or clock frequency allows for the reduction of timing safety margins, leading to improvements in power consumption or throughput performance through the exploitation of better-than worst-case operation. This thesis describes a novel online timing slack measurement method which can directly measure the timing performance of a circuit, accurately and with minimal overhead. Enhancements allow for the improvement of absolute accuracy and resolution. A compilation flow is reported that can automatically instrument arbitrary circuits on FPGAs with the measurement circuitry. On its own this measurement method is able to track the "health" of an integrated circuit, from commissioning through its lifetime, warning of impending failure or instigating pre-emptive degradation mitigation techniques. The use of the measurement method in a closed-loop dynamic voltage and frequency scaling scheme has been demonstrated, achieving significant improvements in power consumption and throughput performance.Open Acces

Neuromorphic perception for greenhouse technology using event-based sensors

Event-Based Cameras (EBCs), unlike conventional cameras, feature independent pixels that asynchronously generate outputs upon detecting changes in their field of view. Short calculations are performed on each event to mimic the brain. The output is a sparse sequence of events with high temporal precision. Conventional computer vision algorithms do not leverage these properties. Thus a new paradigm has been devised. While event cameras are very efficient in representing sparse sequences of events with high temporal precision, many approaches are challenged in applications where a large amount of spatially-temporally rich information must be processed in real-time. In reality, most tasks in everyday life take place in complex and uncontrollable environments, which require sophisticated models and intelligent reasoning. Typical hard problems in real-world scenes are detecting various non-uniform objects or navigation in an unknown and complex environment. In addition, colour perception is an essential fundamental property in distinguishing objects in natural scenes. Colour is a new aspect of event-based sensors, which work fundamentally differently from standard cameras, measuring per-pixel brightness changes per colour filter asynchronously rather than measuring “absolute” brightness at a constant rate. This thesis explores neuromorphic event-based processing methods for high-noise and cluttered environments with imbalanced classes. A fully event-driven processing pipeline was developed for agricultural applications to perform fruits detection and classification to unlock the outstanding properties of event cameras. The nature of features in such data was explored, and methods to represent and detect features were demonstrated. A framework for detecting and classifying features was developed and evaluated on the N-MNIST and Dynamic Vision Sensor (DVS) gesture datasets. The same network was evaluated on laboratory recorded and real-world data with various internal variations for fruits detection such as overlap, variation in size and appearance. In addition, a method to handle highly imbalanced data was developed. We examined the characteristics of spatio-temporal patterns for each colour filter to help expand our understanding of this novel data and explored their applications in classification tasks where colours were more relevant features than shapes and appearances. The results presented in this thesis demonstrate the potential and efficacy of event- based systems by demonstrating the applicability of colour event data and the viability of event-driven classification