A sub-mW IoT-endnode for always-on visual monitoring and smart triggering

This work presents a fully-programmable Internet of Things (IoT) visual sensing node that targets sub-mW power consumption in always-on monitoring scenarios. The system features a spatial-contrast $128\mathrm{x}64$ binary pixel imager with focal-plane processing. The sensor, when working at its lowest power mode ($10\mu W$ at 10 fps), provides as output the number of changed pixels. Based on this information, a dedicated camera interface, implemented on a low-power FPGA, wakes up an ultra-low-power parallel processing unit to extract context-aware visual information. We evaluate the smart sensor on three always-on visual triggering application scenarios. Triggering accuracy comparable to RGB image sensors is achieved at nominal lighting conditions, while consuming an average power between $193\mu W$ and $277\mu W$, depending on context activity. The digital sub-system is extremely flexible, thanks to a fully-programmable digital signal processing engine, but still achieves 19x lower power consumption compared to MCU-based cameras with significantly lower on-board computing capabilities.Comment: 11 pages, 9 figures, submitteted to IEEE IoT Journa

Bio-Inspired Optic Flow Sensors for Artificial Compound Eyes.

Compound eyes in flying insects have been studied to reveal the mysterious cues of vision-based flying mechanisms inside the smallest flying creatures in nature. Especially, researchers in the robotic area have made efforts to transfer the findings into their less than palm-sized unmanned air vehicles, micro-air-vehicles (MAVs). The miniaturized artificial compound eye is one of the key components in this system to provide visual information for navigation. Multi-directional sensing and motion estimation capabilities can give wide field-of-view (FoV) optic flows up to 360 solid angle. By deciphering the wide FoV optic flows, relevant information on the self-status of flight is parsed and utilized for flight command generation. In this work, we realize the wide-field optic flow sensing in a pseudo-hemispherical configuration realized by mounting a number of 2D array optic flow sensors on a flexible PCB module. The flexible PCBs can be bent to form a compound eye shape by origami packaging. From this scheme, the multiple 2D optic flow sensors can provide a modular, expandable configuration to meet low power constraints. The 2D optic flow sensors satisfy the low power constraint by employing a novel bio-inspired algorithm. We have modified the conventional elementary motion detector (EMD), which is known to be a basic operational unit in the insect’s visual pathways. We have implemented a bio-inspired time-stamp-based algorithm in mixed-mode circuits for robust operation. By optimal partitioning of analog to digital signal domains, we can realize the algorithm mostly in digital domain in a column-parallel circuits. Only the feature extraction algorithm is incorporated inside a pixel in analog circuits. In addition, the sensors integrate digital peripheral circuits to provide modular expandability. The on-chip data compressor can reduce the data rate by a factor of 8, so that it can connect a total of 25 optic flow sensors in a 4-wired Serial Peripheral Interface (SPI) bus. The packaged compound eye can transmit full-resolution optic flow data through the single 3MB/sec SPI bus. The fabricated 2D optic flow prototype sensor has achieved the power consumption of 243.3pJ/pixel and the maximum detectable optic flow of 1.96rad/sec at 120fps and 60 FoV.PhDElectrical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/108841/1/sssjpark_1.pd