82,743 research outputs found
Fast vision through frameless event-based sensing and convolutional processing: Application to texture recognition
Address-event representation (AER) is an emergent hardware technology which shows a high potential for providing in the near future a solid technological substrate for emulating brain-like processing structures. When used for vision, AER sensors and processors are not restricted to capturing and processing still image frames, as in commercial frame-based video technology, but sense and process visual information in a pixel-level event-based frameless manner. As a result, vision processing is practically simultaneous to vision sensing, since there is no need to wait for sensing full frames. Also, only meaningful information is sensed, communicated, and processed. Of special interest for brain-like vision processing are some already reported AER convolutional chips, which have revealed a very high computational throughput as well as the possibility of assembling large convolutional neural networks in a modular fashion. It is expected that in a near future we may witness the appearance of large scale convolutional neural networks with hundreds or thousands of individual modules. In the meantime, some research is needed to investigate how to assemble and configure such large scale convolutional networks for specific applications. In this paper, we analyze AER spiking convolutional neural networks for texture recognition hardware applications. Based on the performance figures of already available individual AER convolution chips, we emulate large scale networks using a custom made event-based behavioral simulator. We have developed a new event-based processing architecture that emulates with AER hardware Manjunath's frame-based feature recognition software algorithm, and have analyzed its performance using our behavioral simulator. Recognition rate performance is not degraded. However, regarding speed, we show that recognition can be achieved before an equivalent frame is fully sensed and transmitted.Ministerio de EducaciĂłn y Ciencia TEC-2006-11730-C03-01Junta de AndalucĂa P06-TIC-01417European Union IST-2001-34124, 21677
Sistema de reconocimiento de caracteres de alta velocidad basado en eventos
Spike-based processing technology is capable of very
high speed throughput, as it does not rely on sensing and
processing sequences of frames. Besides, it allows building
complex and hierarchically structured cortical-like layers for
sophisticated processing. In this paper we summarize the
fundamental properties of this sensing and processing
technology applied to artificial vision systems and the AER
(Address Event Representation) protocol used in hardware
spiking systems. Finally a four-layer system is described for
character recognition. The system is slightly based on the
Fukushima´s Neocognitron. Realistic simulations using figures
of already existing AER devices are provided, which show
recognition delays under 10ÎĽs.Ministerio de Ciencia e InnovaciĂłn (VULCANO) TEC2009-10639-C04-0
A Vision-Based Driver Nighttime Assistance and Surveillance System Based on Intelligent Image Sensing Techniques and a Heterogamous Dual-Core Embedded System Architecture
This study proposes a vision-based intelligent nighttime driver assistance and surveillance system (VIDASS system) implemented by a set of embedded software components and modules, and integrates these modules to accomplish a component-based system framework on an embedded heterogamous dual-core platform. Therefore, this study develops and implements computer vision and sensing techniques of nighttime vehicle detection, collision warning determination, and traffic event recording. The proposed system processes the road-scene frames in front of the host car captured from CCD sensors mounted on the host vehicle. These vision-based sensing and processing technologies are integrated and implemented on an ARM-DSP heterogamous dual-core embedded platform. Peripheral devices, including image grabbing devices, communication modules, and other in-vehicle control devices, are also integrated to form an in-vehicle-embedded vision-based nighttime driver assistance and surveillance system
High-Speed Character Recognition System based on a complex hierarchical AER architecture
In this paper we briefly summarize the fundamental
properties of spikes processing applied to artificial vision
systems. This sensing and processing technology is capable of
very high speed throughput, because it does not rely on
sensing and processing sequences of frames, and because it
allows for complex hierarchically structured cortical-like
layers for sophisticated processing. The paper describes
briefly cortex-like spiking vision processing principles, and
the AER (Address Event Representation) technique used in
hardware spiking systems. Afterwards an example
application is described, which is a simplification of
Fukushima’s Neocognitron. Realistic behavioral simulations
based on existing AER hardware characteristics, reveal that
the simplified neocognitron, although it processes 52 large
kernel convolutions, is capable of performing recognition in
less than 10µs.Ministerio de EducaciĂłn y Ciencia TIC-2003-08164-C03-01Ministerio de EducaciĂłn y Ciencia TEC-2006-11730-C03-01European Union IST-2001-34124 (CAVIAR)Junta de AndalucĂa P06-TIC-0141
Hybrid Neural Network, An Efficient Low-Power Digital Hardware Implementation of Event-based Artificial Neural Network
Interest in event-based vision sensors has proliferated
in recent years, with innovative technology becoming more
accessible to new researchers and highlighting such sensors’
potential to enable low-latency sensing at low computational
cost. These sensors can outperform frame-based vision sensors
regarding data compression, dynamic range, temporal resolution
and power efficiency. However, available mature framebased
processing methods by using Artificial Neural Networks
(ANNs) surpass Spiking Neural Networks (SNNs) in terms of
accuracy of recognition. In this paper, we introduce a Hybrid
Neural Network which is an intermediate solution to exploit
advantages of both event-based and frame-based processing.We
have implemented this network in FPGA and benchmarked its
performance by using different event-based versions of MNIST
dataset. HDL codes for this project are available for academic
purpose upon request
ColibriUAV: An Ultra-Fast, Energy-Efficient Neuromorphic Edge Processing UAV-Platform with Event-Based and Frame-Based Cameras
The interest in dynamic vision sensor (DVS)-powered unmanned aerial vehicles
(UAV) is raising, especially due to the microsecond-level reaction time of the
bio-inspired event sensor, which increases robustness and reduces latency of
the perception tasks compared to a RGB camera. This work presents ColibriUAV, a
UAV platform with both frame-based and event-based cameras interfaces for
efficient perception and near-sensor processing. The proposed platform is
designed around Kraken, a novel low-power RISC-V System on Chip with two
hardware accelerators targeting spiking neural networks and deep ternary neural
networks.Kraken is capable of efficiently processing both event data from a DVS
camera and frame data from an RGB camera. A key feature of Kraken is its
integrated, dedicated interface with a DVS camera. This paper benchmarks the
end-to-end latency and power efficiency of the neuromorphic and event-based UAV
subsystem, demonstrating state-of-the-art event data with a throughput of 7200
frames of events per second and a power consumption of 10.7 \si{\milli\watt},
which is over 6.6 times faster and a hundred times less power-consuming than
the widely-used data reading approach through the USB interface. The overall
sensing and processing power consumption is below 50 mW, achieving latency in
the milliseconds range, making the platform suitable for low-latency autonomous
nano-drones as well
RN-Net: Reservoir Nodes-Enabled Neuromorphic Vision Sensing Network
Event-based cameras are inspired by the sparse and asynchronous spike
representation of the biological visual system. However, processing the event
data requires either using expensive feature descriptors to transform spikes
into frames, or using spiking neural networks that are expensive to train. In
this work, we propose a neural network architecture, Reservoir Nodes-enabled
neuromorphic vision sensing Network (RN-Net), based on simple convolution
layers integrated with dynamic temporal encoding reservoirs for local and
global spatiotemporal feature detection with low hardware and training costs.
The RN-Net allows efficient processing of asynchronous temporal features, and
achieves the highest accuracy of 99.2% for DVS128 Gesture reported to date, and
one of the highest accuracy of 67.5% for DVS Lip dataset at a much smaller
network size. By leveraging the internal device and circuit dynamics,
asynchronous temporal feature encoding can be implemented at very low hardware
cost without preprocessing and dedicated memory and arithmetic units. The use
of simple DNN blocks and standard backpropagation-based training rules further
reduces implementation costs.Comment: 12 pages, 5 figures, 4 table
Pervasive Monitoring - An Intelligent Sensor Pod Approach for Standardised Measurement Infrastructures
Geo-sensor networks have traditionally been built up in closed monolithic systems, thus limiting trans-domain usage of real-time measurements. This paper presents the technical infrastructure of a standardised embedded sensing device, which has been developed in the course of the Live Geography approach. The sensor pod implements data provision standards of the Sensor Web Enablement initiative, including an event-based alerting mechanism and location-aware Complex Event Processing functionality for detection of threshold transgression and quality assurance. The goal of this research is that the resultant highly flexible sensing architecture will bring sensor network applications one step further towards the realisation of the vision of a “digital skin for planet earth”. The developed infrastructure can potentially have far-reaching impacts on sensor-based monitoring systems through the deployment of ubiquitous and fine-grained sensor networks. This in turn allows for the straight-forward use of live sensor data in existing spatial decision support systems to enable better-informed decision-making.Seventh Framework Programme (European Commission) (FP7 project GENESIS no. 223996)Austria. Federal Ministry of Transport, Innovation and TechnologyERA-STAR Regions Project (G2real)Austria. Federal Ministry of Science and Researc
High Speed Neuromorphic Vision-Based Inspection of Countersinks in Automated Manufacturing Processes
Countersink inspection is crucial in various automated assembly lines,
especially in the aerospace and automotive sectors. Advancements in machine
vision introduced automated robotic inspection of countersinks using laser
scanners and monocular cameras. Nevertheless, the aforementioned sensing
pipelines require the robot to pause on each hole for inspection due to high
latency and measurement uncertainties with motion, leading to prolonged
execution times of the inspection task. The neuromorphic vision sensor, on the
other hand, has the potential to expedite the countersink inspection process,
but the unorthodox output of the neuromorphic technology prohibits utilizing
traditional image processing techniques. Therefore, novel event-based
perception algorithms need to be introduced. We propose a countersink detection
approach on the basis of event-based motion compensation and the mean-shift
clustering principle. In addition, our framework presents a robust event-based
circle detection algorithm to precisely estimate the depth of the countersink
specimens. The proposed approach expedites the inspection process by a factor
of 10 compared to conventional countersink inspection methods. The work
in this paper was validated for over 50 trials on three countersink workpiece
variants. The experimental results show that our method provides a precision of
0.025 mm for countersink depth inspection despite the low resolution of
commercially available neuromorphic cameras.Comment: 14 pages, 11 figures, 7 tables, submitted to Journal of Intelligent
Manufacturin
A sub-mW IoT-endnode for always-on visual monitoring and smart triggering
This work presents a fully-programmable Internet of Things (IoT) visual
sensing node that targets sub-mW power consumption in always-on monitoring
scenarios. The system features a spatial-contrast binary
pixel imager with focal-plane processing. The sensor, when working at its
lowest power mode ( at 10 fps), provides as output the number of
changed pixels. Based on this information, a dedicated camera interface,
implemented on a low-power FPGA, wakes up an ultra-low-power parallel
processing unit to extract context-aware visual information. We evaluate the
smart sensor on three always-on visual triggering application scenarios.
Triggering accuracy comparable to RGB image sensors is achieved at nominal
lighting conditions, while consuming an average power between and
, depending on context activity. The digital sub-system is extremely
flexible, thanks to a fully-programmable digital signal processing engine, but
still achieves 19x lower power consumption compared to MCU-based cameras with
significantly lower on-board computing capabilities.Comment: 11 pages, 9 figures, submitteted to IEEE IoT Journa
- …