2,292 research outputs found
Bio-Inspired Stereo Vision Calibration for Dynamic Vision Sensors
Many advances have been made in the eld of computer vision. Several recent research trends
have focused on mimicking human vision by using a stereo vision system. In multi-camera systems, a
calibration process is usually implemented to improve the results accuracy. However, these systems generate
a large amount of data to be processed; therefore, a powerful computer is required and, in many cases,
this cannot be done in real time. Neuromorphic Engineering attempts to create bio-inspired systems that
mimic the information processing that takes place in the human brain. This information is encoded using
pulses (or spikes) and the generated systems are much simpler (in computational operations and resources),
which allows them to perform similar tasks with much lower power consumption, thus these processes
can be developed over specialized hardware with real-time processing. In this work, a bio-inspired stereovision
system is presented, where a calibration mechanism for this system is implemented and evaluated
using several tests. The result is a novel calibration technique for a neuromorphic stereo vision system,
implemented over specialized hardware (FPGA - Field-Programmable Gate Array), which allows obtaining
reduced latencies on hardware implementation for stand-alone systems, and working in real time.Ministerio de Economía y Competitividad TEC2016-77785-PMinisterio de Economía y Competitividad TIN2016-80644-
Stereo Matching in Address-Event-Representation (AER) Bio-Inspired Binocular Systems in a Field-Programmable Gate Array (FPGA)
In stereo-vision processing, the image-matching step is essential for results, although it
involves a very high computational cost. Moreover, the more information is processed, the more time
is spent by the matching algorithm, and the more ine cient it is. Spike-based processing is a relatively
new approach that implements processing methods by manipulating spikes one by one at the time
they are transmitted, like a human brain. The mammal nervous system can solve much more complex
problems, such as visual recognition by manipulating neuron spikes. The spike-based philosophy
for visual information processing based on the neuro-inspired address-event-representation (AER)
is currently achieving very high performance. The aim of this work was to study the viability of a
matching mechanism in stereo-vision systems, using AER codification and its implementation in
a field-programmable gate array (FPGA). Some studies have been done before in an AER system
with monitored data using a computer; however, this kind of mechanism has not been implemented
directly on hardware. To this end, an epipolar geometry basis applied to AER systems was studied
and implemented, with other restrictions, in order to achieve good results in a real-time scenario.
The results and conclusions are shown, and the viability of its implementation is proven.Ministerio de Economía y Competitividad TEC2016-77785-
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world
Address-Event Based Stereo Vision with Bio-Inspired Silicon Retina Imagers
Artificial intelligenc
On the AER Stereo-Vision Processing: A Spike Approach to Epipolar Matching
Image processing in digital computer systems usually considers
visual information as a sequence of frames. These frames are from cameras that
capture reality for a short period of time. They are renewed and transmitted at a
rate of 25-30 fps (typical real-time scenario). Digital video processing has to
process each frame in order to detect a feature on the input. In stereo vision,
existing algorithms use frames from two digital cameras and process them pixel
by pixel until it finds a pattern match in a section of both stereo frames. To
process stereo vision information, an image matching process is essential, but it
needs very high computational cost. Moreover, as more information is
processed, the more time spent by the matching algorithm, the more inefficient
it is. Spike-based processing is a relatively new approach that implements
processing by manipulating spikes one by one at the time they are transmitted,
like a human brain. The mammal nervous system is able to solve much more
complex problems, such as visual recognition by manipulating neuron’s spikes.
The spike-based philosophy for visual information processing based on the
neuro-inspired Address-Event- Representation (AER) is achieving nowadays
very high performances. The aim of this work is to study the viability of a
matching mechanism in a stereo-vision system, using AER codification. This
kind of mechanism has not been done before to an AER system. To do that,
epipolar geometry basis applied to AER system are studied, and several tests
are run, using recorded data and a computer. The results and an average error
are shown (error less than 2 pixels per point); and the viability is proved
Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery
One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions
CED: Color Event Camera Dataset
Event cameras are novel, bio-inspired visual sensors, whose pixels output
asynchronous and independent timestamped spikes at local intensity changes,
called 'events'. Event cameras offer advantages over conventional frame-based
cameras in terms of latency, high dynamic range (HDR) and temporal resolution.
Until recently, event cameras have been limited to outputting events in the
intensity channel, however, recent advances have resulted in the development of
color event cameras, such as the Color-DAVIS346. In this work, we present and
release the first Color Event Camera Dataset (CED), containing 50 minutes of
footage with both color frames and events. CED features a wide variety of
indoor and outdoor scenes, which we hope will help drive forward event-based
vision research. We also present an extension of the event camera simulator
ESIM that enables simulation of color events. Finally, we present an evaluation
of three state-of-the-art image reconstruction methods that can be used to
convert the Color-DAVIS346 into a continuous-time, HDR, color video camera to
visualise the event stream, and for use in downstream vision applications.Comment: Conference on Computer Vision and Pattern Recognition Workshop
Event-driven stereo vision with orientation filters
The recently developed Dynamic Vision Sensors
(DVS) sense dynamic visual information asynchronously and
code it into trains of events with sub-micro second temporal
resolution. This high temporal precision makes the output of
these sensors especially suited for dynamic 3D visual
reconstruction, by matching corresponding events generated by
two different sensors in a stereo setup. This paper explores the
use of Gabor filters to extract information about the orientation
of the object edges that produce the events, applying the
matching algorithm to the events generated by the Gabor filters
and not to those produced by the DVS. This strategy provides
more reliably matched pairs of events, improving the final 3D
reconstruction.European Union PRI-PIMCHI-2011-0768Ministerio de Economía y Competitividad TEC2009-10639-C04-01Ministerio de Economía y Competitividad TEC2012-37868-C04-01Junta de Andalucía TIC-609
Event-Driven Technologies for Reactive Motion Planning: Neuromorphic Stereo Vision and Robot Path Planning and Their Application on Parallel Hardware
Die Robotik wird immer mehr zu einem Schlüsselfaktor des technischen Aufschwungs. Trotz beeindruckender Fortschritte in den letzten Jahrzehnten, übertreffen Gehirne von Säugetieren in den Bereichen Sehen und Bewegungsplanung
noch immer selbst die leistungsfähigsten Maschinen. Industrieroboter sind sehr schnell und präzise, aber ihre Planungsalgorithmen sind in hochdynamischen Umgebungen, wie sie für die Mensch-Roboter-Kollaboration (MRK) erforderlich sind, nicht leistungsfähig genug. Ohne schnelle und adaptive Bewegungsplanung kann sichere MRK nicht garantiert werden. Neuromorphe Technologien, einschließlich visueller Sensoren und Hardware-Chips, arbeiten asynchron und verarbeiten so raum-zeitliche Informationen sehr effizient. Insbesondere ereignisbasierte visuelle Sensoren sind konventionellen, synchronen Kameras bei vielen Anwendungen bereits überlegen. Daher haben ereignisbasierte Methoden
ein großes Potenzial, schnellere und energieeffizientere Algorithmen zur Bewegungssteuerung in der MRK zu ermöglichen. In dieser Arbeit wird ein Ansatz zur flexiblen reaktiven Bewegungssteuerung eines Roboterarms vorgestellt. Dabei
wird die Exterozeption durch ereignisbasiertes Stereosehen erreicht und die Pfadplanung ist in einer neuronalen Repräsentation des Konfigurationsraums implementiert. Die Multiview-3D-Rekonstruktion wird durch eine qualitative Analyse in Simulation evaluiert und auf ein Stereo-System ereignisbasierter Kameras übertragen. Zur Evaluierung der reaktiven kollisionsfreien Online-Planung wird ein Demonstrator mit einem industriellen Roboter genutzt. Dieser wird auch für eine vergleichende Studie zu sample-basierten Planern verwendet. Ergänzt wird
dies durch einen Benchmark von parallelen Hardwarelösungen wozu als Testszenario Bahnplanung in der Robotik gewählt wurde. Die Ergebnisse zeigen, dass die vorgeschlagenen neuronalen Lösungen einen effektiven Weg zur Realisierung einer Robotersteuerung für dynamische Szenarien darstellen. Diese Arbeit schafft eine Grundlage für neuronale Lösungen bei adaptiven Fertigungsprozesse, auch in Zusammenarbeit mit dem Menschen, ohne Einbußen bei Geschwindigkeit und Sicherheit. Damit ebnet sie den Weg für die Integration von dem Gehirn nachempfundener Hardware und Algorithmen in die Industrierobotik und MRK
Neuronal Specialization for Fine-Grained Distance Estimation using a Real-Time Bio-Inspired Stereo Vision System
The human binocular system performs very complex operations in real-time tasks thanks
to neuronal specialization and several specialized processing layers. For a classic computer vision
system, being able to perform the same operation requires high computational costs that, in many
cases, causes it to not work in real time: this is the case regarding distance estimation. This work
details the functionality of the biological processing system, as well as the neuromorphic engineering
research branch—the main purpose of which is to mimic neuronal processing. A distance estimation
system based on the calculation of the binocular disparities with specialized neuron populations is
developed. This system is characterized by several tests and executed in a real-time environment.
The response of the system proves the similarity between it and human binocular processing. Further,
the results show that the implemented system can work in a real-time environment, with a distance
estimation error of 15% (8% for the characterization tests).Ministerio de Ciencia, Innovación y Universidades TEC2016-77785-
- …