Event-based Vision: A Survey

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world

Flyglow: Single-fly observations of simultaneous molecular and behavioural circadian oscillations in controls and an Alzheimer's model.

Circadian rhythms are essential for health and are frequently disturbed in disease. A full understanding of the causal relationships between behavioural and molecular circadian rhythms requires simultaneous longitudinal observations over time in individual organisms. Current experimental paradigms require the measurement of each rhythm separately across distinct populations of experimental organisms, rendering the comparability of the resulting datasets uncertain. We therefore developed FLYGLOW, an assay using clock gene controlled luciferase expression detected by exquisitely sensitive EM-CCD imaging, to enable simultaneous quantification of parameters including locomotor, sleep consolidation and molecular rhythms in single flies over days/weeks. FLYGLOW combines all the strengths of existing techniques, and also allows powerful multiparametric paired statistics. We found the age-related transition from rhythmicity to arrhythmicity for each parameter occurs unpredictably, with some flies showing loss of one or more rhythms during middle-age. Using single-fly correlation analysis of rhythm robustness and period we demonstrated the independence of the peripheral clock from circadian behaviours in wild type flies as well as in an Alzheimer's model. FLYGLOW is a useful tool for investigating the deterioration of behavioural and molecular rhythms in ageing and neurodegeneration. This approach may be applied more broadly within behavioural neurogenetics research

PIONIER: a visitor instrument for the VLTI

PIONIER is a 4-telescope visitor instrument for the VLTI, planned to see its first fringes in 2010. It combines four ATs or four UTs using a pairwise ABCD integrated optics combiner that can also be used in scanning mode. It provides low spectral resolution in H and K band. PIONIER is designed for imaging with a specific emphasis on fast fringe recording to allow closure-phases and visibilities to be precisely measured. In this work we provide the detailed description of the instrument and present its updated status.Comment: Proceedings of SPIE conference Optical and Infrared Interferometry II (Conference 7734) San Diego 201

Video processing techniques for the contactless investigation of large oscillations

The experimental acquisition of large vibrations presents various technical difficulties. Especially in the case of geometric nonlinearities, dealing with very flexible, very light structures causes minimal variations in mass or stiffness to affect severely the dynamical response. Thus, sensors' added masses change the behaviour of the structure with respect to the unloaded condition. Moreover, the most common tools regularly employed for acquisition in vibration analysis - that is to say, laser vibrometers and accelerometers - are often designed with small amplitudes in mind. Their recordings are known to lack accuracy when the investigated structure undergoes large or very large motions, due to geometrical reasons. Image-based measurement techniques offer a valid solution to this problem. Here, an ensemble of three video processing techniques are benchmarked against each other and tested as viable options for the non-contact dynamic characterisation of slender beam-like structures. The methods have been applied to the case study of an aluminium spar for a highly-flexible airwing prototype and compared to the measurements recorded by a laser velocimeter and several Raspberry PI Inertial Measurement Units (IMUs), which also proved to be minimally invasive

NELIOTA: The wide-field, high-cadence lunar monitoring system at the prime focus of the Kryoneri telescope

We present the technical specifications and first results of the ESA-funded, lunar monitoring project "NELIOTA" (NEO Lunar Impacts and Optical TrAnsients) at the National Observatory of Athens, which aims to determine the size-frequency distribution of small Near-Earth Objects (NEOs) via detection of impact flashes on the surface of the Moon. For the purposes of this project a twin camera instrument was specially designed and installed at the 1.2 m Kryoneri telescope utilizing the fast-frame capabilities of scientific Complementary Metal-Oxide Semiconductor detectors (sCMOS). The system provides a wide field-of-view (17.0' $\times$ 14.4') and simultaneous observations in two photometric bands (R and I), reaching limiting magnitudes of 18.7 mag in 10 sec in both bands at a 2.5 signal-to-noise level. This makes it a unique instrument that can be used for the detection of NEO impacts on the Moon, as well as for any astronomy projects that demand high-cadence multicolor observations. The wide field-of-view ensures that a large portion of the Moon is observed, while the simultaneous, high-cadence, monitoring in two photometric bands makes possible, for the first time, the determination of the temperatures of the impacts on the Moon's surface and the validation of the impact flashes from a single site. Considering the varying background level on the Moon's surface we demonstrate that the NELIOTA system can detect NEO impact flashes at a 2.5 signal-to-noise level of ~12.4 mag in the I-band and R-band for observations made at low lunar phases ~0.1. We report 31 NEO impact flashes detected during the first year of the NELIOTA campaign. The faintest flash was at 11.24 mag in the R-band (about two magnitudes fainter than ever observed before) at lunar phase 0.32. Our observations suggest a detection rate of $1.96 \times 10^{-7}$ events $km^{-2} h^{-1}$.Comment: Accepted for publication in A&

Distributed, Low-Cost, Non-Expert Fine Dust Sensing with Smartphones

Diese Dissertation behandelt die Frage, wie mit kostengünstiger Sensorik Feinstäube in hoher zeitlicher und räumlicher Auflösung gemessen werden können. Dazu wird ein neues Sensorsystem auf Basis kostengünstiger off-the-shelf-Sensoren und Smartphones vorgestellt, entsprechende robuste Algorithmen zur Signalverarbeitung entwickelt und Erkenntnisse zur Interaktions-Gestaltung für die Messung durch Laien präsentiert. Atmosphärische Aerosolpartikel stellen im globalen Maßstab ein gravierendes Problem für die menschliche Gesundheit dar, welches sich in Atemwegs- und Herz-Kreislauf-Erkrankungen äußert und eine Verkürzung der Lebenserwartung verursacht. Bisher wird Luftqualität ausschließlich anhand von Daten relativ weniger fester Messstellen beurteilt und mittels Modellen auf eine hohe räumliche Auflösung gebracht, so dass deren Repräsentativität für die flächendeckende Exposition der Bevölkerung ungeklärt bleibt. Es ist unmöglich, derartige räumliche Abbildungen mit den derzeitigen statischen Messnetzen zu bestimmen. Bei der gesundheitsbezogenen Bewertung von Schadstoffen geht der Trend daher stark zu räumlich differenzierenden Messungen. Ein vielversprechender Ansatz um eine hohe räumliche und zeitliche Abdeckung zu erreichen ist dabei Participatory Sensing, also die verteilte Messung durch Endanwender unter Zuhilfenahme ihrer persönlichen Endgeräte. Insbesondere für Luftqualitätsmessungen ergeben sich dabei eine Reihe von Herausforderungen - von neuer Sensorik, die kostengünstig und tragbar ist, über robuste Algorithmen zur Signalauswertung und Kalibrierung bis hin zu Anwendungen, die Laien bei der korrekten Ausführung von Messungen unterstützen und ihre Privatsphäre schützen. Diese Arbeit konzentriert sich auf das Anwendungsszenario Partizipatorischer Umweltmessungen, bei denen Smartphone-basierte Sensorik zum Messen der Umwelt eingesetzt wird und üblicherweise Laien die Messungen in relativ unkontrollierter Art und Weise ausführen. Die Hauptbeiträge hierzu sind: 1. Systeme zum Erfassen von Feinstaub mit Smartphones (Low-cost Sensorik und neue Hardware): Ausgehend von früher Forschung zur Feinstaubmessung mit kostengünstiger off-the-shelf-Sensorik wurde ein Sensorkonzept entwickelt, bei dem die Feinstaub-Messung mit Hilfe eines passiven Aufsatzes auf einer Smartphone-Kamera durchgeführt wird. Zur Beurteilung der Sensorperformance wurden teilweise Labor-Messungen mit künstlich erzeugtem Staub und teilweise Feldevaluationen in Ko-Lokation mit offiziellen Messstationen des Landes durchgeführt. 2. Algorithmen zur Signalverarbeitung und Auswertung: Im Zuge neuer Sensordesigns werden Kombinationen bekannter OpenCV-Bildverarbeitungsalgorithmen (Background-Subtraction, Contour Detection etc.) zur Bildanalyse eingesetzt. Der resultierende Algorithmus erlaubt im Gegensatz zur Auswertung von Lichtstreuungs-Summensignalen die direkte Zählung von Partikeln anhand individueller Lichtspuren. Ein zweiter neuartiger Algorithmus nutzt aus, dass es bei solchen Prozessen ein signalabhängiges Rauschen gibt, dessen Verhältnis zum Mittelwert des Signals bekannt ist. Dadurch wird es möglich, Signale die von systematischen unbekannten Fehlern betroffen sind auf Basis ihres Rauschens zu analysieren und das "echte" Signal zu rekonstruieren. 3. Algorithmen zur verteilten Kalibrierung bei gleichzeitigem Schutz der Privatsphäre: Eine Herausforderung partizipatorischer Umweltmessungen ist die wiederkehrende Notwendigkeit der Sensorkalibrierung. Dies beruht zum einen auf der Instabilität insbesondere kostengünstiger Luftqualitätssensorik und zum anderen auf der Problematik, dass Endbenutzern die Mittel für eine Kalibrierung üblicherweise fehlen. Bestehende Ansätze zur sogenannten Cross-Kalibrierung von Sensoren, die sich in Ko-Lokation mit einer Referenzstation oder anderen Sensoren befinden, wurden auf Daten günstiger Feinstaubsensorik angewendet sowie um Mechanismen erweitert, die eine Kalibrierung von Sensoren untereinander ohne Preisgabe privater Informationen (Identität, Ort) ermöglicht. 4. Mensch-Maschine-Interaktions-Gestaltungsrichtlinien für Participatory Sensing: Auf Basis mehrerer kleiner explorativer Nutzerstudien wurde empirisch eine Taxonomie der Fehler erstellt, die Laien beim Messen von Umweltinformationen mit Smartphones machen. Davon ausgehend wurden mögliche Gegenmaßnahmen gesammelt und klassifiziert. In einer großen summativen Studie mit einer hohen Teilnehmerzahl wurde der Effekt verschiedener dieser Maßnahmen durch den Vergleich vier unterschiedlicher Varianten einer App zur partizipatorischen Messung von Umgebungslautstärke evaluiert. Die dabei gefundenen Erkenntnisse bilden die Basis für Richtlinien zur Gestaltung effizienter Nutzerschnittstellen für Participatory Sensing auf Mobilgeräten. 5. Design Patterns für Participatory Sensing Games auf Mobilgeräten (Gamification): Ein weiterer erforschter Ansatz beschäftigt sich mit der Gamifizierung des Messprozesses um Nutzerfehler durch den Einsatz geeigneter Spielmechanismen zu minimieren. Dabei wird der Messprozess z.B. in ein Smartphone-Spiel (sog. Minigame) eingebettet, das im Hintergrund bei geeignetem Kontext die Messung durchführt. Zur Entwicklung dieses "Sensified Gaming" getauften Konzepts wurden Kernaufgaben im Participatory Sensing identifiziert und mit aus der Literatur zu sammelnden Spielmechanismen (Game Design Patterns) gegenübergestellt

Review of machine-vision based methodologies for displacement measurement in civil structures

This is the author accepted manuscript. The final version is available from Springer Verlag via the DOI in this record.Vision-based systems are promising tools for displacement measurement in civil structures, possessing advantages over traditional displacement sensors in instrumentation cost, installation efforts and measurement capacity in terms of frequency range and spatial resolution. Approximately one hundred papers to date have appeared on this subject, investigating topics like: system development and improvement, the viability on field applications and the potential for structural condition assessment. The main contribution of this paper is to present a literature review of vision-based displacement measurement, from the perspectives of methodologies and applications. Video processing procedures in this paper are summarised as a three-component framework, camera calibration, target tracking and structural displacement calculation. Methods for each component are presented in principle, with discussions about the relative advantages and limitations. Applications in the two most active fields: bridge deformation and cable vibration measurement are examined followed by a summary of field challenges observed in monitoring tests. Important gaps requiring further investigation are presented e.g. robust tracking methods, non-contact sensing and measurement accuracy evaluation in field conditions

A microfluidic study of transient flow states in permeable media using fluorescent particle image velocimetry

Velocity fields in flow in permeable media are of great importance to many subsurface processes such as geologic storage of CO2 , oil and gas extraction, and geothermal systems. Steady-state flow is characterized by velocity fields that do not change significantly over time. The flow field transitions to a new steady state once it experiences a disturbance such as a change in flow rate or in pressure gradient. This transition is often assumed to be instantaneous, which justifies the expression of constitutive relations as functions of instantaneous phase saturations. This work examines the evolution of velocity fields in a surrogate quasi-2D permeable medium using a microfluidic device, a microscopy system, and a high-speed camera. Tracer particles are injected into the medium along with Deionized water. The evolution of the velocity field is examined by tracing these particles in the captured images using the standard high-density particle image velocimetry algorithm founded on cross-correlation. The results suggest that the transition between steady states for an incompressible fluid takes a finite and non-negligible amount of time that is independent of the magnitude of the change in pressure gradient. The existence of transient states and the nature of the response during these states are readily interpreted by the principle of least action where flow gradually establishes an optimal configuration such that energy dissipation is minimized. The findings provide evidence against the applicability of the assumption that flowing phases relax instantaneously to their steady states and, hence, against the accuracy of the classical multiphase extension of Darcy’s law.Cited as: Sun, J., Li, Z., Furtado, F., Aryana, S. A. A microfluidic study of transient flow states in permeable media using fluorescent particle image velocimetry. Capillarity, 2021, 4(4): 76-86, doi: 10.46690/capi.2021.04.0