Online learning and detection of faces with low human supervision

The final publication is available at link.springer.comWe present an efficient,online,and interactive approach for computing a classifier, called Wild Lady Ferns (WiLFs), for face learning and detection using small human supervision. More precisely, on the one hand, WiLFs combine online boosting and extremely randomized trees (Random Ferns) to compute progressively an efficient and discriminative classifier. On the other hand, WiLFs use an interactive human-machine approach that combines two complementary learning strategies to reduce considerably the degree of human supervision during learning. While the first strategy corresponds to query-by-boosting active learning, that requests human assistance over difficult samples in function of the classifier confidence, the second strategy refers to a memory-based learning which uses ¿ Exemplar-based Nearest Neighbors (¿ENN) to assist automatically the classifier. A pre-trained Convolutional Neural Network (CNN) is used to perform ¿ENN with high-level feature descriptors. The proposed approach is therefore fast (WilFs run in 1 FPS using a code not fully optimized), accurate (we obtain detection rates over 82% in complex datasets), and labor-saving (human assistance percentages of less than 20%). As a byproduct, we demonstrate that WiLFs also perform semi-automatic annotation during learning, as while the classifier is being computed, WiLFs are discovering faces instances in input images which are used subsequently for training online the classifier. The advantages of our approach are demonstrated in synthetic and publicly available databases, showing comparable detection rates as offline approaches that require larger amounts of handmade training data.Peer ReviewedPostprint (author's final draft

Learning nuisances to track pedestrians in autonomous vehicles

Autonomous vehicles rely on an accurate perception module. One of the fundamental challenges is to efficiently track pedestrians surrounding a vehicle to anticipate risky situations. Over the past decades, researchers have formulated the tracking problem as a data association one where they proposed various representations aiming for invariance to nuisances such as viewpoint changes, body deformation, object occlusion, and illumination changes. However, these methods still suffer to address abrupt changes since they do not explicitly model the nature of the nuisances. In this work, we propose to train a classifier that recognizes these nuisances, more specifically rotational body deformation of pedestrians. We aim to detect deformations as a method to find a good representation that will lead to better tracking of pedestrians as well as other tasks

Robust and real-time hand detection and tracking in monocular video

In recent years, personal computing devices such as laptops, tablets and smartphones have become ubiquitous. Moreover, intelligent sensors are being integrated into many consumer devices such as eyeglasses, wristwatches and smart televisions. With the advent of touchscreen technology, a new human-computer interaction (HCI) paradigm arose that allows users to interface with their device in an intuitive manner. Using simple gestures, such as swipe or pinch movements, a touchscreen can be used to directly interact with a virtual environment. Nevertheless, touchscreens still form a physical barrier between the virtual interface and the real world. An increasingly popular field of research that tries to overcome this limitation, is video based gesture recognition, hand detection and hand tracking. Gesture based interaction allows the user to directly interact with the computer in a natural manner by exploring a virtual reality using nothing but his own body language. In this dissertation, we investigate how robust hand detection and tracking can be accomplished under real-time constraints. In the context of human-computer interaction, real-time is defined as both low latency and low complexity, such that a complete video frame can be processed before the next one becomes available. Furthermore, for practical applications, the algorithms should be robust to illumination changes, camera motion, and cluttered backgrounds in the scene. Finally, the system should be able to initialize automatically, and to detect and recover from tracking failure. We study a wide variety of existing algorithms, and propose significant improvements and novel methods to build a complete detection and tracking system that meets these requirements. Hand detection, hand tracking and hand segmentation are related yet technically different challenges. Whereas detection deals with finding an object in a static image, tracking considers temporal information and is used to track the position of an object over time, throughout a video sequence. Hand segmentation is the task of estimating the hand contour, thereby separating the object from its background. Detection of hands in individual video frames allows us to automatically initialize our tracking algorithm, and to detect and recover from tracking failure. Human hands are highly articulated objects, consisting of finger parts that are connected with joints. As a result, the appearance of a hand can vary greatly, depending on the assumed hand pose. Traditional detection algorithms often assume that the appearance of the object of interest can be described using a rigid model and therefore can not be used to robustly detect human hands. Therefore, we developed an algorithm that detects hands by exploiting their articulated nature. Instead of resorting to a template based approach, we probabilistically model the spatial relations between different hand parts, and the centroid of the hand. Detecting hand parts, such as fingertips, is much easier than detecting a complete hand. Based on our model of the spatial configuration of hand parts, the detected parts can be used to obtain an estimate of the complete hand's position. To comply with the real-time constraints, we developed techniques to speed-up the process by efficiently discarding unimportant information in the image. Experimental results show that our method is competitive with the state-of-the-art in object detection while providing a reduction in computational complexity with a factor 1 000. Furthermore, we showed that our algorithm can also be used to detect other articulated objects such as persons or animals and is therefore not restricted to the task of hand detection. Once a hand has been detected, a tracking algorithm can be used to continuously track its position in time. We developed a probabilistic tracking method that can cope with uncertainty caused by image noise, incorrect detections, changing illumination, and camera motion. Furthermore, our tracking system automatically determines the number of hands in the scene, and can cope with hands entering or leaving the video canvas. We introduced several novel techniques that greatly increase tracking robustness, and that can also be applied in other domains than hand tracking. To achieve real-time processing, we investigated several techniques to reduce the search space of the problem, and deliberately employ methods that are easily parallelized on modern hardware. Experimental results indicate that our methods outperform the state-of-the-art in hand tracking, while providing a much lower computational complexity. One of the methods used by our probabilistic tracking algorithm, is optical flow estimation. Optical flow is defined as a 2D vector field describing the apparent velocities of objects in a 3D scene, projected onto the image plane. Optical flow is known to be used by many insects and birds to visually track objects and to estimate their ego-motion. However, most optical flow estimation methods described in literature are either too slow to be used in real-time applications, or are not robust to illumination changes and fast motion. We therefore developed an optical flow algorithm that can cope with large displacements, and that is illumination independent. Furthermore, we introduce a regularization technique that ensures a smooth flow-field. This regularization scheme effectively reduces the number of noisy and incorrect flow-vector estimates, while maintaining the ability to handle motion discontinuities caused by object boundaries in the scene. The above methods are combined into a hand tracking framework which can be used for interactive applications in unconstrained environments. To demonstrate the possibilities of gesture based human-computer interaction, we developed a new type of computer display. This display is completely transparent, allowing multiple users to perform collaborative tasks while maintaining eye contact. Furthermore, our display produces an image that seems to float in thin air, such that users can touch the virtual image with their hands. This floating imaging display has been showcased on several national and international events and tradeshows. The research that is described in this dissertation has been evaluated thoroughly by comparing detection and tracking results with those obtained by state-of-the-art algorithms. These comparisons show that the proposed methods outperform most algorithms in terms of accuracy, while achieving a much lower computational complexity, resulting in a real-time implementation. Results are discussed in depth at the end of each chapter. This research further resulted in an international journal publication; a second journal paper that has been submitted and is under review at the time of writing this dissertation; nine international conference publications; a national conference publication; a commercial license agreement concerning the research results; two hardware prototypes of a new type of computer display; and a software demonstrator

Video-based Pedestrian Intention Recognition and Path Prediction for Advanced Driver Assistance Systems

Fortgeschrittene Fahrerassistenzsysteme (FAS) spielen eine sehr wichtige Rolle in zukünftigen Fahrzeugen um die Sicherheit für den Fahrer, der Fahrgäste und ungeschützte Verkehrsteilnehmer wie Fußgänger und Radfahrer zu erhöhen. Diese Art von Systemen versucht in begrenztem Rahmen, Zusammenstöße in gefährlichen Situationen mit einem unaufmerksamen Fahrer und Fußgänger durch das Auslösen einer automatischen Notbremsung zu vermeiden. Aufgrund der hohen Variabilität an Fußgängerbewegungsmustern werden bestehende Systeme in einer konservativen Art und Weise konzipiert, um durch eine Restriktion auf beherrschbare Umgebungen mögliche Fehlauslöseraten drastisch zu reduzieren, wie z.B. in Szenarien in denen Fußgänger plötzlich anhalten und dadurch die Situation deeskalieren. Um dieses Problem zu überwinden, stellt eine zuverlässige Fußgängerabsichtserkennung und Pfad\-vorhersage einen großen Wert dar. In dieser Arbeit wird die gesamte Ablaufkette eines Stereo-Video basierten Systems zur Intentionsschätzung und Pfadvorhersage von Fußgängern beschrieben, welches in einer späteren Funktionsentscheidung für eine automatische Notbremsung verwendet wird. Im ersten von drei Hauptbestandteilen wird ein Echtzeit-Verfahren vorgeschlagen, das in niedrig aufgelösten Bildern aus komplexen und hoch dynamischen Innerstadt-Szenarien versucht, die Köpfe von Fußgängern zu lokalisieren und deren Pose zu schätzen. Einzelbild-basierte Schätzungen werden aus den Wahrscheinlichkeitsausgaben von acht angelernten Kopfposen-spezifischen Detektoren abgeleitet, die im Bildbereich eines Fußgängerkandidaten angewendet werden. Weitere Robustheit in der Kopflokalisierung wird durch Hinzunahme von Stereo-Tiefeninformation erreicht. Darüber hinaus werden die Kopfpositionen und deren Pose über die Zeit durch die Implementierung eines Partikelfilters geglättet. Für die Intentionsschätzung von Fußgängern wird die Verwendung eines robusten und leistungsstarken Ansatzes des Maschinellen Lernens in unterschiedlichen Szenarien untersucht. Dieser Ansatz ist in der Lage, für Zeitreihen von Beobachtungen, die inneren Unterstrukturen einer bestimmten Absichtsklasse zu modellieren und zusätzlich die extrinsische Dynamik zwischen unterschiedlichen Absichtsklassen zu erfassen. Das Verfahren integriert bedeutsame extrahierte Merkmale aus der Fußgängerdynamik sowie Kontextinformationen mithilfe der menschlichen Kopfpose. Zum Schluss wird ein Verfahren zur Pfadvorhersage vorgestellt, welches die Prädiktionsschritte eines Filters für multiple Bewegungsmodelle für einen Zeithorizont von ungefähr einer Sekunde durch Einbeziehung der geschätzten Fußgängerabsichten steuert. Durch Hilfestellungen für den Filter das geeignete Bewegungsmodell zu wählen, kann der resultierende Pfadprädiktionsfehler um ein signifikantes Maß reduziert werden. Eine Vielzahl von Szenarien wird behandelt, einschließlich seitlich querender oder anhaltender Fußgänger oder Personen, die zunächst entlang des Bürgersteigs gehen aber dann plötzlich in Richtung der Fahrbahn einbiegen

Second International Workshop on Harmonic Oscillators

The Second International Workshop on Harmonic Oscillators was held at the Hotel Hacienda Cocoyoc from March 23 to 25, 1994. The Workshop gathered 67 participants; there were 10 invited lecturers, 30 plenary oral presentations, 15 posters, and plenty of discussion divided into the five sessions of this volume. The Organizing Committee was asked by the chairman of several Mexican funding agencies what exactly was meant by harmonic oscillators, and for what purpose the new research could be useful. Harmonic oscillators - as we explained - is a code name for a family of mathematical models based on the theory of Lie algebras and groups, with applications in a growing range of physical theories and technologies: molecular, atomic, nuclear and particle physics; quantum optics and communication theory

EVOLUTION OF THE SUBCONTINENTAL LITHOSPHERE DURING MESOZOIC TETHYAN RIFTING: CONSTRAINTS FROM THE EXTERNAL LIGURIAN MANTLE SECTION (NORTHERN APENNINE, ITALY)

Our study is focussed on mantle bodies from the External Ligurian ophiolites, within the Monte Gavi and Monte Sant'Agostino areas. Here, two distinct pyroxenite-bearing mantle sections were recognized, mainly based on their plagioclase-facies evolution. The Monte Gavi mantle section is nearly undeformed and records reactive melt infiltration under plagioclase-facies conditions. This process involved both peridotites (clinopyroxene-poor lherzolites) and enclosed spinel pyroxenite layers, and occurred at 0.7–0.8 GPa. In the Monte Gavi peridotites and pyroxenites, the spinel-facies clinopyroxene was replaced by Ca-rich plagioclase and new orthopyroxene, typically associated with secondary clinopyroxene. The reactive melt migration caused increase of TiO2 contents in relict clinopyroxene and spinel, with the latter also recording a Cr2O3 increase. In the Monte Gavi peridotites and pyroxenites, geothermometers based on slowly diffusing elements (REE and Y) record high temperature conditions (1200-1250 °C) related to the melt infiltration event, followed by subsolidus cooling until ca. 900°C. The Monte Sant'Agostino mantle section is characterized by widespread ductile shearing with no evidence of melt infiltration. The deformation recorded by the Monte Sant'Agostino peridotites (clinopyroxene-rich lherzolites) occurred at 750–800 °C and 0.3–0.6 GPa, leading to protomylonitic to ultramylonitic textures with extreme grain size reduction (10–50 μm). Compared to the peridotites, the enclosed pyroxenite layers gave higher temperature-pressure estimates for the plagioclase-facies re-equilibration (870–930 °C and 0.8–0.9 GPa). We propose that the earlier plagioclase crystallization in the pyroxenites enhanced strain localization and formation of mylonite shear zones in the entire mantle section. We subdivide the subcontinental mantle section from the External Ligurian ophiolites into three distinct domains, developed in response to the rifting evolution that ultimately formed a Middle Jurassic ocean-continent transition: (1) a spinel tectonite domain, characterized by subsolidus static formation of plagioclase, i.e. the Suvero mantle section (Hidas et al., 2020), (2) a plagioclase mylonite domain experiencing melt-absent deformation and (3) a nearly undeformed domain that underwent reactive melt infiltration under plagioclase-facies conditions, exemplified by the the Monte Sant'Agostino and the Monte Gavi mantle sections, respectively. We relate mantle domains (1) and (2) to a rifting-driven uplift in the late Triassic accommodated by large-scale shear zones consisting of anhydrous plagioclase mylonites. Hidas K., Borghini G., Tommasi A., Zanetti A. & Rampone E. 2021. Interplay between melt infiltration and deformation in the deep lithospheric mantle (External Liguride ophiolite, North Italy). Lithos 380-381, 105855

Impact of geogenic degassing on C-isotopic composition of dissolved carbon in karst systems of Greece

The Earth C-cycle is complex, where endogenic and exogenic sources are interconnected, operating in a multiple spatial and temporal scale (Lee et al., 2019). Non-volcanic CO2 degassing from active tectonic structures is one of the less defined components of this cycle (Frondini et al., 2019). Carbon mass-balance (Chiodini et al., 2000) is a useful tool to quantify the geogenic carbon output from regional karst hydrosystems. This approach has been demonstrated for central Italy and may be valid also for Greece, due to the similar geodynamic settings. Deep degassing in Greece has been ascertained mainly at hydrothermal and volcanic areas, but the impact of geogenic CO2 released by active tectonic areas has not yet been quantified. The main aim of this research is to investigate the possible deep degassing through the big karst aquifers of Greece. Since 2016, 156 karst springs were sampled along most of the Greek territory. To discriminate the sources of carbon, the analysis of the isotopic composition of carbon was carried out. δ13CTDIC values vary from -16.61 to -0.91‰ and can be subdivided into two groups characterized by (a) low δ13CTDIC, and (b) intermediate to high δ13CTDIC with a threshold value of -6.55‰. The composition of the first group can be related to the mixing of organic-derived CO2 and the dissolution of marine carbonates. Springs of the second group, mostly located close to Quaternary volcanic areas, are linked to possible carbon input from deep sources

Impact of Etna’s volcanic emission on major ions and trace elements composition of the atmospheric deposition

Mt. Etna, on the eastern coast of Sicily (Italy), is one of the most active volcanoes on the planet and it is widely recognized as a big source of volcanic gases (e.g., CO2 and SO2), halogens, and a lot of trace elements, to the atmosphere in the Mediterranean region. Especially during eruptive periods, Etna’s emissions can be dispersed over long distances and cover wide areas. A group of trace elements has been recently brought to attention for their possible environmental and human health impacts, the Technology-critical elements. The current knowledge about their geochemical cycles is still scarce, nevertheless, recent studies (Brugnone et al., 2020) evidenced a contribution from the volcanic activity for some of them (Te, Tl, and REE). In 2021, in the framework of the research project “Pianeta Dinamico”, by INGV, a network of 10 bulk collectors was implemented to collect, monthly, atmospheric deposition samples. Four of these collectors are located on the flanks of Mt. Etna, other two are in the urban area of Catania and three are in the industrial area of Priolo, all most of the time downwind of the main craters. The last one, close to Cesarò (Nebrodi Regional Park), represents the regional background. The research aims to produce a database on major ions and trace element compositions of the bulk deposition and here we report the values of the main physical-chemical parameters and the deposition fluxes of major ions and trace elements from the first year of research. The pH ranged from 3.1 to 7.7, with a mean value of 5.6, in samples from the Etna area, while it ranged between 5.2 and 7.6, with a mean value of 6.4, in samples from the other study areas. The EC showed values ranging from 5 to 1032 μS cm-1, with a mean value of 65 μS cm-1. The most abundant ions were Cl- and SO42- for anions, Na+ and Ca+ for cations, whose mean deposition fluxes, considering all sampling sites, were 16.6, 6.8, 8.4, and 6.0 mg m-2 d, respectively. The highest deposition fluxes of volcanic refractory elements, such as Al, Fe, and Ti, were measured in the Etna’s sites, with mean values of 948, 464, and 34.3 μg m-2 d-1, respectively, higher than those detected in the other sampling sites, further away from the volcanic source (26.2, 12.4, 0.5 μg m-2 d-1, respectively). The same trend was also observed for volatile elements of prevailing volcanic origin, such as Tl (0.49 μg m-2 d-1), Te (0.07 μg m-2 d-1), As (0.95 μg m-2 d-1), Se (1.92 μg m-2 d-1), and Cd (0.39 μg m-2 d-1). Our preliminary results show that, close to a volcanic area, volcanic emissions must be considered among the major contributors of ions and trace elements to the atmosphere. Their deposition may significantly impact the pedosphere, hydrosphere, and biosphere and directly or indirectly human health

Actas de las XXXIV Jornadas de Automática

Postprint (published version