Computer vision models in surveillance robotics

2009/2010In questa Tesi, abbiamo sviluppato algoritmi che usano l’informazione visiva per eseguire, in tempo reale, individuazione, riconoscimento e classificazione di oggetti in movimento, indipendentemente dalle condizioni ambientali e con l’accurattezza migliore. A tal fine, abbiamo sviluppato diversi concetti di visione artificial, cioè l'identificazione degli oggetti di interesse in tutta la scena visiva (monoculare o stereo), e la loro classificazione. Nel corso della ricerca, sono stati provati diversi approcci, inclusa l’individuazione di possibili candidati tramite la segmentazione di immagini con classificatori deboli e centroidi, algoritmi per la segmentazione di immagini rafforzate tramite informazioni stereo e riduzione del rumore, combinazione di popolari caratteristiche quali quelle invarianti a fattori di scala (SIFT) combinate con informazioni di distanza. Abbiamo sviluppato due grandi categorie di soluzioni associate al tipo di sistema usato. Con camera mobile, abbiamo favorito l’individuazione di oggetti conosciuti tramite scansione dell’immagine; con camera fissa abbiamo anche utilizzato algoritmi per l’individuazione degli oggetti in primo piano ed in movimento (foreground detection). Nel caso di “foreground detection”, il tasso di individuazione e classificazione aumenta se la qualita’ degli oggetti estratti e’ alta. Noi proponiamo metodi per ridurre gli effetti dell’ombra, illuminazione e movimenti ripetitivi prodotti dagli oggetti in movimento. Un aspetto importante studiato e’ la possibilita’ di usare algoritmi per l’individuazione di oggetti in movimento tramite camera mobile. Soluzioni efficienti stanno diventando sempre piu’ complesse, ma anche gli strumenti di calcolo per elaborare gli algoritmi sono piu’ potenti e negli anni recenti, le architetture delle schede video (GPU) offrono un grande potenziale. Abbiamo proposto una soluzione per architettura GPU di una gestione delle immagini di sfondo, al fine di aumentare le prestazioni di individuazione. In questa Tesi abbiamo studiato l’individuazione ed inseguimento di persone for applicazioni come la prevenzione di situazione di rischio (attraversamento delle strade), e conteggio per l’analisi del traffico. Noi abbiamo studiato questi problemi ed esplorato vari aspetti dell’individuazione delle persone, gruppi ed individuazione in scenari affollati. Comunque, in un ambiente generico, e’ impossibile predire la configurazione di oggetti che saranno catturati dalla telecamera. In questi casi, e’ richiesto di “astrarre il concetto” di oggetti. Con questo requisito in mente, abbiamo esplorato le proprieta’ dei metodi stocastici e mostrano che buoni tassi di classificazione possono essere ottenuti a condizione che l’insieme di addestramento sia abbastanza grande. Una struttura flessibile deve essere in grado di individuare le regioni in movimento e riconoscere gli oggetti di interesse. Abbiamo sviluppato una struttura per la gestione dei problemi di individuazione e classificazione. Rispetto ad altri metodi, i metodi proposti offrono una struttura flessibile per l’individuazione e classificazione degli oggetti, e che puo’ essere usata in modo efficiente in diversi ambienti interni ed esterni.XXII Cicl

High-statistics pedestrian dynamics on stairways and their probabilistic fundamental diagrams

Staircases play an essential role in crowd dynamics, allowing pedestrians to flow across large multi-level public facilities such as transportation hubs, and office buildings. Achieving a robust understanding of pedestrian behavior in these facilities is a key societal necessity. What makes this an outstanding scientific challenge is the extreme randomness intrinsic to pedestrian behavior. Any quantitative understanding necessarily needs to be probabilistic, including average dynamics and fluctuations. In this work, we analyze data from an unprecedentedly high statistics year-long pedestrian tracking campaign, in which we anonymously collected millions of trajectories across a staircase within Eindhoven train station (NL). Made possible thanks to a state-of-the-art, faster than real-time, computer vision approach hinged on 3D depth imaging, and YOLOv7-based depth localization. We consider both free-stream conditions, i.e. pedestrians walking in undisturbed, and trafficked conditions, uni/bidirectional flows. We report the position vs density, considering the crowd as a 'compressible' physical medium. We show how pedestrians willingly opt to occupy fewer space than available, accepting a certain degree of compressibility. This is a non-trivial physical feature of pedestrian dynamics and we introduce a novel way to quantify this effect. As density increases, pedestrians strive to keep a minimum distance d = 0.6 m from the person in front of them. Finally, we establish first-of-kind fully resolved probabilistic fundamental diagrams, where we model the pedestrian walking velocity as a mixture of a slow and fast-paced component. Notably, averages and modes of velocity distribution turn out to be substantially different. Our results, including probabilistic parametrizations based on few variables, are key towards improved facility design and realistic simulation of pedestrians on staircases

Contextual Human Trajectory Forecasting within Indoor Environments and Its Applications

A human trajectory is the likely path a human subject would take to get to a destination. Human trajectory forecasting algorithms try to estimate or predict this path. Such algorithms have wide applications in robotics, computer vision and video surveillance. Understanding the human behavior can provide useful information towards the design of these algorithms. Human trajectory forecasting algorithm is an interesting problem because the outcome is influenced by many factors, of which we believe that the destination, geometry of the environment, and the humans in it play a significant role. In addressing this problem, we propose a model to estimate the occupancy behavior of humans based on the geometry and behavioral norms. We also develop a trajectory forecasting algorithm that understands this occupancy and leverages it for trajectory forecasting in previously unseen geometries. The algorithm can be useful in a variety of applications. In this work, we show its utility in three applications, namely person re-identification, camera placement optimization, and human tracking. Experiments were performed with real world data and compared to state-of-the-art methods to assess the quality of the forecasting algorithm and the enhancement in the quality of the applications. Results obtained suggests a significant enhancement in the accuracy of trajectory forecasting and the computer vision applications.Computer Science, Department o

Advances in Monocular Exemplar-based Human Body Pose Analysis: Modeling, Detection and Tracking

Esta tesis contribuye en el análisis de la postura del cuerpo humano a partir de secuencias de imágenes adquiridas con una sola cámara. Esta temática presenta un amplio rango de potenciales aplicaciones en video-vigilancia, video-juegos o aplicaciones biomédicas. Las técnicas basadas en patrones han tenido éxito, sin embargo, su precisión depende de la similitud del punto de vista de la cámara y de las propiedades de la escena entre las imágenes de entrenamiento y las de prueba. Teniendo en cuenta un conjunto de datos de entrenamiento capturado mediante un número reducido de cámaras fijas, paralelas al suelo, se han identificado y analizado tres escenarios posibles con creciente nivel de dificultad: 1) una cámara estática paralela al suelo, 2) una cámara de vigilancia fija con un ángulo de visión considerablemente diferente, y 3) una secuencia de video capturada con una cámara en movimiento o simplemente una sola imagen estática

Text Extraction From Natural Scene: Methodology And Application

With the popularity of the Internet and the smart mobile device, there is an increasing demand for the techniques and applications of image/video-based analytics and information retrieval. Most of these applications can benefit from text information extraction in natural scene. However, scene text extraction is a challenging problem to be solved, due to cluttered background of natural scene and multiple patterns of scene text itself. To solve these problems, this dissertation proposes a framework of scene text extraction. Scene text extraction in our framework is divided into two components, detection and recognition. Scene text detection is to find out the regions containing text from camera captured images/videos. Text layout analysis based on gradient and color analysis is performed to extract candidates of text strings from cluttered background in natural scene. Then text structural analysis is performed to design effective text structural features for distinguishing text from non-text outliers among the candidates of text strings. Scene text recognition is to transform image-based text in detected regions into readable text codes. The most basic and significant step in text recognition is scene text character (STC) prediction, which is multi-class classification among a set of text character categories. We design robust and discriminative feature representations for STC structure, by integrating multiple feature descriptors, coding/pooling schemes, and learning models. Experimental results in benchmark datasets demonstrate the effectiveness and robustness of our proposed framework, which obtains better performance than previously published methods. Our proposed scene text extraction framework is applied to 4 scenarios, 1) reading print labels in grocery package for hand-held object recognition; 2) combining with car detection to localize license plate in camera captured natural scene image; 3) reading indicative signage for assistant navigation in indoor environments; and 4) combining with object tracking to perform scene text extraction in video-based natural scene. The proposed prototype systems and associated evaluation results show that our framework is able to solve the challenges in real applications

COACHES: an assistance multi-robot system in public areas

In this paper, we present a robust system of self-directed autonomous robots evolving in a complex and public spaces and interacting with people. This system integrates highlevel skills of environment modeling using knowledge-based modeling and reasoning and scene understanding with robust image and video analysis, distributed autonomous decisionmaking using Markov decision process and Petri-Net planning, short-term interacting with humans and robust and safe navigation in overcrowding spaces. This system has been deployed in a variety of public environments such as a shopping mall, a center of congress and in a lab to assist people and visitors. The results are very satisfying showing the effectiveness of the system and going beyond just a simple proof of concepts

Portable Robotic Navigation Aid for the Visually Impaired

This dissertation aims to address the limitations of existing visual-inertial (VI) SLAM methods - lack of needed robustness and accuracy - for assistive navigation in a large indoor space. Several improvements are made to existing SLAM technology, and the improved methods are used to enable two robotic assistive devices, a robot cane, and a robotic object manipulation aid, for the visually impaired for assistive wayfinding and object detection/grasping. First, depth measurements are incorporated into the optimization process for device pose estimation to improve the success rate of VI SLAM\u27s initialization and reduce scale drift. The improved method, called depth-enhanced visual-inertial odometry (DVIO), initializes itself immediately as the environment\u27s metric scale can be derived from the depth data. Second, a hybrid PnP (perspective n-point) method is introduced for a more accurate estimation of the pose change between two camera frames by using the 3D data from both frames. Third, to implement DVIO on a smartphone with variable camera intrinsic parameters (CIP), a method called CIP-VMobile is devised to simultaneously estimate the intrinsic parameters and motion states of the camera. CIP-VMobile estimates in real time the CIP, which varies with the smartphone\u27s pose due to the camera\u27s optical image stabilization mechanism, resulting in more accurate device pose estimates. Various experiments are performed to validate the VI-SLAM methods with the two robotic assistive devices. Beyond these primary objectives, SM-SLAM is proposed as a potential extension for the existing SLAM methods in dynamic environments. This forward-looking exploration is premised on the potential that incorporating dynamic object detection capabilities in the front-end could improve SLAM\u27s overall accuracy and robustness. Various experiments have been conducted to validate the efficacy of this newly proposed method, using both public and self-collected datasets. The results obtained substantiate the viability of this innovation, leaving a deeper investigation for future work