Real-Time High-Accuracy 2D Localization with Structured Patterns

Building over algorithms previously developed for digital pens, this article introduces a novel 2D localization technique for mobile robots, based on simple printed patterns. This method combines high absolute accuracy (below 0.3mm), unlimited scalability, low computational requirements (the presented open-source implementation runs at above 45Hz on a low-cost microcontroller) and low cost (below 30 Euros per device at prototype stage). The article first presents the underlying algorithms and localization pipeline. It then describes our reference hardware and software implementations, and finally evaluates the performance of this technique for mobile robots

3D Human Face Reconstruction and 2D Appearance Synthesis

3D human face reconstruction has been an extensive research for decades due to its wide applications, such as animation, recognition and 3D-driven appearance synthesis. Although commodity depth sensors are widely available in recent years, image based face reconstruction are significantly valuable as images are much easier to access and store. In this dissertation, we first propose three image-based face reconstruction approaches according to different assumption of inputs. In the first approach, face geometry is extracted from multiple key frames of a video sequence with different head poses. The camera should be calibrated under this assumption. As the first approach is limited to videos, we propose the second approach then focus on single image. This approach also improves the geometry by adding fine grains using shading cue. We proposed a novel albedo estimation and linear optimization algorithm in this approach. In the third approach, we further loose the constraint of the input image to arbitrary in the wild images. Our proposed approach can robustly reconstruct high quality model even with extreme expressions and large poses. We then explore the applicability of our face reconstructions on four interesting applications: video face beautification, generating personalized facial blendshape from image sequences, face video stylizing and video face replacement. We demonstrate great potentials of our reconstruction approaches on these real-world applications. In particular, with the recent surge of interests in VR/AR, it is increasingly common to see people wearing head-mounted displays. However, the large occlusion on face is a big obstacle for people to communicate in a face-to-face manner. Our another application is that we explore hardware/software solutions for synthesizing the face image with presence of HMDs. We design two setups (experimental and mobile) which integrate two near IR cameras and one color camera to solve this problem. With our algorithm and prototype, we can achieve photo-realistic results. We further propose a deep neutral network to solve the HMD removal problem considering it as a face inpainting problem. This approach doesn\u27t need special hardware and run in real-time with satisfying results

Algorithms for trajectory integration in multiple views

PhDThis thesis addresses the problem of deriving a coherent and accurate localization of moving objects from partial visual information when data are generated by cameras placed in di erent view angles with respect to the scene. The framework is built around applications of scene monitoring with multiple cameras. Firstly, we demonstrate how a geometric-based solution exploits the relationships between corresponding feature points across views and improves accuracy in object location. Then, we improve the estimation of objects location with geometric transformations that account for lens distortions. Additionally, we study the integration of the partial visual information generated by each individual sensor and their combination into one single frame of observation that considers object association and data fusion. Our approach is fully image-based, only relies on 2D constructs and does not require any complex computation in 3D space. We exploit the continuity and coherence in objects' motion when crossing cameras' elds of view. Additionally, we work under the assumption of planar ground plane and wide baseline (i.e. cameras' viewpoints are far apart). The main contributions are: i) the development of a framework for distributed visual sensing that accounts for inaccuracies in the geometry of multiple views; ii) the reduction of trajectory mapping errors using a statistical-based homography estimation; iii) the integration of a polynomial method for correcting inaccuracies caused by the cameras' lens distortion; iv) a global trajectory reconstruction algorithm that associates and integrates fragments of trajectories generated by each camera

From raw audio to a seamless mix : creating an automated DJ system for drum and bass

We present the open-source implementation of the first fully automatic and comprehensive DJ system, able to generate seamless music mixes using songs from a given library much like a human DJ does. The proposed system is built on top of several enhanced music information retrieval (MIR) techniques, such as for beat tracking, downbeat tracking, and structural segmentation, to obtain an understanding of the musical structure. Leveraging the understanding of the music tracks offered by these state-of-the-art MIR techniques, the proposed system surpasses existing automatic DJ systems both in accuracy and completeness. To the best of our knowledge, it is the first fully integrated solution that takes all basic Wing best practices into account, from beat and downbeat matching to identification of suitable cue points, determining a suitable cross-fade profile and compiling an interesting playlist that trades off innovation with continuity. To make this possible, we focused on one specific sub-genre of electronic dance music, namely Drum and Bass. This allowed us to exploit genre-specific properties, resulting in a more robust performance and tailored mixing behavior. Evaluation on a corpus of 160 Drum and Bass songs and an additional hold-out set of 220 songs shows that the used MIR algorithms can annotate 91% of the songs with fully correct annotations (tempo, beats, downbeats, and structure for cue points). On these songs, the proposed song selection process and the implemented Wing techniques enable the system to generate mixes of high quality, as confirmed by a subjective user test in which 18 Drum and Bass fans participated

A relocatable ocean modelling platform for downscaling to shelf-coastal areas to support disaster risk reduction

High-impact ocean weather events and climate extremes can have devastating effects on coastal zones and small islands. Marine Disaster Risk Reduction (DRR) is a systematic approach to such events, through which the risk of disaster can be identified, assessed and reduced. This can be done by improving ocean and atmosphere prediction models, data assimilation for better initial conditions and developing an efficient and sustainable impact forecasting methodology for Early Warnings Systems. A common user request during disaster remediation actions is for high-resolution information, which can be derived from easily deployable numerical models nested into operational larger-scale ocean models. The Structured and Unstructured Relocatable Ocean Model for Forecasting (SURF) enables users to rapidly deploy a nested high-resolution numerical model into larger-scale ocean forecasts. Rapidly downscaling the currents, sea level, temperature, and salinity fields is critical in supporting emergency responses to extreme events and natural hazards in the world’s oceans. The most important requirement in a relocatable model is to ensure that the interpolation of low-resolution ocean model fields (analyses and reanalyses) and atmospheric forcing is tested for different model domains. The provision of continuous ocean circulation forecasts through the Copernicus Marine Environment Monitoring Service (CMEMS) enables this testing. High-resolution SURF ocean circulation forecasts can be provided to specific application models such as oil spill fate and transport models, search and rescue trajectory models, and ship routing models requiring knowledge of meteooceanographic conditions. SURF was used to downscale CMEMS circulation analyses in four world ocean regions, and the high-resolution currents it can simulate for specific applications are examined. The SURF downscaled circulation fields show that the marine current resolutions affect the quality of the application models to be used for assessing disaster risks, particularly near coastal areas where the coastline geometry must be resolved through a numerical grid, and high-frequency coastal currents must be accurately simulated