Smart environment monitoring through micro unmanned aerial vehicles

In recent years, the improvements of small-scale Unmanned Aerial Vehicles (UAVs) in terms of flight time, automatic control, and remote transmission are promoting the development of a wide range of practical applications. In aerial video surveillance, the monitoring of broad areas still has many challenges due to the achievement of different tasks in real-time, including mosaicking, change detection, and object detection. In this thesis work, a small-scale UAV based vision system to maintain regular surveillance over target areas is proposed. The system works in two modes. The first mode allows to monitor an area of interest by performing several flights. During the first flight, it creates an incremental geo-referenced mosaic of an area of interest and classifies all the known elements (e.g., persons) found on the ground by an improved Faster R-CNN architecture previously trained. In subsequent reconnaissance flights, the system searches for any changes (e.g., disappearance of persons) that may occur in the mosaic by a histogram equalization and RGB-Local Binary Pattern (RGB-LBP) based algorithm. If present, the mosaic is updated. The second mode, allows to perform a real-time classification by using, again, our improved Faster R-CNN model, useful for time-critical operations. Thanks to different design features, the system works in real-time and performs mosaicking and change detection tasks at low-altitude, thus allowing the classification even of small objects. The proposed system was tested by using the whole set of challenging video sequences contained in the UAV Mosaicking and Change Detection (UMCD) dataset and other public datasets. The evaluation of the system by well-known performance metrics has shown remarkable results in terms of mosaic creation and updating, as well as in terms of change detection and object detection

JigsawNet: Shredded Image Reassembly using Convolutional Neural Network and Loop-based Composition

This paper proposes a novel algorithm to reassemble an arbitrarily shredded image to its original status. Existing reassembly pipelines commonly consist of a local matching stage and a global compositions stage. In the local stage, a key challenge in fragment reassembly is to reliably compute and identify correct pairwise matching, for which most existing algorithms use handcrafted features, and hence, cannot reliably handle complicated puzzles. We build a deep convolutional neural network to detect the compatibility of a pairwise stitching, and use it to prune computed pairwise matches. To improve the network efficiency and accuracy, we transfer the calculation of CNN to the stitching region and apply a boost training strategy. In the global composition stage, we modify the commonly adopted greedy edge selection strategies to two new loop closure based searching algorithms. Extensive experiments show that our algorithm significantly outperforms existing methods on solving various puzzles, especially those challenging ones with many fragment pieces

Image stitching algorithm based on feature extraction

This paper proposes a novel edge-based stitching method to detect moving objects and construct\ud mosaics from images. The method is a coarse-to-fine scheme which first estimates a\ud good initialization of camera parameters with two complementary methods and then refines\ud the solution through an optimization process. The two complementary methods are the edge\ud alignment and correspondence-based approaches, respectively. The edge alignment method\ud estimates desired image translations by checking the consistencies of edge positions between\ud images. This method has better capabilities to overcome larger displacements and lighting variations\ud between images. The correspondence-based approach estimates desired parameters from\ud a set of correspondences by using a new feature extraction scheme and a new correspondence\ud building method. The method can solve more general camera motions than the edge alignment\ud method. Since these two methods are complementary to each other, the desired initial estimate\ud can be obtained more robustly. After that, a Monte-Carlo style method is then proposed for\ud integrating these two methods together. In this approach, a grid partition scheme is proposed to\ud increase the accuracy of each try for finding the correct parameters. After that, an optimization\ud process is then applied to refine the above initial parameters. Different from other optimization\ud methods minimizing errors on the whole images, the proposed scheme minimizes errors only on\ud positions of features points. Since the found initialization is very close to the exact solution and\ud only errors on feature positions are considered, the optimization process can be achieved very\ud quickly. Experimental results are provided to verify the superiority of the proposed method

Image Stitching for UAV remote sensing application

The objective of the project is to write an algorithm that is able to join top view images to create a big map. The project is done in the School of Castelldefels of UPC, within the research laboratory Icarus of EETAC Faculty. The goal of the project is to detect an area of this map, thanks to the analysis of this images. The images are taken by the two camera aboard on an Unmanned Aerial Vehicle (UAV) built by the Icarus group leaded by Enric Pastor. The implemented code is uploaded in Upc' svn at the adress: https://svn.fib.upc.es/svn/vincenzo.can

MusA: Using Indoor Positioning and Navigation to Enhance Cultural Experiences in a museum

In recent years there has been a growing interest into the use of multimedia mobile guides in museum environments. Mobile devices have the capabilities to detect the user context and to provide pieces of information suitable to help visitors discovering and following the logical and emotional connections that develop during the visit. In this scenario, location based services (LBS) currently represent an asset, and the choice of the technology to determine users' position, combined with the definition of methods that can effectively convey information, become key issues in the design process. In this work, we present MusA (Museum Assistant), a general framework for the development of multimedia interactive guides for mobile devices. Its main feature is a vision-based indoor positioning system that allows the provision of several LBS, from way-finding to the contextualized communication of cultural contents, aimed at providing a meaningful exploration of exhibits according to visitors' personal interest and curiosity. Starting from the thorough description of the system architecture, the article presents the implementation of two mobile guides, developed to respectively address adults and children, and discusses the evaluation of the user experience and the visitors' appreciation of these application

A method for three-dimensional reconstruction of a train accident scene using photographs

Railway accidents that usually cause numerous property and life losses occurred in recent years all around the world. In general, resources such as financial supports and incident rescue programs are required to minimize the losses after an accident. Due to lack of comprehensive information collected from accident sites, most railway emergency management departments face a predicament in setting up rescue schemes. To tackle the issue, realistic three-dimensional virtual accident scene reconstruction technology is developed, which provides and visualises supplementary materials and information about a train accident and can offer assistance to emergency crews when making decisions. We propose a photo-based three-dimensional reconstruction framework of vehicles for measuring the positions and poses of carriages involved in an accident. We implement and examine two case studies to validate this reconstruction method, which performs well in the assigned tasks

Automatic Workflow for Narrow-Band Laryngeal Video Stitching

In narrow band (NB) laryngeal endoscopy, the clinician usually positions the endoscope near the tissue for a correct inspection of possible vascular pattern alterations, indicative of laryngeal malignancies. The video is usually reviewed many times to refine the diagnosis, resulting in loss of time since the salient frames of the video are mixed with blurred, noisy, and redundant frames caused by the endoscope movements. The aim of this work is to provide to the clinician a unique larynx panorama, obtained through an automatic frame selection strategy to discard non-informative frames. Anisotropic diffusion filtering was exploited to lower the noise level while encouraging the selection of meaningful image features, and a feature-based stitching approach was carried out to generate the panorama. The frame selection strategy, tested on on six pathological NB endoscopic videos, was compared with standard strategies, as uniform and random sampling, showing higher performance of the subsequent stitching procedure, both visually, in terms of vascular structure preservation, and numerically, through a blur estimation metric