Computer Vision and Image Understanding xxx

Abstract 13 This paper presents a panoramic virtual stereo vision approach to the problem of detecting 14 and localizing multiple moving objects (e.g., humans) in an indoor scene. Two panoramic 15 cameras, residing on different mobile platforms, compose a virtual stereo sensor with a flexible 16 baseline. A novel ''mutual calibration'' algorithm is proposed, where panoramic cameras on 17 two cooperative moving platforms are dynamically calibrated by looking at each other. A de-18 tailed numerical analysis of the error characteristics of the panoramic virtual stereo vision 19 (mutual calibration error, stereo matching error, and triangulation error) is given to derive 20 rules for optimal view planning. Experimental results are discussed for detecting and localizing 21 multiple humans in motion using two cooperative robot platforms. 2

Geomatics and Forensic: Progress and Challenges

Since graphics hold qualitative and quantitative information of complex crime scenes, it becomes a basic key to develop hypothesis in police investigations and also to prove these hypotheses in court. Forensic analysis involves tasks of scene information mining as well as its reconstruction in order to extract elements for explanatory police test or to show forensic evidence in legal proceedings. Currently, the combination of sensors and technologies allows the integration of spatial data and the generation of virtual infographic products (orthoimages, solid images, point clouds, cross‐sections, etc.) which are extremely attractive. These products, which successfully retain accurate 3D metric information, are revolutionizing dimensional reconstruction of objects and crime scenes. Thus, it can be said that the reconstruction and 3D visualization of complex scenes are one of the main challenges for the international scientific community. To overcome this challenge, techniques related with computer vision, computer graphics and geomatics work closely. This chapter reviews a set of geomatic techniques, applied to improve infographic forensic products, and its evolution. The integration of data from different sensors whose final purpose is 3D accurate modelling is also described. As we move into a highly active research area, where there are still many uncertainties to be resolved, the final section addresses these challenges and outlines future perspectives

Large Area 3-D Reconstructions from Underwater Optical Surveys

Robotic underwater vehicles are regularly performing vast optical surveys of the ocean floor. Scientists value these surveys since optical images offer high levels of detail and are easily interpreted by humans. Unfortunately, the coverage of a single image is limited by absorption and backscatter while what is generally desired is an overall view of the survey area. Recent works on underwater mosaics assume planar scenes and are applicable only to situations without much relief. We present a complete and validated system for processing optical images acquired from an underwater robotic vehicle to form a 3D reconstruction of the ocean floor. Our approach is designed for the most general conditions of wide-baseline imagery (low overlap and presence of significant 3D structure) and scales to hundreds or thousands of images. We only assume a calibrated camera system and a vehicle with uncertain and possibly drifting pose information (e.g., a compass, depth sensor, and a Doppler velocity log). Our approach is based on a combination of techniques from computer vision, photogrammetry, and robotics. We use a local to global approach to structure from motion, aided by the navigation sensors on the vehicle to generate 3D sub-maps. These sub-maps are then placed in a common reference frame that is refined by matching overlapping sub-maps. The final stage of processing is a bundle adjustment that provides the 3D structure, camera poses, and uncertainty estimates in a consistent reference frame. We present results with ground truth for structure as well as results from an oceanographic survey over a coral reef.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/86036/1/opizarro-12.pd

3D Fine-scale Terrain Variables from Underwater Photogrammetry: A New Approach to Benthic Microhabitat Modeling in a Circalittoral Rocky Shelf

The relationship between 3D terrain complexity and fine-scale localization and distribution of species is poorly understood. Here we present a very fine-scale 3D reconstruction model of three zones of circalittoral rocky shelf in the Bay of Biscay. Detailed terrain variables are extracted from 3D models using a structure-from-motion (SfM) approach applied to ROTV images. Significant terrain variables that explain species location were selected using general additive models (GAMs) and micro-distribution of the species were predicted. Two models combining BPI, curvature and rugosity can explain 55% and 77% of the Ophiuroidea and Crinoidea distribution, respectively. The third model contributes to explaining the terrain variables that induce the localization of Dendrophyllia cornigera. GAM univariate models detect the terrain variables for each structural species in this third zone (Artemisina transiens, D. cornigera and Phakellia ventilabrum). To avoid the time-consuming task of manual annotation of presence, a deep-learning algorithm (YOLO v4) is proposed. This approach achieves very high reliability and low uncertainty in automatic object detection, identification and location. These new advances applied to underwater imagery (SfM and deep-learning) can resolve the very-high resolution information needed for predictive microhabitat modeling in a very complex zone.En prens

Sustainability of irrigated agriculture under salinity pressure – A study in semiarid Tunisia

In semiarid and arid Tunisia, water quality and agricultural practices are the major contributing factors to the degradation of soil resources threatening the sustainability of irrigation systems and agricultural productivity. Nowadays, about 50% of the total irrigated areas in Tunisia are considered at high risk for salinization. The aim of this thesis was to study soil management and salinity relationships in order to assure sustainable irrigated agriculture in areas under salinity pressure. To prevent further soil degradation, farmers and rural development officers need guidance and better tools for the measurement, prediction, and monitoring of soil salinity at different observation scales, and associated agronomical strategy. Field experiments were performed in semi-arid Nabeul (sandy soil), semi-arid Kalâat Landalous (clay soil), and the desertic Fatnassa oasis (gypsiferous soil). The longest observation period represented 17 years. Besides field studies, laboratory experiments were used to develop accurate soil salinity measurements and prediction techniques. In saline gypsiferous soil, the WET sensor can give similar accuracy of soil salinity as the TDR if calibrated values of the soil parameters are used instead of standard values. At the Fatnassa oasis scale, the predicted values of ECe and depth of shallow groundwater Dgw using electromagnetic induction EM-38 were found to be in agreement with observed values with acceptable accuracy. At Kalâat Landalous (1400 ha), the applicability of artificial neural network (ANN) models for predicting the spatial soil salinity (ECe) was found to be better than multivariate linear regression (MLR) models. In semi-arid and desertic Tunisia, irrigation and drainage reduce soil salinity and dilute the shallow groundwater. However, the ECgw has a larger impact than soil salinity variation on salt balance. Based on the findings related to variation in the spatial and temporal soil and groundwater properties, soil salinization factors were identified and the level of soil “salinization risk unit” (SRU) was developed. The groundwater properties, especially the Dgw, could be considered as the main cause of soil salinization risk in arid Tunisia. However, under an efficient drainage network and water management, the soil salinization could be considered as a reversible process. The SRU mapping can be used by both land planners and farmers to make appropriate decisions related to crop production and soil and water management

A systematic approach to airborne sensor orientation and calibration: method and models

Avui dia, s'estima que el mercat de la geom atica mou pels volts de 30 bilions d'euros. Al darrera del creixement d'aquest mercat hi trobem noves tecnologies,projectes i aplicacions, com per exemple, \Global Positioning System"(GPS), Galileo, \Global Monitoring for Environment and Security"(GMES), Google Earth, etc. Actualment, la demanda i el consum de geoinformaci o est a incrementant i, a m es a m es, aquesta ha de ser precisa, exacta, actualitzada i assequible. Amb l'objectiu d'acomplir aquests requisits t ecnics i, en general, la demanda del mercat, la ind ustria i l' ambit acad emic estan introduint un darrera l'altre sistemes d'imatgeria, plataformes a eries i plataformes satel.litals. Per o alhora, aquests sistemes d'adquisici o introdueixen nous problemes com el calibratge i l'orientaci o de sensors, la navegaci o de les plataformes (de manera precisa i exacta segons el seu rendiment), la combinaci o de diferents tipus de sensors, la integraci o de dades auxiliars que provenen de diverses fonts, aspectes temporals com la gravaci o \cont nua"en el temps dels sensors, la feble geometria d'alguns d'ells, etc. Alguns d'aquests problemes es poden resoldre amb els m etodes i estrat egies actuals, sovint afegint pegats, per o la majoria no es poden resoldre amb els m etodes vigents o no es poden resoldre amb els m etodes vigents amb fiabilitat i robustesa. Aquesta tesi presenta les abstraccions i generalitzacions necess aries que permeten desenvolupar la propera generaci o d'ajustos de xarxes i m etodes d'estimaci o amb l'objectiu de resoldre aquests problemes. A m es, basada en aquestes idees, s'ha desenvolupat la principal eina d'aquesta recerca: la plataforma de software \Generic Extensible Network Approach"(GENA). L'objectiu d'aquesta recerca es establir les bases met odiques d'un concepte sistem atic per l'orientaci o i el calibratge de sensors aeris i provar la seva validesa amb nous models i aplicacions. Aix , en primer lloc, prenent dist ancia sobre el que s'ha fet tradicionalment i tenint en compte tot el que ens ofereix la tecnologia INS/GNSS, aquesta tesi genera un m etode per l'explotaci o dels sistemes INS/GNSS en l'orientaci o i el calibratge de sensors aeris. I, en segon lloc, s'han proposat i testejat amb dades reals alguns models que conformen aquest concepte, com per exemple, l' us de temps, posici o i actitud donats pel sistema INS/GNSS en mode relatiu (eliminant la necessitat dels par ametres d'absorci o d'errors INS/GNSS o la matriu d'orientaci o relativa IMU-sensor), l' us de temps, posici o, velocitat i actitud pel calibratge de temps (utilitzant aix la soluci o completa que donen els sistemes INS/GNSS per lligar les dimensions espacial i temporal) o reduir el nombre de mesures de l'orientaci o integrada de sensors tradicional, duent a terme la proposta \fast aerotriangulation", Fast AT. Aquesta recerca est a presentada a la tesi com un compendi d'articles. Aix doncs, els resultats de la tesi no s on nom es el document de la tesi en si mateix i les publicacions, hi ha tamb e un software comercial i models i aplicacions que validen el m etode proposat i representen un nou panorama per l'orientaci o i el calibratge de sensors aeris.En la actualidad, el mercado de la geom atica est a valorado en unos 30 billones de euros. Tras el crecimiento de dicho mercado, se hallan nuevas tecnologias, proyectos y aplicaciones, como por ejemplo, \Global Positioning System"(GPS), Galileo, \Global Monitoring for Environment and Security"(GMES), Google Earth, etc. Hoy en d a, la demanda y el consumo de geoinformaci on est a increment andose y, adem as, dicha informaci on debe ser precisa, exacta, actualizada y asequible. Intentando cumplir estos requisitos t ecnicos y, en general, la demanda del mercado, la industria y el ambito acad emico est an introduciendo uno tras otro sistemas de imagen, plataformas a ereas y plataformas satelitales. Pero a su vez, estos sistemas de adquisici on introducen nuevos problemas como la calibraci on y la orientaci on de sensores, la navegaci on de las plataformas (debe ser precisa y exacta teniendo en cuenta su rendimiento particular), la combinación de diferentes tipos de sensor, la integraci on de datos auxiliares que proceden de diversas fuentes, aspectos temporales como la grabaci on \continua" en el tiempo de los sensores, la d ebil geometr a de algunos de ellos, etc. Algunos de estos problemas pueden resolverse con los m etodos y estrategias actuales, generalmente aplicando parches, pero la mayor a no se pueden resolver con los m etodos vigentes o no se pueden resolver con los m etodos vigentes con fi abilidad y robustez. Esta tesis presenta las abstracciones y generalizaciones necesarias que permiten desarrollar la pr oxima generaci on de ajustes de redes y m etodos de estimaci on con el objetivo de resolver estos problemas. Es m as, basada en estas ideas, se ha desarrollado la herramienta principal de esta investigaci on: la plataforma de software \Generic Extensible Network Approach"(GENA). El objetivo de esta investigaci on es establecer las bases met odicas de un concepto sistem atico para la orientaci on y la calibraci on de sensores a ereos, y probar su validez con nuevos modelos y aplicaciones. As pues, en primer lugar, distanci andonos de lo que tradicionalmente se ha realizado y considerando lo que la tecnolog a INS/GNSS nos ofrece, esta tesis crea un m etodo para la explotaci on de los sistemas INS/GNSS en la orientaci on y la calibraci on de sensores a ereos. Y, en segundo lugar, se proponen y testean con datos reales algunos modelos que constituyen este concepto, como por ejemplo, el uso de tiempo, posici on y actitud dados por el sistema INS/GNSS en modo relativo (eliminando la necesidad de los par ametros de absorci on de errores INS/GNSS o la matriz de orientaci on relativa IMU-sensor), el uso de tiempo, posici on, velocidad y actitud para la calibraci on temporal (utilizando as la soluci on completa que dan los sistemas INS/GNSS para enlazar las dimensiones espacial y temporal) o reducir el n umero de medidas de la orientaci on integrada de sensores tradicional, llevando a cabo la propuesta \fast aerotriangulation", Fast AT. Esta investigaci on est a presentada en la tesis como un compendio de art culos. Resumiendo, los resultados de la tesis no son s olo el documento de la tesis en sí mismo y las publicaciones, existe tambi en un software comercial y modelos y aplicaciones que validan el m etodo propuesto y presentan un nuevo panorama para la orientaci on y la calibraci on de sensores a ereos.The geomatic market has an estimated value of some 30 thirty trillion euros. Behind this growing market, there are new technologies, projects and applications like Global Positioning System (GPS), Galileo, Global Monitoring for Environment and Security (GMES), Google Earth, etc. Modern society is increasingly demanding and consuming geoinformation that must be precise, accurate, up-to-date and affordable. In an attempt to meet these technical requirements and general market demand, industry and academia are introducing one imaging system, airborne platform and satellite platform after another. These acquisitions are introducing new problems such as calibration and orientation of the sensors, navigation of the platforms (with an accurate and precise processing of their individual performances), combination of different types of sensors, integration of auxiliary data provided from various sources, temporal issues of the ¿continuously¿ recording sensors, weak geometry of some sensors, etc. Some of the previous problems can be solved with current methods and strategies, oftentimes with a dose of patchwork. However, the vast majority of these problems cannot be solved with the current methods, or at least not with a like degree of robustness and reliability. This thesis presents the abstractions and generalizations needed to facilitate the development of the next generation of network adjustment and estimation methods that will make it possible to solve these problems. Moreover, the main tool of this research is a commercial software platform, ¿Generic Extensible Network Approach' (GENA), based on the proposed network approach. The goal of this research is to establish a methodical basis of a systematic approach to airborne sensor orientation and calibration and to prove its validity with newly-developed models and applications. On one hand, viewing the traditional DiSO and ISO from a distance and considering the possibilities that the INS/GNSS technology offers, this thesis generates a method to exploit the INS/GNSS systems for airborne sensor orientation and calibration. On the other hand, several models that constitute this method are proposed and tested with independent actual data sets; for example, the use of INS/GNSS-derived time, position and attitude in relative mode (avoiding the need for GNSS linear shift parameters, that absorb the INS/GNSS errors, or the relative orientation IMU-to-sensor, boresight, matrix), the use of INS/GNSS-derived time, position, velocity and attitude for time calibration (exploiting the full solution of the INS/GNSS systems to link the space and time dimensions) or the measurement reduction of the traditional integrated sensor orientation to perform the proposed "fast aerotriangulation", or Fast AT. This research is presented in the thesis as compiled papers. Therefore, the results of this thesis are not only the thesis document itself and a number of publications, but also a commercial software platform and models and applications that validate the proposed method and present a new panorama for airborne sensor orientation and calibration

Towards Reliable and Accurate Global Structure-from-Motion

Reconstruction of objects or scenes from sparse point detections across multiple views is one of the most tackled problems in computer vision. Given the coordinates of 2D points tracked in multiple images, the problem consists of estimating the corresponding 3D points and cameras\u27 calibrations (intrinsic and pose), and can be solved by minimizing reprojection errors using bundle adjustment. However, given bundle adjustment\u27s nonlinear objective function and iterative nature, a good starting guess is required to converge to global minima. Global and Incremental Structure-from-Motion methods appear as ways to provide good initializations to bundle adjustment, each with different properties. While Global Structure-from-Motion has been shown to result in more accurate reconstructions compared to Incremental Structure-from-Motion, the latter has better scalability by starting with a small subset of images and sequentially adding new views, allowing reconstruction of sequences with millions of images. Additionally, both Global and Incremental Structure-from-Motion methods rely on accurate models of the scene or object, and under noisy conditions or high model uncertainty might result in poor initializations for bundle adjustment. Recently pOSE, a class of matrix factorization methods, has been proposed as an alternative to conventional Global SfM methods. These methods use VarPro - a second-order optimization method - to minimize a linear combination of an approximation of reprojection errors and a regularization term based on an affine camera model, and have been shown to converge to global minima with a high rate even when starting from random camera calibration estimations.This thesis aims at improving the reliability and accuracy of global SfM through different approaches. First, by studying conditions for global optimality of point set registration, a point cloud averaging method that can be used when (incomplete) 3D point clouds of the same scene in different coordinate systems are available. Second, by extending pOSE methods to different Structure-from-Motion problem instances, such as Non-Rigid SfM or radial distortion invariant SfM. Third and finally, by replacing the regularization term of pOSE methods with an exponential regularization on the projective depth of the 3D point estimations, resulting in a loss that achieves reconstructions with accuracy close to bundle adjustment

Simultaneous localisation and mapping with prior information

This thesis is concerned with Simultaneous Localisation and Mapping (SLAM), a technique by which a platform can estimate its trajectory with greater accuracy than odometry alone, especially when the trajectory incorporates loops. We discuss some of the shortcomings of the "classical" SLAM approach (in particular EKF-SLAM), which assumes that no information is known about the environment a priori. We argue that in general this assumption is needlessly stringent; for most environments, such as cities some prior information is known. We introduce an initial Bayesian probabilistic framework which considers the world as a hierarchy of structures, and maps (such as those produced by SLAM systems) as consisting of features derived from them. Common underlying structure between features in maps allows one to express and thus exploit geometric relations between them to improve their estimates. We apply the framework to EKF-SLAM for the case of a vehicle equipped with a range-bearing sensor operating in an urban environment, building up a metric map of point features, and using a prior map consisting of line segments representing building footprints. We develop a novel method called the Dual Representation, which allows us to use information from the prior map to not only improve the SLAM estimate, but also reduce the severity of errors associated with the EKF. Using the Dual Representation, we investigate the effect of varying the accuracy of the prior map for the case where the underlying structures and thus relations between the SLAM map and prior map are known. We then generalise to the more realistic case, where there is "clutter" - features in the environment that do not relate with the prior map. This involves forming a hypothesis for whether a pair of features in the SLAMstate and prior map were derived from the same structure, and evaluating this based on a geometric likelihood model. Initially we try an incrementalMultiple Hypothesis SLAM(MHSLAM) approach to resolve hypotheses, developing a novel method called the Common State Filter (CSF) to reduce the exponential growth in computational complexity inherent in this approach. This allows us to use information from the prior map immediately, thus reducing linearisation and EKF errors. However we find that MHSLAM is still too inefficient, even with the CSF, so we use a strategy that delays applying relations until we can infer whether they apply; we defer applying information from structure hypotheses until their probability of holding exceeds a threshold. Using this method we investigate the effect of varying degrees of "clutter" on the performance of SLAM