Multiple cue integration for robust tracking in dynamic environments: application to video relighting

L'anàlisi de moviment i seguiment d'objectes ha estat un dels pricipals focus d'atenció en la comunitat de visió per computador durant les dues darreres dècades. L'interès per aquesta àrea de recerca resideix en el seu ample ventall d'aplicabilitat, que s'extén des de tasques de navegació de vehicles autònoms i robots, fins a aplications en la indústria de l'entreteniment i realitat virtual.Tot i que s'han aconseguit resultats espectaculars en problemes específics, el seguiment d'objectes continua essent un problema obert, ja que els mètodes disponibles són propensos a ser sensibles a diversos factors i condicions no estacionàries de l'entorn, com ara moviments impredictibles de l'objecte a seguir, canvis suaus o abruptes de la il·luminació, proximitat d'objectes similars o fons confusos. Enfront aquests factors de confusió la integració de múltiples característiques ha demostrat que permet millorar la robustesa dels algoritmes de seguiment. En els darrers anys, degut a la creixent capacitat de càlcul dels ordinadors, hi ha hagut un significatiu increment en el disseny de complexes sistemes de seguiment que consideren simultàniament múltiples característiques de l'objecte. No obstant, la majoria d'aquests algoritmes estan basats enheurístiques i regles ad-hoc formulades per aplications específiques, fent-ne impossible l'extrapolació a noves condicions de l'entorn.En aquesta tesi proposem un marc probabilístic general per integrar el nombre de característiques de l'objecte que siguin necessàries, permetent que interactuin mútuament per tal d'estimar-ne el seu estat amb precisió, i per tant, estimar amb precisió la posició de l'objecte que s'està seguint. Aquest marc, s'utilitza posteriorment per dissenyar un algoritme de seguiment, que es valida en diverses seqüències de vídeo que contenen canvis abruptes de posició i il·luminació, camuflament de l'objecte i deformacions no rígides. Entre les característiques que s'han utilitzat per representar l'objecte, cal destacar la paramatrització robusta del color en un espai de color dependent de l'objecte, que permet distingir-lo del fons més clarament que altres espais de color típicament ulitzats al llarg de la literatura.En la darrera part de la tesi dissenyem una tècnica per re-il·luminar tant escenes estàtiques com en moviment, de les que s'en desconeix la geometria. La re-il·luminació es realitza amb un mètode 'basat en imatges', on la generació de les images de l'escena sota noves condicions d'il·luminació s'aconsegueix a partir de combinacions lineals d'un conjunt d'imatges de referència pre-capturades, i que han estat generades il·luminant l'escena amb patrons de llum coneguts. Com que la posició i intensitat de les fonts d'il.luminació que formen aquests patrons de llum es pot controlar, és natural preguntar-nos: quina és la manera més òptima d'il·luminar una escena per tal de reduir el nombre d'imatges de referència? Demostrem que la millor manera d'il·luminar l'escena (és a dir, la que minimitza el nombre d'imatges de referència) no és utilitzant una seqüència de fonts d'il·luminació puntuals, com es fa generalment, sinó a través d'una seqüència de patrons de llum d'una base d'il·luminació depenent de l'objecte. És important destacar que quan es re-il·luminen seqüències de vídeo, les imatges successives s'han d'alinear respecte a un sistema de coordenades comú. Com que cada imatge ha estat generada per un patró de llum diferent il·uminant l'escena, es produiran canvis d'il·luminació bruscos entre imatges de referència consecutives. Sota aquestes circumstàncies, el mètode de seguiment proposat en aquesta tesi juga un paper fonamental. Finalment, presentem diversos resultats on re-il·luminem seqüències de vídeo reals d'objectes i cares d'actors en moviment. En cada cas, tot i que s'adquireix un únic vídeo, som capaços de re-il·luminar una i altra vegada, controlant la direcció de la llum, la seva intensitat, i el color.Motion analysis and object tracking has been one of the principal focus of attention over the past two decades within the computer vision community. The interest of this research area lies in its wide range of applicability, extending from autonomous vehicle and robot navigation tasks, to entertainment and virtual reality applications.Even though impressive results have been obtained in specific problems, object tracking is still an open problem, since available methods are prone to be sensitive to several artifacts and non-stationary environment conditions, such as unpredictable target movements, gradual or abrupt changes of illumination, proximity of similar objects or cluttered backgrounds. Multiple cue integration has been proved to enhance the robustness of the tracking algorithms in front of such disturbances. In recent years, due to the increasing power of the computers, there has been a significant interest in building complex tracking systems which simultaneously consider multiple cues. However, most of these algorithms are based on heuristics and ad-hoc rules formulated for specific applications, making impossible to extrapolate them to new environment conditions.In this dissertation we propose a general probabilistic framework to integrate as many object features as necessary, permitting them to mutually interact in order to obtain a precise estimation of its state, and thus, a precise estimate of the target position. This framework is utilized to design a tracking algorithm, which is validated on several video sequences involving abrupt position and illumination changes, target camouflaging and non-rigid deformations. Among the utilized features to represent the target, it is important to point out the use of a robust parameterization of the target color in an object dependent colorspace which allows to distinguish the object from the background more clearly than other colorspaces commonly used in the literature.In the last part of the dissertation, we design an approach for relighting static and moving scenes with unknown geometry. The relighting is performed through an -image-based' methodology, where the rendering under new lighting conditions is achieved by linear combinations of a set of pre-acquired reference images of the scene illuminated by known light patterns. Since the placement and brightness of the light sources composing such light patterns can be controlled, it is natural to ask: what is the optimal way to illuminate the scene to reduce the number of reference images that are needed? We show that the best way to light the scene (i.e., the way that minimizes the number of reference images) is not using a sequence of single, compact light sources as is most commonly done, but rather to use a sequence of lighting patterns as given by an object-dependent lighting basis. It is important to note that when relighting video sequences, consecutive images need to be aligned with respect to a common coordinate frame. However, since each frame is generated by a different light pattern illuminating the scene, abrupt illumination changes between consecutive reference images are produced. Under these circumstances, the tracking framework designed in this dissertation plays a central role. Finally, we present several relighting results on real video sequences of moving objects, moving faces, and scenes containing both. In each case, although a single video clip was captured, we are able to relight again and again, controlling the lighting direction, extent, and color.Postprint (published version

ENHANCED GRAIN CROP YIELD MONITOR ACCURACY THROUGH SENSOR FUSION AND POST-PROCESSING ALGORITHMS

Yield monitors have become an indispensable part of precision agriculture systemsbecause of their ability to measure the yield variability. Accurate yield monitor data availabilityis essential for the assessment of farm practices. The current technology of measuring grainyields is prone to errors that can be attributed to mass flow variations caused by the mechanismswithin a grain combine. Because of throughput variations, there are doubts regarding thecorrelation between the mass flow measurement and the actual grain volume produced at aspecific location. Another inaccuracy observed in yield monitor data can be attributed to inexactcut-widths values entered by the machine operator.To effectively address these yield monitor errors, two crop mass flow sensing deviceswere developed and used to correct yield monitor data. The two quantities associated with cropmaterial mass flow that were sensed were tension on the feeder housing drive chain and thehydraulic pressure on the threshing cylinder\u27s variable speed drive. Both sensing approacheswere capable of detecting zero mass flow conditions better than the traditional grain mass flowsensor. The alternative sensors also operate without being adversely affected by materialtransport delays. The feeder housing-based sensor was more sensitive to variations in cropmaterial throughput than the hydraulic pressure sensor. Crop mass flow is not a surrogate forgrain mass flow because of a weak relationship (R2 andlt; 0.60) between the two quantities. The cropmass flow signal does denote the location and magnitude of material throughput variations intothe combine. This delineation was used to redistribute grain mass flow by aligning grain andcrop mass flow transitions using sensor fusion techniques. Significant improvements (?? = 0.05)in yield distribution profile were found after the correction was applied.To address the cut-width entry error, a GIS-based post-processing algorithm wasdeveloped to calculate the true harvest area for each yield monitor data point. Based on theresults of this method, a combine operator can introduce yield calculation errors of 15%. Whenthese two correction methods applied to yield monitor data, the result is yield maps withdramatically improved yield estimates and enhanced spatial accuracy

Modeling, Simulation and Control of Very Flexible Unmanned Aerial Vehicle

This dissertation presents research on modeling, simulation and control of very flexible aircraft. This work includes theoretical and numerical developments, as well as experimental validations. On the theoretical front, new kinematic equations for modeling sensors are derived. This formulation uses geometrically nonlinear strain-based finite elements developed as part of University of Michigan Nonlinear Aeroelastic Simulation Toolbox (UM/NAST). Numerical linearizations of both the flexible vehicle and the sensor measurements are developed, allowing a linear time invariant model to be extracted for control analysis and design. Two different algorithms to perform sensor fusion from different sensor sources to extract elastic deformation are investigated. Nonlinear least square method uses geometry and nonlinear beam strain-displacement kinematics to reconstruct the wing shape. Detailed information such as material properties or loading conditions are not required. The second method is the Kalman filter, implemented in a multi-rate form. This method requires a dynamical system representation to be available. However, it is more robust to noise corruption in sensor measurements. In order to control maneuver loads, Model Predictive Control is applied to maneuver load alleviation of a representative very flexible aircraft (X-HALE). Numerical studies are performed in UM/NAST for pitch up and roll maneuvers. Both control and state constraints are successfully enforced, while reference commands are still being tracked. MPC execution is also timed and current implementation is capable of almost real-time operation. On the experimental front, two aeroelastic testbed vehicles (ATV-6B and RRV-6B) are instrumented with sensors. On ATV-6B, an extensive set of sensors measuring structural, flight dynamic, and aerodynamic information are integrated on-board. A novel stereo-vision measurement system mounted on the body center looking towards the wing tip measures wing deformation. High brightness LEDs are used as target markers for easy detection and to allow each view to be captured with fast camera shutter speed. Experimental benchmarks are conducted to verify the accuracy of this methodology. RRV-6B flight test results are presented. System identification is applied to the experimental data to generate a SISO description of the flexible aircraft. System identification results indicate that the UM/NAST X-HALE model requires some tuning to match observed dynamics. However, the general trends predicted by the numerical model are in agreement with flight test results. Finally, using this identified plant, a stability augmentation autopilot is designed and flight tested. This augmentation autopilot utilizes a cascaded two-loop proportional integral control design, with the inner loop regulating angular rates and outer loop regulating attitude. Each of the three axes is assumed to be decoupled and designed using SISO methodology. This stabilization system demonstrates significant improvements in the RRV-6B handling qualities. This dissertation ends with a summary of the results and conclusions, and its main contribution to the field. Suggestions for future work are also presented.PHDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/144019/1/pziyang_1.pd

Gravity Gradiometry and Map Matching: An Aid to Aircraft Inertial Navigation Systems

Inertial navigation systems (INS) offer passive, all-weather, and undeniable navigation information, which military customers often view as especially appealing strengths. Unfortunately, Airmen and engineers still struggle with INS’s drifting position errors, and navigation aids generally detract from INS’s strengths. At this year’s Air, Space, and Cyberspace in the 21st Century Conference, the Chief of Staff of the Air Force identified the Global Positioning System (GPS) as a widely-known and exploitable vulnerability, saying that it’s critical the Joint force reduce GPS dependence. Recent advances provide an opportunity for gravity gradient instruments (GGI), which measure spatial derivatives of the gravity vector, to aid an INS and preserve its strengths. This thesis shows that a GGI and map matching enhanced (GAME) INS improves navigation accuracy, presents the conditions that make GAME feasible for aircraft, and identifies opportunities for improvement. The methodology includes computer models and algorithms, where a GGI and map matching aid an INS through a Kalman filter. Simulations cover different terrains, altitudes, velocities, flight durations, INS drifts, update rates, components of the gravity gradient tensor, GGI and map noise levels, map resolutions, and levels of interpolation. Although GAME with today’s technology only appears worthwhile for long range and long endurance flights, the technologies expected in 10 years promise a broad spectrum of scenarios where GAME potentially provides great returns on investments and dominates the market for secure and covert navigation

Autonomous Vehicles

This edited volume, Autonomous Vehicles, is a collection of reviewed and relevant research chapters, offering a comprehensive overview of recent developments in the field of vehicle autonomy. The book comprises nine chapters authored by various researchers and edited by an expert active in the field of study. All chapters are complete in itself but united under a common research study topic. This publication aims to provide a thorough overview of the latest research efforts by international authors, open new possible research paths for further novel developments, and to inspire the younger generations into pursuing relevant academic studies and professional careers within the autonomous vehicle field

NASA Tech Briefs, December 1989

Topics include: Electronic Components and Circuits. Electronic Systems, Physical Sciences, Materials, Computer Programs, Mechanics, Machinery, Fabrication Technology, Mathematics and Information Sciences, and Life Sciences

Applications in Electronics Pervading Industry, Environment and Society

This book features the manuscripts accepted for the Special Issue “Applications in Electronics Pervading Industry, Environment and Society—Sensing Systems and Pervasive Intelligence” of the MDPI journal Sensors. Most of the papers come from a selection of the best papers of the 2019 edition of the “Applications in Electronics Pervading Industry, Environment and Society” (APPLEPIES) Conference, which was held in November 2019. All these papers have been significantly enhanced with novel experimental results. The papers give an overview of the trends in research and development activities concerning the pervasive application of electronics in industry, the environment, and society. The focus of these papers is on cyber physical systems (CPS), with research proposals for new sensor acquisition and ADC (analog to digital converter) methods, high-speed communication systems, cybersecurity, big data management, and data processing including emerging machine learning techniques. Physical implementation aspects are discussed as well as the trade-off found between functional performance and hardware/system costs