Image and Information Fusion Experiments with a Software-Defined Multi-Spectral Imaging System for Aviation and Marine Sensor Networks

The availability of Internet, line-of-sight and satellite identification and surveillance information as well as low-power, low-cost embedded systems-on-a-chip and a wide range of visible to long-wave infrared cameras prompted Embry Riddle Aeronautical University to collaborate with the University of Alaska Arctic Domain Awareness Center (ADAC) in summer 2016 to prototype a camera system we call the SDMSI (Software-Defined Multi-spectral Imager). The concept for the camera system from the start has been to build a sensor node that is drop-in-place for simple roof, marine, pole-mount, or buoy-mounts. After several years of component testing, the integrated SDMSI is now being tested, first on a roof-mount at Embry Riddle Prescott. The roof-mount testing demonstrates simple installation for the high spatial, temporal and spectral resolution SDMSI. The goal is to define and develop software and systems technology to complement satellite remote sensing and human monitoring of key resources such as drones, aircraft and marine vessels in and around airports, roadways, marine ports and other critical infrastructure. The SDMSI was installed at Embry Riddle Prescott in fall 2016 and continuous recording of long-wave infrared and visible images have been assessed manually and compared to salient object detection to automatically record only frames containing objects of interest (e.g. aircraft and drones). It is imagined that ultimately users of the SDMSI can pair with it via wireless to browse salient images. Further, both ADS-B (Automatic Dependent Surveillance-Broadcast) and S-AIS (Satellite Automatic Identification System) data are envisioned to be used by the SDMSI to form expectations for observing in future tests. This paper presents the preliminary results of several experiments and compares human review with smart image processing in terms of the receiver-operator characteristic. The system design and software are open architecture, such that other researchers are encouraged to construct and participate in sharing results and networking identical or improved versions of the SDMSI for safety, security and drop-in-place scientific image sensor networking

Smart Cities: Inverse Design of 3D Urban Procedural Models with Traffic and Weather Simulation

Urbanization, the demographic transition from rural to urban, has changed how we envision and share the world. From just one-fourth of the population living in cities one hundred years ago, now more than half of the population does, and this ratio is expected to grow in the near future. Creating more sustainable, accessible, safe, and enjoyable cities has become an imperative

Methods for Real-time Visualization and Interaction with Landforms

This thesis presents methods to enrich data modeling and analysis in the geoscience domain with a particular focus on geomorphological applications. First, a short overview of the relevant characteristics of the used remote sensing data and basics of its processing and visualization are provided. Then, two new methods for the visualization of vector-based maps on digital elevation models (DEMs) are presented. The first method uses a texture-based approach that generates a texture from the input maps at runtime taking into account the current viewpoint. In contrast to that, the second method utilizes the stencil buffer to create a mask in image space that is then used to render the map on top of the DEM. A particular challenge in this context is posed by the view-dependent level-of-detail representation of the terrain geometry. After suitable visualization methods for vector-based maps have been investigated, two landform mapping tools for the interactive generation of such maps are presented. The user can carry out the mapping directly on the textured digital elevation model and thus benefit from the 3D visualization of the relief. Additionally, semi-automatic image segmentation techniques are applied in order to reduce the amount of user interaction required and thus make the mapping process more efficient and convenient. The challenge in the adaption of the methods lies in the transfer of the algorithms to the quadtree representation of the data and in the application of out-of-core and hierarchical methods to ensure interactive performance. Although high-resolution remote sensing data are often available today, their effective resolution at steep slopes is rather low due to the oblique acquisition angle. For this reason, remote sensing data are suitable to only a limited extent for visualization as well as landform mapping purposes. To provide an easy way to supply additional imagery, an algorithm for registering uncalibrated photos to a textured digital elevation model is presented. A particular challenge in registering the images is posed by large variations in the photos concerning resolution, lighting conditions, seasonal changes, etc. The registered photos can be used to increase the visual quality of the textured DEM, in particular at steep slopes. To this end, a method is presented that combines several georegistered photos to textures for the DEM. The difficulty in this compositing process is to create a consistent appearance and avoid visible seams between the photos. In addition to that, the photos also provide valuable means to improve landform mapping. To this end, an extension of the landform mapping methods is presented that allows the utilization of the registered photos during mapping. This way, a detailed and exact mapping becomes feasible even at steep slopes

An active vision system for tracking and mosaicking on UAV.

Lin, Kai Wun.Thesis (M.Phil.)--Chinese University of Hong Kong, 2011.Includes bibliographical references (leaves 120-127).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Overview of the UAV Project --- p.1Chapter 1.2 --- Challenges on Vision System for UAV --- p.2Chapter 1.3 --- Contributions of this Work --- p.4Chapter 1.4 --- Organization of Thesis --- p.6Chapter 2 --- Image Sensor Selection and Evaluation --- p.8Chapter 2.1 --- Image Sensor Overview --- p.8Chapter 2.1.1 --- Comparing Sensor Features and Performance --- p.9Chapter 2.1.2 --- Rolling Shutter vsGlobal Shutter --- p.10Chapter 2.2 --- Sensor Evaluation through USB Peripheral --- p.11Chapter 2.2.1 --- Interfacing Image Sensor and USB Controller --- p.12Chapter 2.2.2 --- Image Sensor Configuration --- p.14Chapter 2.3 --- Image Data Transmitting and Processing --- p.17Chapter 2.3.1 --- Data Transfer Mode and Buffering on USB Controller --- p.18Chapter 2.3.2 --- Demosaicking of Bayer Image Data --- p.20Chapter 2.4 --- Splitting Images and Exposure Problem --- p.22Chapter 2.4.1 --- Buffer Overflow on USB Controller --- p.22Chapter 2.4.2 --- Image Luminance and Exposure Adjustment --- p.24Chapter 3 --- Embedded System for Vision Processing --- p.26Chapter 3.1 --- Overview of the Embedded System --- p.26Chapter 3.1.1 --- TI OMAP3530 Processor --- p.27Chapter 3.1.2 --- Gumstix Overo Fire Computer-on-Module --- p.27Chapter 3.2 --- Interfacing Camera Module to the Embedded System --- p.28Chapter 3.2.1 --- Image Signal Processing Subsystem --- p.29Chapter 3.2.2 --- Camera Module Adapting Board --- p.30Chapter 3.2.3 --- Image Sensor Driver and Program Development --- p.31Chapter 3.3 --- View-stabilizing Biaxial Camera Platform --- p.34Chapter 3.3.1 --- The New Camera System iv --- p.35Chapter 3.3.2 --- View-stabilizing Pan-tilt Platform --- p.41Chapter 3.4 --- Overall System Architecture and UAV Integration --- p.46Chapter 4 --- Target Tracking and Geo-locating --- p.50Chapter 4.1 --- Camera Calibration --- p.51Chapter 4.1.1 --- The Perspective Camera Model --- p.51Chapter 4.1.2 --- Camera Lens Distortions --- p.53Chapter 4.1.3 --- Calibration Toolbox and Results --- p.54Chapter 4.2 --- Selection of Object Features and Trackers --- p.56Chapter 4.2.1 --- Harris Corner Detection --- p.58Chapter 4.2.2 --- Color Histogram --- p.59Chapter 4.2.3 --- KLT and Mean-shift Tracker --- p.59Chapter 4.3 --- Target Auto-centering --- p.64Chapter 4.3.1 --- Formulation of the PID Controller --- p.65Chapter 4.3.2 --- Control Gain Settings and Tuning --- p.69Chapter 4.4 --- Geo-locating of Tracked Target --- p.69Chapter 4.4.1 --- Coordinate Frame Transformation --- p.70Chapter 4.4.2 --- Depth Estimation and Target Locating --- p.74Chapter 4.5 --- Results and Discussion --- p.77Chapter 5 --- Real-time Aerial Mosaic Building --- p.89Chapter 5.1 --- Motion Model Selection --- p.90Chapter 5.1.1 --- Planar Perspective Motion Model --- p.90Chapter 5.2 --- Feature-based Image Alignment --- p.91Chapter 5.2.1 --- Image Preprocessing --- p.91Chapter 5.2.2 --- Feature Extraction and Matching --- p.92Chapter 5.2.3 --- Image Alignment using RANSAC Algorithm --- p.94Chapter 5.3 --- Image Composition --- p.95Chapter 5.3.1 --- Image Blending with Distance Map --- p.96Chapter 5.3.2 --- Overall Stitching Process --- p.98Chapter 5.4 --- Mosaic Simulation using Google Earth --- p.99Chapter 5.5 --- Results and Discussion --- p.100Chapter 6 --- Conclusion and Further Work --- p.108Chapter A --- System Schematics --- p.111Chapter B --- Image Sensor Sensitivity --- p.118Bibliography --- p.12

Efficient Algorithms for Large-Scale Image Analysis

This work develops highly efficient algorithms for analyzing large images. Applications include object-based change detection and screening. The algorithms are 10-100 times as fast as existing software, sometimes even outperforming FGPA/GPU hardware, because they are designed to suit the computer architecture. This thesis describes the implementation details and the underlying algorithm engineering methodology, so that both may also be applied to other applications

Real-time rendering of large surface-scanned range data natively on a GPU

This thesis presents research carried out for the visualisation of surface anatomy data stored as large range images such as those produced by stereo-photogrammetric, and other triangulation-based capture devices. As part of this research, I explored the use of points as a rendering primitive as opposed to polygons, and the use of range images as the native data representation. Using points as a display primitive as opposed to polygons required the creation of a pipeline that solved problems associated with point-based rendering. The problems inves tigated were scattered-data interpolation (a common problem with point-based rendering), multi-view rendering, multi-resolution representations, anti-aliasing, and hidden-point re- moval. In addition, an efficient real-time implementation on the GPU was carried out

Remotely Operated Aerial Vehicles and Their Applications

This project examines relevant designs and applications of unmanned aerial vehicles (UAVs). We propose UAV design solutions, which can be refined and incorporated into emergency medical services. Mathematical and engineering concepts are used to select the design solutions. We believe that the proposed design solutions will enhance the quality of care in emergency medical services