Pattern-theoretic foundations of automatic target recognition in clutter

Issued as final reportAir Force Office of Scientific Research (U.S.

DEEP INFERENCE ON MULTI-SENSOR DATA

Computer vision-based intelligent autonomous systems engage various types of sensors to perceive the world they navigate in. Vision systems perceive their environments through inferences on entities (structures, humans) and their attributes (pose, shape, materials) that are sensed using RGB and Near-InfraRed (NIR) cameras, LAser Detection And Ranging (LADAR), radar and so on. This leads to challenging and interesting problems in efficient data-capture, feature extraction, and attribute estimation, not only for RGB but various other sensors. In some cases, we encounter very limited amounts of labeled training data. In certain other scenarios we have sufficient data, but annotations are unavailable for supervised learning. This dissertation explores two approaches to learning under conditions of minimal to no ground truth. The first approach applies projections on training data that make learning efficient by improving training dynamics. The first and second topics in this dissertation belong to this category. The second approach makes learning without ground-truth possible via knowledge transfer from a labeled source domain to an unlabeled target domain through projections to domain-invariant shared latent spaces. The third and fourth topics in this dissertation belong to this category. For the first topic we study the feasibility and efficacy of identifying shapes in LADAR data in several measurement modes. We present results on efficient parameter learning with less data (for both traditional machine learning as well as deep models) on LADAR images. We use a LADAR apparatus to obtain range information from a 3-D scene by emitting laser beams and collecting the reflected rays from target objects in the region of interest. The Agile Beam LADAR concept makes the measurement and interpretation process more efficient using a software-defined architecture that leverages computational imaging principles. Using these techniques, we show that object identification and scene understanding can be accurately performed in the LADARmeasurement domain thereby rendering the efforts of pixel-based scene reconstruction superfluous. Next, we explore the effectiveness of deep features extracted by Convolutional Neural Networks (CNNs) in the Discrete Cosine Transform (DCT) domain for various image classification tasks such as pedestrian and face detection, material identification and object recognition. We perform the DCT operation on the feature maps generated by convolutional layers in CNNs. We compare the performance of the same network with the same hyper-parameters with or without the DCT step. Our results indicate that a DCT operation incorporated into the network after the first convolution layer can have certain advantages such as convergence over fewer training epochs and sparser weight matrices that are more conducive to pruning and hashing techniques. Next, we present an adversarial deep domain adaptation (ADA)-based approach for training deep neural networks that fit 3Dmeshes on humans in monocular RGB input images. Estimating a 3D mesh from a 2D image is helpful in harvesting complete 3Dinformation about body pose and shape. However, learning such an estimation task in a supervised way is challenging owing to the fact that ground truth 3D mesh parameters for real humans do not exist. We propose a domain adaptation based single-shot (no re-projection, no iterative refinement), end-to-end training approach with joint optimization on real and synthetic images on a shared common task. Through joint inference on real and synthetic data, the network extracts domain invariant features that are further used to estimate the 3D mesh parameters in a single shot with no supervision on real samples. While we compute regression loss on synthetic samples with ground truth mesh parameters, knowledge is transferred from synthetic to real data through ADA without direct ground truth for supervision. Finally, we propose a partially supervised method for satellite image super-resolution by learning a unified representation of samples from different domains (captured by different sensors) in a shared latent space. The training samples are drawn from two datasets which we refer to as source and target domains. The source domain consists of fewer samples which are of higher resolution and contain very detailed and accurate annotations. In contrast, samples from the target domain are low-resolution and available ground truth is sparse. The pipeline consists of a feature extractor and a super-resolving module which are trained end-to-end. Using a deep feature extractor, we jointly learn (on two datasets) a common embedding space for all samples. Partial supervision is available for the samples in the source domain which have high-resolution ground truth. Adversarial supervision is used to successfully super-resolve low-resolution RGB satellite imagery from target domain without direct paired supervision from high resolution counterparts

Visual Prediction of Rover Slip: Learning Algorithms and Field Experiments

Perception of the surrounding environment is an essential tool for intelligent navigation in any autonomous vehicle. In the context of Mars exploration, there is a strong motivation to enhance the perception of the rovers beyond geometry-based obstacle avoidance, so as to be able to predict potential interactions with the terrain. In this thesis we propose to remotely predict the amount of slip, which reflects the mobility of the vehicle on future terrain. The method is based on learning from experience and uses visual information from stereo imagery as input. We test the algorithm on several robot platforms and in different terrains. We also demonstrate its usefulness in an integrated system, onboard a Mars prototype rover in the JPL Mars Yard. Another desirable capability for an autonomous robot is to be able to learn about its interactions with the environment in a fully automatic fashion. We propose an algorithm which uses the robot's sensors as supervision for vision-based learning of different terrain types. This algorithm can work with noisy and ambiguous signals provided from onboard sensors. To be able to cope with rich, high-dimensional visual representations we propose a novel, nonlinear dimensionality reduction technique which exploits automatic supervision. The method is the first to consider supervised nonlinear dimensionality reduction in a probabilistic framework using supervision which can be noisy or ambiguous. Finally, we consider the problem of learning to recognize different terrains, which addresses the time constraints of an onboard autonomous system. We propose a method which automatically learns a variable-length feature representation depending on the complexity of the classification task. The proposed approach achieves a good trade-off between decrease in computational time and recognition performance.</p

Vision-Aided Autonomous Precision Weapon Terminal Guidance Using a Tightly-Coupled INS and Predictive Rendering Techniques

This thesis documents the development of the Vision-Aided Navigation using Statistical Predictive Rendering (VANSPR) algorithm which seeks to enhance the endgame navigation solution possible by inertial measurements alone. The eventual goal is a precision weapon that does not rely on GPS, functions autonomously, thrives in complex 3-D environments, and is impervious to jamming. The predictive rendering is performed by viewpoint manipulation of computer-generated of target objects. A navigation solution is determined by an Unscented Kalman Filter (UKF) which corrects positional errors by comparing camera images with a collection of statistically significant virtual images. Results indicate that the test algorithm is a viable method of aiding an inertial-only navigation system to achieve the precision necessary for most tactical strikes. On 14 flight test runs, the average positional error was 166 feet at endgame, compared with an inertial-only error of 411 feet

U.S. Unmanned Aerial Vehicles (UAVS) and Network Centric Warfare (NCW) impacts on combat aviation tactics from Gulf War I through 2007 Iraq

Unmanned, aerial vehicles (UAVs) are an increasingly important element of many modern militaries. Their success on battlefields in Afghanistan, Iraq, and around the globe has driven demand for a variety of types of unmanned vehicles. Their proven value consists in low risk and low cost, and their capabilities include persistent surveillance, tactical and combat reconnaissance, resilience, and dynamic re-tasking. This research evaluates past, current, and possible future operating environments for several UAV platforms to survey the changing dynamics of combat-aviation tactics and make recommendations regarding UAV employment scenarios to the Turkish military. While UAVs have already established their importance in military operations, ongoing evaluations of UAV operating environments, capabilities, technologies, concepts, and organizational issues inform the development of future systems. To what extent will UAV capabilities increasingly define tomorrow's missions, requirements, and results in surveillance and combat tactics? Integrating UAVs and concepts of operations (CONOPS) on future battlefields is an emergent science. Managing a transition from manned- to unmanned and remotely piloted aviation platforms involves new technological complexity and new aviation personnel roles, especially for combat pilots. Managing a UAV military transformation involves cultural change, which can be measured in decades.http://archive.org/details/usunmannedaerial109454211Turkish Air Force authors.Approved for public release; distribution is unlimited

A hardware-in-the-loop testing facility for unmanned aerial vehicle sensor suites and control algorithms

In the past decade Unmanned Aerial Vehicles (UAVs) have rapidly grown into a major field of robotics in both industry and academia. Many well established platforms have been developed, and the demand continues to grow. However, the UAVs utilized in industry are predominately remotely piloted aircraft offering very limited levels of autonomy. In contrast, fully autonomous flight has been achieved in research, and the degree of autonomy continues to grow, with research now focusing on advanced tasks such as navigating cluttered terrain and formation ying.The gap between academia and industry is the robustness of control algorithms. Academic research often focuses on proof of concept demonstrations with little or no consideration to real world concerns such as adverse weather or sensor integration.One of the goals of this thesis is to integrate real world issues into the design process. A testing environment was designed and built that allows sensors and control algorithms to be tested against real obstacles and environmental conditions in a controlled, repeatable fashion. The use of this facility is demonstrated in the implementation of a safe landing zone algorithm for a robotic helicopter equipped with a laser scanner. Results from tests conducted in the testing facility are used to analyze results from ights in the field.Controlling the testing environment also provides a baseline to evaluate different control solutions. In the current research paradigm, it is difficult to determine which research questions have been solved because the testing conditions vary from researcher to researcher. A common testing environment eliminates ambiguities and allows solutions to be characterized based on their performance in different terrains and environmental conditions.This thesis explores how flight tests can be conducted in the lab using the actual hardware and control algorithms. The sensor package is attached to a 6 DOF gantry whose motion is governed by the dynamic model of the aircraft. To provide an expansive terrain over which the flight can be conducted, a scaled model of the environment was created.The the feasibility of using a scaled environment is demonstrated with a common sensor package and control task: using computer vision to guide an autonomous helicopter. The effcts of scaling are investigated, and the approach validated by comparing results in the scaled model to actual flights. Finally, it is demonstrated how the facility can be used to investigate the effect of adverse conditions on control algorithm performance. The overarching philosophy of this work is that incorporating real world concerns into the design process leads to more fully developed and robust solutions.Ph.D., Mechanical Engineering -- Drexel University, 201

High-resolution, slant-angle scene generation and validation of concealed targets in DIRSIG

Traditionally, synthetic imagery has been constructed to simulate images captured with low resolution, nadir-viewing sensors. Advances in sensor design have driven a need to simulate scenes not only at higher resolutions but also from oblique view angles. The primary efforts of this research include: real image capture, scene construction and modeling, and validation of the synthetic imagery in the reflective portion of the spectrum. High resolution imagery was collected of an area named MicroScene at the Rochester Institute of Technology using the Chester F. Carlson Center for Imaging Science\u27s MISI and WASP sensors using an oblique view angle. Three Humvees, the primary targets, were placed in the scene under three different levels of concealment. Following the collection, a synthetic replica of the scene was constructed and then rendered with the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model configured to recreate the scene both spatially and spectrally based on actual sensor characteristics. Finally, a validation of the synthetic imagery against the real images of MicroScene was accomplished using a combination of qualitative analysis, Gaussian maximum likelihood classification, grey-level co-occurrence matrix derived texture metrics, and the RX algorithm. The model was updated following each validation using a cyclical development approach. The purpose of this research is to provide a level of confidence in the synthetic imagery produced by DIRSIG so that it can be used to train and develop algorithms for real world concealed target detection

Spectral Detection of Human Skin in VIS-SWIR Hyperspectral Imagery without Radiometric Calibration

Many spectral detection algorithms require precise ground truth measurements that are hand-selected in the image to apply radiometric calibration, converting image pixels into estimated reflectance vectors. That process is impractical for mobile, real-time hyperspectral target detection systems, which cannot empirically derive a pixel-to-reflectance relationship from objects in the image. Implementing automatic target recognition on high-speed snapshot hyperspectral cameras requires the ability to spectrally detect targets without performing radiometric calibration. This thesis demonstrates human skin detection on hyperspectral data collected at a high frame rate without using calibration panels, even as the illumination in the scene changes. Compared to an established skin detection method that requires calibration panels, the illumination-invariant methods in this thesis achieve nearly as good detection performance in sunny scenes and superior detection performance in cloudy scenes

Measuring pedestrian gait using low resolution infrared people counters

This thesis describes research conducted into the measure- ment of pedestrian movement. It starts with an examination of current pedestrian detection and tracking systems, looking at several different technologies including image-processing systems. It highlights, as other authors have, that there is still a substantial gap between the abilities of existing pedestrian measurement and tracking systems and the requirements of users of such systems. After the review it provides an introduction to human gait and its use as a biometric. It then examines the IRISYS people counter, a low resolution infrared detector, used for this research. The detector's advantages and disadvantages are discussed, a detailed description of the data produced is provided. The thesis then describes in detail a study establishing that human gait information can be measured by the IRISYS people counter. It examines the use of the detectors in stereo to measure the height of the people; however the results are not impressive. During this investigation the presence of oscillations likely to relate to this walking gait is noted in the data. A second study is carried out confirming that the noted oscillation originates from human gait and further data is gathered to enable the development of measurement algorithms. The magnitude of the walking oscillation noted is examined in detail. It is found to be both individualistic and highly correlated to gender. A gender distribution algorithm is presented and evaluated on data captured in two different locations. These show very promising results. Several different methods are described for processing the infor-mation to extract a measure of cadence. The cadence is found to be individualistic and shows interesting correlations to height and leg length. This thesis advances the field of pedestrian measurement by conducting pedestrian motion studies and developing algorithms for measuring human gait.EThOS - Electronic Theses Online ServiceGBUnited Kingdo