Progressively communicating rich telemetry from autonomous underwater vehicles via relays

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution June 2012As analysis of imagery and environmental data plays a greater role in mission construction and execution, there is an increasing need for autonomous marine vehicles to transmit this data to the surface. Without access to the data acquired by a vehicle, surface operators cannot fully understand the state of the mission. Communicating imagery and high-resolution sensor readings to surface observers remains a significant challenge – as a result, current telemetry from free-roaming autonomous marine vehicles remains limited to ‘heartbeat’ status messages, with minimal scientific data available until after recovery. Increasing the challenge, longdistance communication may require relaying data across multiple acoustic hops between vehicles, yet fixed infrastructure is not always appropriate or possible. In this thesis I present an analysis of the unique considerations facing telemetry systems for free-roaming Autonomous Underwater Vehicles (AUVs) used in exploration. These considerations include high-cost vehicle nodes with persistent storage and significant computation capabilities, combined with human surface operators monitoring each node. I then propose mechanisms for interactive, progressive communication of data across multiple acoustic hops. These mechanisms include wavelet-based embedded coding methods, and a novel image compression scheme based on texture classification and synthesis. The specific characteristics of underwater communication channels, including high latency, intermittent communication, the lack of instantaneous end-to-end connectivity, and a broadcast medium, inform these proposals. Human feedback is incorporated by allowing operators to identify segments of data thatwarrant higher quality refinement, ensuring efficient use of limited throughput. I then analyze the performance of these mechanisms relative to current practices. Finally, I present CAPTURE, a telemetry architecture that builds on this analysis. CAPTURE draws on advances in compression and delay tolerant networking to enable progressive transmission of scientific data, including imagery, across multiple acoustic hops. In concert with a physical layer, CAPTURE provides an endto- end networking solution for communicating science data from autonomous marine vehicles. Automatically selected imagery, sonar, and time-series sensor data are progressively transmitted across multiple hops to surface operators. Human operators can request arbitrarily high-quality refinement of any resource, up to an error-free reconstruction. The components of this system are then demonstrated through three field trials in diverse environments on SeaBED, OceanServer and Bluefin AUVs, each in different software architectures.Thanks to the National Science Foundation, and the National Oceanic and Atmospheric Administration for their funding of my education and this work

Deep Cellular Recurrent Neural Architecture for Efficient Multidimensional Time-Series Data Processing

Efficient processing of time series data is a fundamental yet challenging problem in pattern recognition. Though recent developments in machine learning and deep learning have enabled remarkable improvements in processing large scale datasets in many application domains, most are designed and regulated to handle inputs that are static in time. Many real-world data, such as in biomedical, surveillance and security, financial, manufacturing and engineering applications, are rarely static in time, and demand models able to recognize patterns in both space and time. Current machine learning (ML) and deep learning (DL) models adapted for time series processing tend to grow in complexity and size to accommodate the additional dimensionality of time. Specifically, the biologically inspired learning based models known as artificial neural networks that have shown extraordinary success in pattern recognition, tend to grow prohibitively large and cumbersome in the presence of large scale multi-dimensional time series biomedical data such as EEG. Consequently, this work aims to develop representative ML and DL models for robust and efficient large scale time series processing. First, we design a novel ML pipeline with efficient feature engineering to process a large scale multi-channel scalp EEG dataset for automated detection of epileptic seizures. With the use of a sophisticated yet computationally efficient time-frequency analysis technique known as harmonic wavelet packet transform and an efficient self-similarity computation based on fractal dimension, we achieve state-of-the-art performance for automated seizure detection in EEG data. Subsequently, we investigate the development of a novel efficient deep recurrent learning model for large scale time series processing. For this, we first study the functionality and training of a biologically inspired neural network architecture known as cellular simultaneous recurrent neural network (CSRN). We obtain a generalization of this network for multiple topological image processing tasks and investigate the learning efficacy of the complex cellular architecture using several state-of-the-art training methods. Finally, we develop a novel deep cellular recurrent neural network (CDRNN) architecture based on the biologically inspired distributed processing used in CSRN for processing time series data. The proposed DCRNN leverages the cellular recurrent architecture to promote extensive weight sharing and efficient, individualized, synchronous processing of multi-source time series data. Experiments on a large scale multi-channel scalp EEG, and a machine fault detection dataset show that the proposed DCRNN offers state-of-the-art recognition performance while using substantially fewer trainable recurrent units

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Development of an optical sensor for real-time weed detection using laser based spectroscopy

The management of weeds in agriculture is a time consuming and expensive activity, including in Australia where the predominant strategy is blanket spraying of herbicides. This approach wastes herbicide by applying it in areas where there are no weeds. Discrimination of different plant species can be performed based on the spectral reflectance of the leaves. This thesis describes the development of a sensor for automatic spot spraying of weeds within crop rows. The sensor records the relative intensity of reflected light in three narrow wavebands using lasers as an illumination source. A prototype weed sensor which had been previously developed was evaluated and redesigned to improve its plant discrimination performance. A line scan image sensor replacement was chosen which reduced the noise in the recorded spectral reflectance properties. The switching speed of the laser sources was increased by replacing the laser drivers. The optical properties of the light source were improved to provide a more uniform illumination across the viewing area of the sensor. A new opto-mechanical system was designed and constructed with the required robustness to operate the weed sensor in outdoor conditions. Independent operation of the sensor was made possible by the development of hardware and software for an embedded controller which operated the opto-electronic components and performed plant discrimination. The first revised prototype was capable of detecting plants at a speed of 10 km/h in outdoor conditions with the sensor attached to a quad bike. However, it was not capable of discriminating different plants. The final prototype included a line scan sensor with increased dynamic range and pixel resolution as well as improved stability of the output laser power. These changes improved the measurement of spectral reflectance properties of plants and provided reliable discrimination of three different broadleaved plants using only three narrow wavelength bands. A field trial with the final prototype demonstrated successful discrimination of these three different plants at 5 km/h when a shroud was used to block ambient light. A survey of spectral reflectance of four crops (sugarcane, cotton, wheat and sorghum) and the weeds growing amongst these crops was conducted to determine the potential for use of the prototype weed sensor to control spot-spraying of herbicides. Visible reflectance spectra were recorded from individual leaves using a fibre spectrometer throughout the growing season for each crop. A discriminant analysis was conducted based on six narrow wavebands extracted from leaf level spectral reflectance measured with a spectrometer. The analysis showed the potential to discriminate cotton and sugarcane fro

Primitives and design of the intelligent pixel multimedia communicator

Communication systems arc an ever more essential component of our modern global society. Mobile communications systems are still in a state of rapid advancement and growth. Technology is constantly evolving at a rapid pace in ever more diverse areas and the emerging mobile multimedia based communication systems offer new challenges for both current and future technologies. To realise the full potential of mobile multimedia communication systems there is a need to explore new options to solve some of the fundamental problems facing the technology. In particular, the complexity of such a system within an infrastructure framework that is inherently limited by its power sources and has very restricted transmission bandwidth demands new methodologies and approaches

Object Detection in 20 Years: A Survey

Object detection, as of one the most fundamental and challenging problems in computer vision, has received great attention in recent years. Its development in the past two decades can be regarded as an epitome of computer vision history. If we think of today's object detection as a technical aesthetics under the power of deep learning, then turning back the clock 20 years we would witness the wisdom of cold weapon era. This paper extensively reviews 400+ papers of object detection in the light of its technical evolution, spanning over a quarter-century's time (from the 1990s to 2019). A number of topics have been covered in this paper, including the milestone detectors in history, detection datasets, metrics, fundamental building blocks of the detection system, speed up techniques, and the recent state of the art detection methods. This paper also reviews some important detection applications, such as pedestrian detection, face detection, text detection, etc, and makes an in-deep analysis of their challenges as well as technical improvements in recent years.Comment: This work has been submitted to the IEEE TPAMI for possible publicatio

Multi-scale data fusion for surface metrology

The major trends in manufacturing are miniaturization, convergence of the traditional research fields and creation of interdisciplinary research areas. These trends have resulted in the development of multi-scale models and multi-scale surfaces to optimize the performance. Multi-scale surfaces that exhibit specific properties at different scales for a specific purpose require multi-scale measurement and characterization. Researchers and instrument developers have developed instruments that are able to perform measurements at multiple scales but lack the much required multi- scale characterization capability. The primary focus of this research was to explore possible multi-scale data fusion strategies and options for surface metrology domain and to develop enabling software tools in order to obtain effective multi-scale surface characterization, maximizing fidelity while minimizing measurement cost and time. This research effort explored the fusion strategies for surface metrology domain and narrowed the focus on Discrete Wavelet Frame (DWF) based multi-scale decomposition. An optimized multi-scale data fusion strategy ‘FWR method’ was developed and was successfully demonstrated on both high aspect ratio surfaces and non-planar surfaces. It was demonstrated that the datum features can be effectively characterized at a lower resolution using one system (Vision CMM) and the actual features of interest could be characterized at a higher resolution using another system (Coherence Scanning Interferometer) with higher capability while minimizing the measurement time

A Neutral Network Based Vehicle Classification System for Pervasive Smart Road Security

Pervasive smart computing environments make people get accustomed to convenient and secure services. The overall goal of this research is to classify vehicles along the I215 freeway in Salt Lake City, USA. This information will be used to predict future roadway needs and the expected life of a roadway. The classification of vehicles will be performed by a synthesis of multiple sets of features. All feature sets have not yet been determined; however, one such set will be the reduced wavelet transform of the image of a vehicle. In order to use such a feature, it is necessary that the image be normalized with respect to size, position, and so on. For example, a car in the right most lane in an image will appear smaller than one in the left most lane, because the right most lane is closest to the camera. Likewise, a vehicle’s size will vary depending on where in a lane its image is captured. In our case, the image capture area for each lane is approximately 100 feet of roadway. A goal of this paper is to normalize the image of a vehicle so that regardless of its lane or position in a lane, the features will be approximately the same. The wavelet transform itself will not be used directly for recognition. Instead, it will be input to a neural network and the output of the neural network will be one element of the feature set used for recognition