UAV based distributed automatic target detection algorithm under realistic simulated environmental effects

Over the past several years, the military has grown increasingly reliant upon the use of unattended aerial vehicles (UAVs) for surveillance missions. There is an increasing trend towards fielding swarms of UAVs operating as large-scale sensor networks in the air [1]. Such systems tend to be used primarily for the purpose of acquiring sensory data with the goal of automatic detection, identification, and tracking objects of interest. These trends have been paralleled by advances in both distributed detection [2], image/signal processing and data fusion techniques [3]. Furthermore, swarmed UAV systems must operate under severe constraints on environmental conditions and sensor limitations. In this work, we investigate the effects of environmental conditions on target detection performance in a UAV network. We assume that each UAV is equipped with an optical camera, and use a realistic computer simulation to generate synthetic images. The automatic target detector is a cascade of classifiers based on Haar-like features. The detector\u27s performance is evaluated using simulated images that closely mimic data acquired in a UAV network under realistic camera and environmental conditions. In order to improve automatic target detection (ATD) performance in a swarmed UAV system, we propose and design several fusion techniques both at the image and score level and analyze both the case of a single observation and the case of multiple observations of the same target

Body swarm interface (BOSI) : controlling robotic swarms using human bio-signals

Traditionally robots are controlled using devices like joysticks, keyboards, mice and other similar human computer interface (HCI) devices. Although this approach is effective and practical for some cases, it is restrictive only to healthy individuals without disabilities, and it also requires the user to master the device before its usage. It becomes complicated and non-intuitive when multiple robots need to be controlled simultaneously with these traditional devices, as in the case of Human Swarm Interfaces (HSI). This work presents a novel concept of using human bio-signals to control swarms of robots. With this concept there are two major advantages: Firstly, it gives amputees and people with certain disabilities the ability to control robotic swarms, which has previously not been possible. Secondly, it also gives the user a more intuitive interface to control swarms of robots by using gestures, thoughts, and eye movement. We measure different bio-signals from the human body including Electroencephalography (EEG), Electromyography (EMG), Electrooculography (EOG), using off the shelf products. After minimal signal processing, we then decode the intended control action using machine learning techniques like Hidden Markov Models (HMM) and K-Nearest Neighbors (K-NN). We employ formation controllers based on distance and displacement to control the shape and motion of the robotic swarm. Comparison for ground truth for thoughts and gesture classifications are done, and the resulting pipelines are evaluated with both simulations and hardware experiments with swarms of ground robots and aerial vehicles

Optical-based ATR algorithms for applications in swarmed UAVs

Swarmed Unmanned Aerial Vehicles (UAVs) with Automatic Target Recognition (ATR) technology are becoming an important element of electronic warfare. The advantages of UAVs over piloted vehicles have increased the need to develop robust and reliable ATR algorithms. Various issues like changing weather, camouflage, low contrast and resolution, clutter, inadequate databases place a limit on the performance capabilities of a typical ATR algorithm. In an effort to deal with these issues, a correlation-based algorithm is proposed in this thesis. This algorithm calculates the correlation between the input image and a target template which is created by projecting a 3-D model from the perspective of the UAV. The locations of correlation peaks are then declared to be the locations of the targets. We apply this algorithm to images with one object of a known class and move on to the more general case of images with an unknown number of targets from one or more classes. We provide an analysis of the performance of this correlation-based algorithm.;We compare the performance of the proposed correlation-based approach with that of a training-based approach. To provide a concrete example of an off-the-shelf training-based ATR algorithm, the open source IntelCV library was used. In the training-based method, a sample set is created and trained (Haar-like features are used for training) to produce results for comparison purpose. We further develop and analyze a method of correlating across multiple frames that have been preprocessed using the correlation-based approach. This method is shown to be useful in detecting true targets and suppressing false alarms in cases where a single image is not sufficient for classification

Airborne Vision-Based Remote Sensing Imagery Datasets From Large Farms Using Autonomous Drones For Monitoring Livestock

Livestock have high economic value and monitoring of them in large farms regularly is a labour-intensive task and costly. The emergence of smart data on individual animals and their surroundings opens up new opportunities for early detection and disease prevention, better animal care and traceability, better sustainability and farm economics. Precision Livestock Farming (PLF) relies on the constant and automated gathering of livestock data to support the expertise and management decisions made by farmers, vets, and authorities. The high mobility of UAVs combined with a high level of autonomy, sensor-driven technologies and AI decision-making abilities can provide many advantages to farmers in exploiting instant information from every corner of a large farm. The key objectives of this research are to i) explore various drone-mounted vision-based remote sensing modalities, particularly, visual band sensing and a thermal imager, ii) develop UAV-assisted autonomous PLF technologies and ii) collect data with various parameters for the researchers to establish further advanced AI-based approaches for monitoring livestock in large farms effectively by fusing a rich set of features acquired using vision-based multi-sensor modalities. The collected data suggest that the fuse of distinctive features of livestock obtained from multiple sensor modalities can be exploited to help farmers experience better livestock management in large farms through PLF

Selected topics in video coding and computer vision

Video applications ranging from multimedia communication to computer vision have been extensively studied in the past decades. However, the emergence of new applications continues to raise questions that are only partially answered by existing techniques. This thesis studies three selected topics related to video: intra prediction in block-based video coding, pedestrian detection and tracking in infrared imagery, and multi-view video alignment.;In the state-of-art video coding standard H.264/AVC, intra prediction is defined on the hierarchical quad-tree based block partitioning structure which fails to exploit the geometric constraint of edges. We propose a geometry-adaptive block partitioning structure and a new intra prediction algorithm named geometry-adaptive intra prediction (GAIP). A new texture prediction algorithm named geometry-adaptive intra displacement prediction (GAIDP) is also developed by extending the original intra displacement prediction (IDP) algorithm with the geometry-adaptive block partitions. Simulations on various test sequences demonstrate that intra coding performance of H.264/AVC can be significantly improved by incorporating the proposed geometry adaptive algorithms.;In recent years, due to the decreasing cost of thermal sensors, pedestrian detection and tracking in infrared imagery has become a topic of interest for night vision and all weather surveillance applications. We propose a novel approach for detecting and tracking pedestrians in infrared imagery based on a layered representation of infrared images. Pedestrians are detected from the foreground layer by a Principle Component Analysis (PCA) based scheme using the appearance cue. To facilitate the task of pedestrian tracking, we formulate the problem of shot segmentation and present a graph matching-based tracking algorithm. Simulations with both OSU Infrared Image Database and WVU Infrared Video Database are reported to demonstrate the accuracy and robustness of our algorithms.;Multi-view video alignment is a process to facilitate the fusion of non-synchronized multi-view video sequences for various applications including automatic video based surveillance and video metrology. In this thesis, we propose an accurate multi-view video alignment algorithm that iteratively aligns two sequences in space and time. To achieve an accurate sub-frame temporal alignment, we generalize the existing phase-correlation algorithm to 3-D case. We also present a novel method to obtain the ground-truth of the temporal alignment by using supplementary audio signals sampled at a much higher rate. The accuracy of our algorithm is verified by simulations using real-world sequences

Remote Sensing Applications to Support Locust Management and Research: Evaluating the Potential of Earth Observation for Locust Outbreaks in Different Regions

This dissertation focuses on satellite remote sensing applications for locust management and additional contributions to locust research. Specifically, the remote sensing-based characterization and interpretation of land surface cover and its dynamics are addressed with a special emphasis on the requirements of different locust species. At first, the aim of this dissertation is to provide a holistic overview of the existing applications using satellite data focusing on different locust species and thus, to present current and new opportunities. Furthermore, remote sensing and geospatial datasets are used in a model to categorize areas with ideal and less than ideal conditions for locust outbreaks. The benefit of up-to-date remote sensing data for preventive locust management is demonstrated using time-series-based Sentinel-2 land cover classification. Due to the diversity of the numerous locust species and their spatial distribution in different geographical locations, this research focuses mainly on two locust species, the Italian locust (Calliptamus italicus) and the Moroccan locust (Dociostaurus maroccanus), as well as on selected study areas within their extensive habitats, respectively. Both selected locust species caused numerous damages in Europe, the Caucasus, Central Asia and North Africa in the past. For both species, there is only a limited number of publications exploiting the capabilities of remote sensing methods. Therefore, this dissertation aims to explore the potential approaches of Earth observation datasets to support preventive locust management and research for both species.Die vorliegende Dissertation beschäftigt sich mit dem Einsatz der Satellitenfernerkundung im Bereich Heuschreckenmanagement und -forschung. Die fernerkundungsbasierte Charakterisierung und Interpretation der Landoberflächen-bedeckung und deren Dynamik stehen dabei - mit Fokus auf die Anforderungen der verschiedenen Heuschreckenarten - im Vordergrund. Ziel dieser Dissertation ist es zunächst, einen ganzheitlichen Überblick über vorhandene Anwendungen von Satellitendaten im Kontext Heuschreckenmanagement zu erarbeiten. Des Weiteren werden fernerkundungs- und geobasierten Datensätzen in einem Model verwendet, um Flächen mit idealen bzw. weniger idealen Bedingungen für Heuschreckenausbrüche zu kategorisieren. Der Vorteil von aktuellen Fernerkundungsdaten für präventives Heuschreckenmanagement wird anhand zeitreihenbasierten Sentinel-2 Landbedeckungsklassifikation demonstriert. Aufgrund der Vielfältigkeit der zahlreichen Heuschreckenarten und deren räumlicher Verteilung in verschiedenen geographischen Lagen, konzentriert sich diese Arbeit im Wesentlichen auf zwei Heuschreckenarten, die Italienische Schönschrecke (Calliptamus italicus) und die Marokkanische Wanderheuschrecke (Dociostaurus maroccanus), sowie auf ausgewählte Studiengebiete innerhalb deren weiträumigen Habitaten. Beide Heuschreckenarten verursachten zahlreiche Ausbrüche in der Vergangenheit mit Schäden in Europa, dem Kaukasus, Zentralasien und Nordafrika. Für beide Heuschreckenarten existieren nur wenige Forschungsarbeiten, die sich mit der Anwendung von Fern-erkundungsdaten auseinandersetzen. Vor diesem Hintergrund zielt diese Dissertation auf die Entwicklung von relevanten Methoden unter Einsatz von Fernerkundungsdaten für beide Heuschreckenarten ab, um präventives Heuschreckenmanagement und -forschung zu unterstützen.Данная диссертация раскрывает тему применения спутникового дистанционного зондирования для контроля саранчовых и проведения дополнительных исследований саранчи. В частности, особое внимание уделяется изучению потребностей различных видов саранчовых при описании характеристик земного покрова и его динамики на основе данных дистанционного зондирования. Первостепенная цель данной диссертации состоит в том, чтобы предоставить целостный обзор существующих приложений, использующих спутниковые данные, в разрезе различных видов саранчовых для того, чтобы раскрыть текущие и потенциальные возможности. Кроме того, дистанционное зондирование и наборы геопространственных данных используются для классификации территорий с идеальными и не идеальными условиями для нашествий саранчи. исследование сосредоточено в основном на двух видах саранчи, итальянского пруса (Calliptamus italicus) и марокканской саранче (Dociostaurus maroccanus), а также на определенных территориях, в пределах их обширногo местообитаний

Third ERTS Symposium: Abstracts

Abstracts are provided for the 112 papers presented at the Earth Resources Program Symposium held at Washington, D.C., 10-14 December, 1973

Modeling and performance analysis of a UAV-based sensor network for improved ATR

Automatic Target Recognition (ATR) is computer processing of images or signals acquired by sensors with the purpose to identify objects of interest (targets). This technology is a critical element for surveillance missions. Over the past several years there has been an increasing trend towards fielding swarms of unattended aerial vehicles (UAVs) operating as sensor networks in the air. This trend offers opportunities of integration ATR systems with a UAV-based sensor network to improve the recognition performance. This dissertation addresses some of design issues of ATR systems, explores recognition capabilities of sensor networks in the presence of various distortions and analyzes the limiting recognition performance of sensor networks.;We assume that each UAV is equipped with an optical camera. A model based recognition method for single and multiple frames is introduced. A complete ATR system, including detection, segmentation, recognition and clutter rejection, is designed and tested using synthetic and realistic images. The effects of environmental conditions on target recognition are also investigated.;To analyze and predict ATR performance of a recognition sensor network, a general methodology from information theory view point is used. Given the encoding method, the recognition system is analyzed using a recognition channel. The concepts of recognition capacity, error exponents and probability of outage are defined and derived for a PCA-based ATR system. Both the case of a single encoded image and the case of encoded correlated multiple frames are analyzed. Numerical evaluations are performed. Finally we discuss the joint recognition and communication problems. Three scenarios of a two node recognition sensor network are analyzed. The communication and recognition performances for each scenario are evaluated numerically

Investigation of remote sensing techniques as inputs to operational resource management

The author has identified the following significant results. Visual interpretation of 1:125,000 color LANDSAT prints produced timely level 1 maps of accuracies in excess of 80% for agricultural land identification. Accurate classification of agricultural land via digital analysis of LANDSAT CCT's required precise timing of the date of data collection with mid to late June optimum for western South Dakota. The LANDSAT repetitive nine day cycle over the state allowed the surface areas of stockdams and small reservoir systems to be monitored to provide a timely approximation of surface water conditions on the range. Combined use of DIRS, K-class, and LANDSAT CCT's demonstrated the ability to produce aspen maps of greater detail and timeliness than was available using US Forest Service maps. Visual temporal analyses of LANDSAT imagery improved highway map drainage information and were used to prepare a seven county drainage network. An optimum map of flood-prone areas was developed, utilizing high altitude aerial photography and USGS maps

Derivation of forest inventory parameters from high-resolution satellite imagery for the Thunkel area, Northern Mongolia. A comparative study on various satellite sensors and data analysis techniques.

With the demise of the Soviet Union and the transition to a market economy starting in the 1990s, Mongolia has been experiencing dramatic changes resulting in social and economic disparities and an increasing strain on its natural resources. The situation is exacerbated by a changing climate, the erosion of forestry related administrative structures, and a lack of law enforcement activities. Mongolia’s forests have been afflicted with a dramatic increase in degradation due to human and natural impacts such as overexploitation and wildfire occurrences. In addition, forest management practices are far from being sustainable. In order to provide useful information on how to viably and effectively utilise the forest resources in the future, the gathering and analysis of forest related data is pivotal. Although a National Forest Inventory was conducted in 2016, very little reliable and scientifically substantiated information exists related to a regional or even local level. This lack of detailed information warranted a study performed in the Thunkel taiga area in 2017 in cooperation with the GIZ. In this context, we hypothesise that (i) tree species and composition can be identified utilising the aerial imagery, (ii) tree height can be extracted from the resulting canopy height model with accuracies commensurate with field survey measurements, and (iii) high-resolution satellite imagery is suitable for the extraction of tree species, the number of trees, and the upscaling of timber volume and basal area based on the spectral properties. The outcomes of this study illustrate quite clearly the potential of employing UAV imagery for tree height extraction (R2 of 0.9) as well as for species and crown diameter determination. However, in a few instances, the visual interpretation of the aerial photographs were determined to be superior to the computer-aided automatic extraction of forest attributes. In addition, imagery from various satellite sensors (e.g. Sentinel-2, RapidEye, WorldView-2) proved to be excellently suited for the delineation of burned areas and the assessment of tree vigour. Furthermore, recently developed sophisticated classifying approaches such as Support Vector Machines and Random Forest appear to be tailored for tree species discrimination (Overall Accuracy of 89%). Object-based classification approaches convey the impression to be highly suitable for very high-resolution imagery, however, at medium scale, pixel-based classifiers outperformed the former. It is also suggested that high radiometric resolution bears the potential to easily compensate for the lack of spatial detectability in the imagery. Quite surprising was the occurrence of dark taiga species in the riparian areas being beyond their natural habitat range. The presented results matrix and the interpretation key have been devised as a decision tool and/or a vademecum for practitioners. In consideration of future projects and to facilitate the improvement of the forest inventory database, the establishment of permanent sampling plots in the Mongolian taigas is strongly advised.2021-06-0