Neural networks application to divergence-based passive ranging

The purpose of this report is to summarize the state of knowledge and outline the planned work in divergence-based/neural networks approach to the problem of passive ranging derived from optical flow. Work in this and closely related areas is reviewed in order to provide the necessary background for further developments. New ideas about devising a monocular passive-ranging system are then introduced. It is shown that image-plan divergence is independent of image-plan location with respect to the focus of expansion and of camera maneuvers because it directly measures the object's expansion which, in turn, is related to the time-to-collision. Thus, a divergence-based method has the potential of providing a reliable range complementing other monocular passive-ranging methods which encounter difficulties in image areas close to the focus of expansion. Image-plan divergence can be thought of as some spatial/temporal pattern. A neural network realization was chosen for this task because neural networks have generally performed well in various other pattern recognition applications. The main goal of this work is to teach a neural network to derive the divergence from the imagery

APPLYING MACHINE LEARNING FOR COP/CTP DATA FILTERING

Student Thesis (NPS NRP Project Related)Accurate tracks and targeting are key to providing decision-makers with the confidence to execute their missions. Increasingly, multiple intelligence, surveillance, and reconnaissance (ISR) assets across different intelligence sources are being used to increase the accuracy of track location, resulting in the need to develop methods to exploit heterogeneous sensor data streams for better target state estimation. One of the algorithms commonly used for target state estimation is the Kalman Filter (KF) algorithm. This algorithm performs well if its covariance matrices are accurate approximations of the uncertainty in sensor measurements. Our research complements the artificial intelligence/machine learning (AI/ML) efforts the U.S. Navy is conducting by quantitatively assessing the potential of using an ML model to predict sensor measurement noise for KF state estimation. We used a computer simulation to generate sensor tracks of a single target and trained a neural network to predict sensor error. The hybrid model (ML-KF) was able to outperform our baseline KF model that uses normalized sensor errors by approximately 20% in target position estimation. Further research in enhancing the ML model with external environment variables as inputs could potentially create an adaptive state estimation system that is capable of operating in varied environment settings.NPS Naval Research ProgramThis project was funded in part by the NPS Naval Research Program.Outstanding ThesisCaptain, Singapore ArmyApproved for public release. Distribution is unlimited

Improving the efficiency and accuracy of nocturnal bird Surveys through equipment selection and partial automation

This thesis was submitted for the degree of Engineering Doctorate and awarded by Brunel University.Birds are a key environmental asset and this is recognised through comprehensive legislation and policy ensuring their protection and conservation. Many species are active at night and surveys are required to understand the implications of proposed developments such as towers and reduce possible conflicts with these structures. Night vision devices are commonly used in nocturnal surveys, either to scope an area for bird numbers and activity, or in remotely sensing an area to determine potential risk. This thesis explores some practical and theoretical approaches that can improve the accuracy, confidence and efficiency of nocturnal bird surveillance. As image intensifiers and thermal imagers have operational differences, each device has associated strengths and limitations. Empirical work established that image intensifiers are best used for species identification of birds against the ground or vegetation. Thermal imagers perform best in detection tasks and monitoring bird airspace usage. The typically used approach of viewing bird survey video from remote sensing in its entirety is a slow, inaccurate and inefficient approach. Accuracy can be significantly improved by viewing the survey video at half the playback speed. Motion detection efficiency and accuracy can be greatly improved through the use of adaptive background subtraction and cumulative image differencing. An experienced ornithologist uses bird flight style and wing oscillations to identify bird species. Changes in wing oscillations can be represented in a single inter-frame similarity matrix through area-based differencing. Bird species classification can then be automated using singular value decomposition to reduce the matrices to one-dimensional vectors for training a feed-forward neural network

Design and performance assessment of correlation filters for the detection of objects in high clutter thermal imagery

The research reported in this thesis has examined means of enhancing the performance of the Optimal Trade-off Maximum Average Correlation Height (OT-MACH) filter for target detection in Forward Looking Infra-Red (FLIR) imagery acquired from a helicopter and border security FLIR camera in northern Kuwait. The data acquired with these FLIR sensors allows real-world evaluation of the comparative performance of the various filters that have been developed in the thesis. The results obtained have been quantified using well known performance measures such as Peak to Side-lobe Ratio (PSR) and Total Detection Error (TDE). The initial focus was to study the effect of modifying the OT-MACH parameters on the correlation metrics. A new optimisation technique has been presented, which computes statistically the filter alpha parameter associated with controlling the response of the filter to clutter noise. A further modification of the OT-MACH filter performance using the Difference of Gaussian bandpass filter (named the D-MACH filter) as a pre-processing stage has been described. The D-MACH has been applied to several test images containing single and multiple targets in the scene. Enhanced performance of the modified filter is demonstrated with improved metrics being obtained with less false side peaks in the correlation plane, especially when multiple targets are present in the test images. A further pre-processing technique was investigated using the Rayleigh distribution as a pre-processing filter (named the R-MACH filter). The R-MACH filter has been applied to multiple target types with tests conducted across various image data sets. The filter demonstrated an improvement over the Difference of Gaussian filter in terms of 6 reducing the number of parameters needing to be tuned whilst producing further enhanced correlation plane metrics. Finally, recommendations for future work has been made to improve the use of the OT-MACH filter in target detection and identification. A novel training image representation is proposed for further investigation, which will minimise the computational intensity of using the MACH filter for unconstrained object recognition

Object detection in dual-band infrared

Dual-Band Infrared (DBIR) offers the advantage of combining Mid-Wave Infrared (MWIR) and Long-Wave Infrared (LWIR) within a single field-of-view (FoV). This provides additional information for each spectral band. DBIR camera systems find applications in both military and civilian contexts. This work introduces a novel labeled DBIR dataset that includes civilian vehicles, aircraft, birds, and people. The dataset is designed for utilization in object detection and tracking algorithms. It comprises 233 objects with tracks spanning up to 1,300 frames, encompassing images in both MW and LW. This research reviews pertinent literature related to object detection, object detection in the infrared spectrum, and data fusion. Two sets of experiments were conducted using this DBIR dataset: Motion Detection and CNNbased object detection. For motion detection, a parallel implementation of the Visual Background Extractor (ViBe) was developed, employing ConnectedComponents analysis to generate bounding boxes. To assess these bounding boxes, Intersection-over-Union (IoU) calculations were performed. The results demonstrate that DBIR enhances the IoU of bounding boxes in 6.11% of cases within sequences where the camera’s field of view remains stationary. A size analysis reveals ViBe’s effectiveness in detecting small and dim objects within this dataset. A subsequent experiment employed You Only Look Once (YOLO) versions 4 and 7 to conduct inference on this dataset, following image preprocessing. The inference models were trained using visible spectrum MS COCO data. The findings confirm that YOLOv4/7 effectively detect objects within the infrared spectrum in this dataset. An assessment of these CNNs’ performance relative to the size of the detected object highlights the significance of object size in detection capabilities. Notably, DBIR substantially enhances detection capabilities in both YOLOv4 and YOLOv7; however, in the latter case, the number of False Positive detections increases. Consequently, while DBIR improves the recall of YOLOv4/7, the introduction of DBIR information reduces the precision of YOLOv7. This study also demonstrates the complementary nature of ViBe and YOLO in their detection capabilities based on object size in this data set. Though this is known prior art, an approach using these two approaches in a hybridized configuration is discussed. ViBe excels in detecting small, distant objects, while YOLO excels in detecting larger, closer objects. The research underscores that DBIR offers multiple advantages over MW or LW alone in modern computer vision algorithms, warranting further research investment

TractorEYE: Vision-based Real-time Detection for Autonomous Vehicles in Agriculture

Agricultural vehicles such as tractors and harvesters have for decades been able to navigate automatically and more efficiently using commercially available products such as auto-steering and tractor-guidance systems. However, a human operator is still required inside the vehicle to ensure the safety of vehicle and especially surroundings such as humans and animals. To get fully autonomous vehicles certified for farming, computer vision algorithms and sensor technologies must detect obstacles with equivalent or better than human-level performance. Furthermore, detections must run in real-time to allow vehicles to actuate and avoid collision.This thesis proposes a detection system (TractorEYE), a dataset (FieldSAFE), and procedures to fuse information from multiple sensor technologies to improve detection of obstacles and to generate a map. TractorEYE is a multi-sensor detection system for autonomous vehicles in agriculture. The multi-sensor system consists of three hardware synchronized and registered sensors (stereo camera, thermal camera and multi-beam lidar) mounted on/in a ruggedized and water-resistant casing. Algorithms have been developed to run a total of six detection algorithms (four for rgb camera, one for thermal camera and one for a Multi-beam lidar) and fuse detection information in a common format using either 3D positions or Inverse Sensor Models. A GPU powered computational platform is able to run detection algorithms online. For the rgb camera, a deep learning algorithm is proposed DeepAnomaly to perform real-time anomaly detection of distant, heavy occluded and unknown obstacles in agriculture. DeepAnomaly is -- compared to a state-of-the-art object detector Faster R-CNN -- for an agricultural use-case able to detect humans better and at longer ranges (45-90m) using a smaller memory footprint and 7.3-times faster processing. Low memory footprint and fast processing makes DeepAnomaly suitable for real-time applications running on an embedded GPU. FieldSAFE is a multi-modal dataset for detection of static and moving obstacles in agriculture. The dataset includes synchronized recordings from a rgb camera, stereo camera, thermal camera, 360-degree camera, lidar and radar. Precise localization and pose is provided using IMU and GPS. Ground truth of static and moving obstacles (humans, mannequin dolls, barrels, buildings, vehicles, and vegetation) are available as an annotated orthophoto and GPS coordinates for moving obstacles. Detection information from multiple detection algorithms and sensors are fused into a map using Inverse Sensor Models and occupancy grid maps. This thesis presented many scientific contribution and state-of-the-art within perception for autonomous tractors; this includes a dataset, sensor platform, detection algorithms and procedures to perform multi-sensor fusion. Furthermore, important engineering contributions to autonomous farming vehicles are presented such as easily applicable, open-source software packages and algorithms that have been demonstrated in an end-to-end real-time detection system. The contributions of this thesis have demonstrated, addressed and solved critical issues to utilize camera-based perception systems that are essential to make autonomous vehicles in agriculture a reality

Ship recognition on the sea surface using aerial images taken by Uav : a deep learning approach

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial TechnologiesOceans are very important for mankind, because they are a very important source of food, they have a very large impact on the global environmental equilibrium, and it is over the oceans that most of the world commerce is done. Thus, maritime surveillance and monitoring, in particular identifying the ships used, is of great importance to oversee activities like fishing, marine transportation, navigation in general, illegal border encroachment, and search and rescue operations. In this thesis, we used images obtained with Unmanned Aerial Vehicles (UAVs) over the Atlantic Ocean to identify what type of ship (if any) is present in a given location. Images generated from UAV cameras suffer from camera motion, scale variability, variability in the sea surface and sun glares. Extracting information from these images is challenging and is mostly done by human operators, but advances in computer vision technology and development of deep learning techniques in recent years have made it possible to do so automatically. We used four of the state-of-art pretrained deep learning network models, namely VGG16, Xception, ResNet and InceptionResNet trained on ImageNet dataset, modified their original structure using transfer learning based fine tuning techniques and then trained them on our dataset to create new models. We managed to achieve very high accuracy (99.6 to 99.9% correct classifications) when classifying the ships that appear on the images of our dataset. With such a high success rate (albeit at the cost of high computing power), we can proceed to implement these algorithms on maritime patrol UAVs, and thus improve Maritime Situational Awareness

Improving the efficiency and accuracy of nocturnal bird Surveys through equipment selection and partial automation

Birds are a key environmental asset and this is recognised through comprehensive legislation and policy ensuring their protection and conservation. Many species are active at night and surveys are required to understand the implications of proposed developments such as towers and reduce possible conflicts with these structures. Night vision devices are commonly used in nocturnal surveys, either to scope an area for bird numbers and activity, or in remotely sensing an area to determine potential risk. This thesis explores some practical and theoretical approaches that can improve the accuracy, confidence and efficiency of nocturnal bird surveillance. As image intensifiers and thermal imagers have operational differences, each device has associated strengths and limitations. Empirical work established that image intensifiers are best used for species identification of birds against the ground or vegetation. Thermal imagers perform best in detection tasks and monitoring bird airspace usage. The typically used approach of viewing bird survey video from remote sensing in its entirety is a slow, inaccurate and inefficient approach. Accuracy can be significantly improved by viewing the survey video at half the playback speed. Motion detection efficiency and accuracy can be greatly improved through the use of adaptive background subtraction and cumulative image differencing. An experienced ornithologist uses bird flight style and wing oscillations to identify bird species. Changes in wing oscillations can be represented in a single inter-frame similarity matrix through area-based differencing. Bird species classification can then be automated using singular value decomposition to reduce the matrices to one-dimensional vectors for training a feed-forward neural network.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Object Detection and Classification in the Visible and Infrared Spectrums

The over-arching theme of this dissertation is the development of automated detection and/or classification systems for challenging infrared scenarios. The six works presented herein can be categorized into four problem scenarios. In the first scenario, long-distance detection and classification of vehicles in thermal imagery, a custom convolutional network architecture is proposed for small thermal target detection. For the second scenario, thermal face landmark detection and thermal cross-spectral face verification, a publicly-available visible and thermal face dataset is introduced, along with benchmark results for several landmark detection and face verification algorithms. Furthermore, a novel visible-to-thermal transfer learning algorithm for face landmark detection is presented. The third scenario addresses near-infrared cross-spectral periocular recognition with a coupled conditional generative adversarial network guided by auxiliary synthetic loss functions. Finally, a deep sparse feature selection and fusion is proposed to detect the presence of textured contact lenses prior to near-infrared iris recognition