Detecting animals in African Savanna with UAVs and the crowds

Unmanned aerial vehicles (UAVs) offer new opportunities for wildlife monitoring, with several advantages over traditional field-based methods. They have readily been used to count birds, marine mammals and large herbivores in different environments, tasks which are routinely performed through manual counting in large collections of images. In this paper, we propose a semi-automatic system able to detect large mammals in semi-arid Savanna. It relies on an animal-detection system based on machine learning, trained with crowd-sourced annotations provided by volunteers who manually interpreted sub-decimeter resolution color images. The system achieves a high recall rate and a human operator can then eliminate false detections with limited effort. Our system provides good perspectives for the development of data-driven management practices in wildlife conservation. It shows that the detection of large mammals in semi-arid Savanna can be approached by processing data provided by standard RGB cameras mounted on affordable fixed wings UAVs

CVABS: Moving Object Segmentation with Common Vector Approach for Videos

Background modelling is a fundamental step for several real-time computer vision applications that requires security systems and monitoring. An accurate background model helps detecting activity of moving objects in the video. In this work, we have developed a new subspace based background modelling algorithm using the concept of Common Vector Approach with Gram-Schmidt orthogonalization. Once the background model that involves the common characteristic of different views corresponding to the same scene is acquired, a smart foreground detection and background updating procedure is applied based on dynamic control parameters. A variety of experiments is conducted on different problem types related to dynamic backgrounds. Several types of metrics are utilized as objective measures and the obtained visual results are judged subjectively. It was observed that the proposed method stands successfully for all problem types reported on CDNet2014 dataset by updating the background frames with a self-learning feedback mechanism.Comment: 12 Pages, 4 Figures, 1 Tabl

Improving Deep Learning-based Defect Detection on Window Frames with Image Processing Strategies

Detecting subtle defects in window frames, including dents and scratches, is vital for upholding product integrity and sustaining a positive brand perception. Conventional machine vision systems often struggle to identify these defects in challenging environments like construction sites. In contrast, modern vision systems leveraging machine and deep learning (DL) are emerging as potent tools, particularly for cosmetic inspections. However, the promise of DL is yet to be fully realized. A few manufacturers have established a clear strategy for AI integration in quality inspection, hindered mainly by issues like scarce clean datasets and environmental changes that compromise model accuracy. Addressing these challenges, our study presents an innovative approach that amplifies defect detection in DL models, even with constrained data resources. The paper proposes a new defect detection pipeline called InspectNet (IPT-enhanced UNET) that includes the best combination of image enhancement and augmentation techniques for pre-processing the dataset and a Unet model tuned for window frame defect detection and segmentation. Experiments were carried out using a Spot Robot doing window frame inspections . 16 variations of the dataset were constructed using different image augmentation settings. Results of the experiments revealed that, on average, across all proposed evaluation measures, Unet outperformed all other algorithms when IPT-enhanced augmentations were applied. In particular, when using the best dataset, the average Intersection over Union (IoU) values achieved were IPT-enhanced Unet, reaching 0.91 of mIoU

Autonomous UAV for suspicious action detection using pictorial human pose estimation and classification

Visual autonomous systems capable of monitoring crowded areas and alerting the authorities in occurrence of a suspicious action can play a vital role in controlling crime rate. Previous atte mpts have been made to monitor crime using posture recognition but nothing exclusive to investigating actions of people in large populated area has been cited. In order resolve this shortcoming, we propose an autonomous unmanned aerial vehicle (UAV) visual surveillance system that locates humans in image frames followed by pose estimation using weak constraints on position, appearance of body parts and image parsing. The estimated pose, represented as a pictorial structure, is flagged using the proposed Hough Orientation Calculator (HOC) on close resemblance with any pose in the suspicious action dataset. The robustness of the system is demonstrated on videos recorded using a UAV with no prior knowledge of background, lighting or location and scale of the human in the image. The system produces an accuracy of 71% and can also be applied on various other video sources such as CCTV camera

Automatic Flood Detection in SentineI-2 Images Using Deep Convolutional Neural Networks

The early and accurate detection of floods from satellite imagery can aid rescue planning and assessment of geophysical damage. Automatic identification of water from satellite images has historically relied on hand-crafted functions, but these often do not provide the accuracy and robustness needed for accurate and early flood detection. To try to overcome these limitations we investigate a tiered methodology combining water index like features with a deep convolutional neural network based solution to flood identification against the MediaEval 2019 flood dataset. Our method builds on existing deep neural network methods, and in particular the VGG16 network. Specifically, we explored different water indexing techniques and proposed a water index function with the use of Green/SWIR and Blue/NIR bands with VGG16. Our experiment shows that our approach outperformed all other water index technique when combined with VGG16 network in order to detect flood in images

Benthic mapping of the Bluefields Bay fish sanctuary, Jamaica

Small island states, such as those in the Caribbean, are dependent on the nearshore marine ecosystem complex and its resources; the goods and services provided by seagrass and coral reef for example, are particularly indispensable to the tourism and fishing industries. In recognition of their valuable contributions and in an effort to promote sustainable use of marine resources, some nearshore areas have been designated as fish sanctuaries, as well as marine parks and protected areas. In order to effectively manage these coastal zones, a spatial basis is vital to understanding the ecological dynamics and ultimately inform management practices. However, the current extent of habitats within designated sanctuaries across Jamaica are currently unknown and owing to this, the Government of Jamaica is desirous of mapping the benthic features in these areas. Given the several habitat mapping methodologies that exist, it was deemed necessary to test the practicality of applying two remote sensing methods - optical and acoustic - at a pilot site in western Jamaica, the Bluefields Bay fish sanctuary. The optical remote sensing method involved a pixel-based supervised classification of two available multispectral images (WorldView-2 and GeoEye-1), whilst the acoustic method comprised a sonar survey using a BioSonics DT-X Portable Echosounder and subsequent indicator kriging interpolation in order to create continuous benthic surfaces. Image classification resulted in the mapping of three benthic classes, namely submerged vegetation, bare substrate and coral reef, with an overall map accuracy of 89.9% for WorldView-2 and 86.8% for GeoEye-1 imagery. These accuracies surpassed those of the acoustic classification method, which attained 76.6% accuracy for vegetation presence, and 53.5% for bottom substrate (silt, sand and coral reef/ hard bottom). Both approaches confirmed that the Bluefields Bay is dominated by submerged aquatic vegetation, with contrastingly smaller areas of bare sediment and coral reef patches. Additionally, the sonar revealed that silty substrate exists along the shoreline, whilst sand is found further offshore. Ultimately, the methods employed in this study were compared and although it was found that satellite image classification was perhaps the most cost-effective and well-suited for Jamaica given current available equipment and expertise, it is acknowledged that acoustic technology offers greater thematic detail required by a number of stakeholders and is capable of operating in turbid waters and cloud covered environments ill-suited for image classification. On the contrary, a major consideration for the acoustic classification process is the interpolation of processed data; this step gives rise to a number of potential limitations, such as those associated with the choice of interpolation algorithm, available software and expertise. The choice in mapping approach, as well as the survey design and processing steps is not an easy task; however the results of this study highlight the various benefits and shortcomings of implementing optical and acoustic classification approaches in Jamaica.Persons automatically associate tropical waters with spectacular views of coral reefs and colourful fish; however many are perhaps not aware that these coral reefs, as well as other living organisms inhabiting the seabed are in fact extremely valuable to our existence. Healthy coral reefs and seagrass assist in maintaining the sand on our beaches and fish populations and are thereby crucial to the tourism and fishing industries in the Caribbean. For this reason, a number of areas are protected by law and have been designated fish sanctuaries or marine protected areas. In order to understand the functioning of theses areas and effectively inform management strategy, the configuration of what exists on the seafloor is crucial. In the same vein that a motorist needs a road map to navigate unknown areas, coastal stakeholders require maps of the seafloor in order to understand what is happening beneath the water’s surface. The location of seafloor habitats within fish sanctuaries in Jamaica are currently unknown and the Government is interested in mapping them. However a myriad of methods exist that could be employed to achieve this goal. Remote sensing is a broad grouping of methods that involve collecting information about an object without being in direct physical contact with it. Many researchers have successfully mapped marine areas using these techniques and it was believed crucial to test the practicality of two such methods, specifically optical and acoustic remote sensing. The main question to be answered from this study was therefore: Which mapping approach is better for benthic habitat mapping in Jamaica and possibly the wider Caribbean? Optical remote sensing relates to the interaction of energy with the Earth’s surface. A digital photograph is taken from a satellite and subsequently interpreted. Acoustic/ sonar technology involves the recording of waveforms reflected from the seabed. Both methods were employed at a pilot site, the Bluefields Bay fish sanctuary, situated in western Jamaica. The optical remote sensing method involved the classification of two satellite images (named WorldView-2 and GeoEye-1) and this process was informed using known positions of seafloor features, this being known as supervised image classification. With regard to the acoustic method, a field survey utilising sonar equipment (BioSonics DT-X Portable Echosounder) was undertaken in order to collect the necessary sonar data. The processed field data was modelled in order to convert lines of field point data to one continuous map of the sanctuary, a process known as interpolation. The accuracy of each method was then tested using field knowledge of what exists in the sanctuary. The map resulting from the image classification revealed three seafloor types, namely submerged vegetation, coral reef and bare seafloor. The overall map accuracy was 89.9% for the WorldView-2 image and 86.8% for GeoEye-1 imagery. These accuracies surpassed those attained from the acoustic classification method (76.6% for vegetation presence and 53.5% for bottom type - silt, sand and coral reef/ hard bottom). Similar to previous studies undertaken, it was shown that the seabed of Bluefields Bay is primarily inhabited by submerged aquatic vegetation (including seagrass and algae), with contrastingly smaller areas of bare sediment and coral reef. Ultimately, the methods employed in this study were compared and the pros and cons of each were weighed in order to deem one method more suitable in Jamaica. Often, the presence of cloud and suspended matter in the water block the view of the seafloor making image classification difficult. On the contrary, acoustic surveys are capable of operating throughout cloudy conditions and attaining more detailed information of the ocean floor, otherwise not possible with optical remote sensing. A major step in the acoustic classification process however, was the interpolation of processed data, which may introduce additional limitations if careful consideration is not given to the intricacies of the process. Lastly, the acoustic survey certainly required greater financial resources than satellite image classification. In answer to the main question of this study, the most cost effective and feasible mapping method for Jamaica is satellite image classification (based on the results attained). It must be stressed however that the effective implementation of any method will depend on a number of factors, such as available software, equipment, expertise and user needs, that must be weighed in order to select the most feasible mapping method for a particular site

Real-time vehicle detection using low-cost sensors

Improving road safety and reducing the number of accidents is one of the top priorities for the automotive industry. As human driving behaviour is one of the top causation factors of road accidents, research is working towards removing control from the human driver by automating functions and finally introducing a fully Autonomous Vehicle (AV). A Collision Avoidance System (CAS) is one of the key safety systems for an AV, as it ensures all potential threats ahead of the vehicle are identified and appropriate action is taken. This research focuses on the task of vehicle detection, which is the base of a CAS, and attempts to produce an effective vehicle detector based on the data coming from a low-cost monocular camera. Developing a robust CAS based on low-cost sensor is crucial to bringing the cost of safety systems down and in this way, increase their adoption rate by end users. In this work, detectors are developed based on the two main approaches to vehicle detection using a monocular camera. The first is the traditional image processing approach where visual cues are utilised to generate potential vehicle locations and at a second stage, verify the existence of vehicles in an image. The second approach is based on a Convolutional Neural Network, a computationally expensive method that unifies the detection process in a single pipeline. The goal is to determine which method is more appropriate for real-time applications. Following the first approach, a vehicle detector based on the combination of HOG features and SVM classification is developed. The detector attempts to optimise performance by modifying the detection pipeline and improve run-time performance. For the CNN-based approach, six different network models are developed and trained end to end using collected data, each with a different network structure and parameters, in an attempt to determine which combination produces the best results. The evaluation of the different vehicle detectors produced some interesting findings; the first approach did not manage to produce a working detector, while the CNN-based approach produced a high performing vehicle detector with an 85.87% average precision and a very low miss rate. The detector managed to perform well under different operational environments (motorway, urban and rural roads) and the results were validated using an external dataset. Additional testing of the vehicle detector indicated it is suitable as a base for safety applications such as CAS, with a run time performance of 12FPS and potential for further improvements.</div

Grounding semantics in robots for Visual Question Answering

In this thesis I describe an operational implementation of an object detection and description system that incorporates in an end-to-end Visual Question Answering system and evaluated it on two visual question answering datasets for compositional language and elementary visual reasoning

Mapping three-dimensional geological features from remotely-sensed images and digital elevation models.

Accurate mapping of geological structures is important in numerous applications, ranging from mineral exploration through to hydrogeological modelling. Remotely sensed data can provide synoptic views of study areas enabling mapping of geological units within the area. Structural information may be derived from such data using standard manual photo-geologic interpretation techniques, although these are often inaccurate and incomplete. The aim of this thesis is, therefore, to compile a suite of automated and interactive computer-based analysis routines, designed to help a the user map geological structure. These are examined and integrated in the context of an expert system. The data used in this study include Digital Elevation Model (DEM) and Airborne Thematic Mapper images, both with a spatial resolution of 5m, for a 5 x 5 km area surrounding Llyn Cow lyd, Snowdonia, North Wales. The geology of this area comprises folded and faulted Ordo vician sediments intruded throughout by dolerite sills, providing a stringent test for the automated and semi-automated procedures. The DEM is used to highlight geomorphological features which may represent surface expressions of the sub-surface geology. The DEM is created from digitized contours, for which kriging is found to provide the best interpolation routine, based on a number of quantitative measures. Lambertian shading and the creation of slope and change of slope datasets are shown to provide the most successful enhancement of DEMs, in terms of highlighting a range of key geomorphological features. The digital image data are used to identify rock outcrops as well as lithologically controlled features in the land cover. To this end, a series of standard spectral enhancements of the images is examined. In this respect, the least correlated 3 band composite and a principal component composite are shown to give the best visual discrimination of geological and vegetation cover types. Automatic edge detection (followed by line thinning and extraction) and manual interpretation techniques are used to identify a set of 'geological primitives' (linear or arc features representing lithological boundaries) within these data. Inclusion of the DEM data provides the three-dimensional co-ordinates of these primitives enabling a least-squares fit to be employed to calculate dip and strike values, based, initially, on the assumption of a simple, linearly dipping structural model. A very large number of scene 'primitives' is identified using these procedures, only some of which have geological significance. Knowledge-based rules are therefore used to identify the relevant. For example, rules are developed to identify lake edges, forest boundaries, forest tracks, rock-vegetation boundaries, and areas of geomorphological interest. Confidence in the geological significance of some of the geological primitives is increased where they are found independently in both the DEM and remotely sensed data. The dip and strike values derived in this way are compared to information taken from the published geological map for this area, as well as measurements taken in the field. Many results are shown to correspond closely to those taken from the map and in the field, with an error of < 1°. These data and rules are incorporated into an expert system which, initially, produces a simple model of the geological structure. The system also provides a graphical user interface for manual control and interpretation, where necessary. Although the system currently only allows a relatively simple structural model (linearly dipping with faulting), in the future it will be possible to extend the system to model more complex features, such as anticlines, synclines, thrusts, nappes, and igneous intrusions