Adversarial Patch Camouflage against Aerial Detection

Detection of military assets on the ground can be performed by applying deep learning-based object detectors on drone surveillance footage. The traditional way of hiding military assets from sight is camouflage, for example by using camouflage nets. However, large assets like planes or vessels are difficult to conceal by means of traditional camouflage nets. An alternative type of camouflage is the direct misleading of automatic object detectors. Recently, it has been observed that small adversarial changes applied to images of the object can produce erroneous output by deep learning-based detectors. In particular, adversarial attacks have been successfully demonstrated to prohibit person detections in images, requiring a patch with a specific pattern held up in front of the person, thereby essentially camouflaging the person for the detector. Research into this type of patch attacks is still limited and several questions related to the optimal patch configuration remain open. This work makes two contributions. First, we apply patch-based adversarial attacks for the use case of unmanned aerial surveillance, where the patch is laid on top of large military assets, camouflaging them from automatic detectors running over the imagery. The patch can prevent automatic detection of the whole object while only covering a small part of it. Second, we perform several experiments with different patch configurations, varying their size, position, number and saliency. Our results show that adversarial patch attacks form a realistic alternative to traditional camouflage activities, and should therefore be considered in the automated analysis of aerial surveillance imagery.Comment: 9 page

Interactive detection of incrementally learned concepts in images with ranking and semantic query interpretation

This research was performed in the GOOSE project, which is jointly funded by the MIST research program of the Dutch Ministry of Defense and the AMSN enabling technology program.The number of networked cameras is growing exponentially. Multiple applications in different domains result in an increasing need to search semantically over video sensor data. In this paper, we present the GOOSE demonstrator, which is a real-time general-purpose search engine that allows users to pose natural language queries to retrieve corresponding images. Top-down, this demonstrator interprets queries, which are presented as an intuitive graph to collect user feedback. Bottomup, the system automatically recognizes and localizes concepts in images and it can incrementally learn novel concepts. A smart ranking combines both and allows effective retrieval of relevant images.peer-reviewe

Recognition and localization of relevant human behavior in videos, SPIE,

ABSTRACT Ground surveillance is normally performed by human assets, since it requires visual intelligence. However, especially for military operations, this can be dangerous and is very resource intensive. Therefore, unmanned autonomous visualintelligence systems are desired. In this paper, we present an improved system that can recognize actions of a human and interactions between multiple humans. Central to the new system is our agent-based architecture. The system is trained on thousands of videos and evaluated on realistic persistent surveillance data in the DARPA Mind's Eye program, with hours of videos of challenging scenes. The results show that our system is able to track the people, detect and localize events, and discriminate between different behaviors, and it performs 3.4 times better than our previous system

Development Of A Neural Network Embedding For Quantifying Crack Pattern Similarity In Masonry Structures

The degree of similarity between damage patterns often correlates with the likelihood of having similar damage causes. Therefore, deciding whether crack patterns are similar is one of the key steps in assessing the conditions of masonry structures. To our knowledge, no literature has been published regarding masonry crack pattern similarity measures that would correlate well with assessment by structural engineers. Hence, currently, similarity assessments are solely performed by experts and require considerable time and effort. Moreover, it is expensive, limited by the availability of experts, and yields only qualitative answers. In this work, we propose an automated approach that has the potential to overcome the above shortcomings and perform comparably with experts. At its core is a deep neural network embedding that can be used to calculate a numerical distance between crack patterns on comparable façades. The embedding is obtained from fitting a deep neural network to perform a classification task; i.e., to predict the crack pattern archetype label from a crack pattern image. The network is fitted to synthetic crack patterns simulated using a statistics-based approach proposed in this work. The simulation process can account for important crack pattern characteristics such as crack location, orientation, and length. The embedding transforms a crack pattern (raster image) into a 64-dimensional real-valued vector space where the closeness between two vectors is calculated as the cosine of their angle. The proposed approach is tested on 2D façades with and without openings, and with synthetic crack patterns that consist of a single crack and multiple cracks.Geo-engineerin

Development Of A Neural Network Embedding For Quantifying Crack Pattern Similarity In Masonry Structures

The degree of similarity between damage patterns often correlates with the likelihood of having similar damage causes. Therefore, deciding whether crack patterns are similar is one of the key steps in assessing the conditions of masonry structures. To our knowledge, no literature has been published regarding masonry crack pattern similarity measures that would correlate well with assessment by structural engineers. Hence, currently, similarity assessments are solely performed by experts and require considerable time and effort. Moreover, it is expensive, limited by the availability of experts, and yields only qualitative answers. In this work, we propose an automated approach that has the potential to overcome the above shortcomings and perform comparably with experts. At its core is a deep neural network embedding that can be used to calculate a numerical distance between crack patterns on comparable façades. The embedding is obtained from fitting a deep neural network to perform a classification task; i.e., to predict the crack pattern archetype label from a crack pattern image. The network is fitted to synthetic crack patterns simulated using a statistics-based approach proposed in this work. The simulation process can account for important crack pattern characteristics such as crack location, orientation, and length. The embedding transforms a crack pattern (raster image) into a 64-dimensional real-valued vector space where the closeness between two vectors is calculated as the cosine of their angle. The proposed approach is tested on 2D façades with and without openings, and with synthetic crack patterns that consist of a single crack and multiple cracks

Flexible image analysis for law enforcement agencies with deep neural networks to determine: where, who and what

International audienceDue to the increasing need for effective security measures and the integration of cameras in commercial products, a hugeamount of visual data is created today. Law enforcement agencies (LEAs) are inspecting images and videos to findradicalization, propaganda for terrorist organizations and illegal products on darknet markets. This is time consuming.Instead of an undirected search, LEAs would like to adapt to new crimes and threats, and focus only on data from specificlocations, persons or objects, which requires flexible interpretation of image content. Visual concept detection with deepconvolutional neural networks (CNNs) is a crucial component to understand the image content. This paper has fivecontributions. The first contribution allows image-based geo-localization to estimate the origin of an image. CNNs andgeotagged images are used to create a model that determines the location of an image by its pixel values. The secondcontribution enables analysis of fine-grained concepts to distinguish sub-categories in a generic concept. The proposedmethod encompasses data acquisition and cleaning and concept hierarchies. The third contribution is the recognition ofperson attributes (e.g., glasses or moustache) to enable query by textual description for a person. The person-attributeproblem is treated as a specific sub-task of concept classification. The fourth contribution is an intuitive image annotationtool based on active learning. Active learning allows users to define novel concepts flexibly and train CNNs with minimalannotation effort. The fifth contribution increases the flexibility for LEAs in the query definition by using query expansion.Query expansion maps user queries to known and detectable concepts. Therefore, no prior knowledge of the detectableconcepts is required for the users. The methods are validated on data with varying locations (popular and non-touristiclocations), varying person attributes (CelebA dataset), and varying number of annotations