Background Adaptive Faster R-CNN for Semi-Supervised Convolutional Object Detection of Threats in X-Ray Images

Recently, progress has been made in the supervised training of Convolutional Object Detectors (e.g. Faster R-CNN) for threat recognition in carry-on luggage using X-ray images. This is part of the Transportation Security Administration's (TSA's) mission to protect air travelers in the United States. While more training data with threats may reliably improve performance for this class of deep algorithm, it is expensive to stage in realistic contexts. By contrast, data from the real world can be collected quickly with minimal cost. In this paper, we present a semi-supervised approach for threat recognition which we call Background Adaptive Faster R-CNN. This approach is a training method for two-stage object detectors which uses Domain Adaptation methods from the field of deep learning. The data sources described earlier make two "domains": a hand-collected data domain of images with threats, and a real-world domain of images assumed without threats. Two domain discriminators, one for discriminating object proposals and one for image features, are adversarially trained to prevent encoding domain-specific information. Without this penalty a Convolutional Neural Network (CNN) can learn to identify domains based on superficial characteristics, and minimize a supervised loss function without improving its ability to recognize objects. For the hand-collected data, only object proposals and image features from backgrounds are used. The losses for these domain-adaptive discriminators are added to the Faster R-CNN losses of images from both domains. This can reduce threat detection false alarm rates by matching the statistics of extracted features from hand-collected backgrounds to real world data. Performance improvements are demonstrated on two independently-collected datasets of labeled threats

‘Unexpected item in the bagging area’: Anomaly Detection in X-ray Security Images

The role of Anomaly Detection in X-ray security imaging, as a supplement to targeted threat detection, is described; and a taxonomy of anomalies types in this domain is presented. Algorithms are described for detecting appearance anomalies, of shape, texture and density; and semantic anomalies of object category presence. The anomalies are detected on the basis of representations extracted from a convolutional neural network pre-trained to identify object categories in photographs: from the final pooling layer for appearance anomalies, and from the logit layer for semantic anomalies. The distribution of representations in normal data are modelled using high-dimensional, full-covariance, Gaussians; and anomalies are scored according to their likelihood relative to those models. The algorithms are tested on X-ray parcel images using stream-of-commerce data as the normal class, and parcels with firearms present as examples of anomalies to be detected. Despite the representations being learnt for photographic images, and the varied contents of stream-of-commerce parcels; the system, trained on stream-of-commerce images only, is able to detect 90% of firearms as anomalies, while raising false alarms on 18% of stream-of-commerce

Automated Analysis of X-ray Images for Cargo Security

Customs and border officers are overwhelmed by the hundreds of millions of cargo containers that constitute the backbone of the global supply chain, any one of which could contain a security- or customs-related threat. Searching for these threats is akin to searching for needles in an ever-growing field of haystacks. This thesis considers novel automated image analysis methods to automate or assist elements of cargo inspection. The four main contributions of this thesis are as follows. Methods are proposed for the measurement and correction of detector wobble in large-scale transmission radiography using Beam Position Detectors (BPDs). Wobble is estimated from BPD measurements using a Random Regression Forest (RRF) model, Bayesian fused with a prior estimate from an Auto-Regression (AR). Next, a series of image corrections are derived, and it is shown that 87% of image error due to wobble can be corrected. This is the first proposed method for correction of wobble in large-scale transmission radiography. A Threat Image Projection (TIP) framework is proposed, for training, probing and evaluating Automated Threat Detection (ATD) algorithms. The TIP method is validated experimentally, and a method is proposed to test whether algorithms can learn to exploit TIP artefacts. A system for Empty Container Verification (ECV) is proposed. The system, trained using TIP, is based on Random Forest (RF) classification of image patches according to fixed geometric features and container location. The method outperforms previous reported results, and is able to detect very small amounts of synthetically concealed smuggled contraband. Finally, a method for ATD is proposed, based on a deep Convolutional Neural Network (CNN), trained from scratch using TIP, and exploits the material information encoded within dual-energy X-ray images to suppress false alarms. The system offers a 100-fold improvement in the false positive rate over prior work

A holistic review of cybersecurity and reliability perspectives in smart airports

Advances in the Internet of Things (IoT) and aviation sector have resulted in the emergence of smart airports. Services and systems powered by the IoT enable smart airports to have enhanced robustness, efficiency and control, governed by real-time monitoring and analytics. Smart sensors control the environmental conditions inside the airport, automate passenger-related actions and support airport security. However, these augmentations and automation introduce security threats to network systems of smart airports. Cyber-attackers demonstrated the susceptibility of IoT systems and networks to Advanced Persistent Threats (APT), due to hardware constraints, software flaws or IoT misconfigurations. With the increasing complexity of attacks, it is imperative to safeguard IoT networks of smart airports and ensure reliability of services, as cyber-attacks can have tremendous consequences such as disrupting networks, cancelling travel, or stealing sensitive information. There is a need to adopt and develop new Artificial Intelligence (AI)-enabled cyber-defence techniques for smart airports, which will address the challenges brought about by the incorporation of IoT systems to the airport business processes, and the constantly evolving nature of contemporary cyber-attacks. In this study, we present a holistic review of existing smart airport applications and services enabled by IoT sensors and systems. Additionally, we investigate several types of cyber defence tools including AI and data mining techniques, and analyse their strengths and weaknesses in the context of smart airports. Furthermore, we provide a classification of smart airport sub-systems based on their purpose and criticality and address cyber threats that can affect the security of smart airport\u27s networks

A holistic review of cybersecurity and reliability perspectives in smart airports

Advances in the Internet of Things (IoT) and aviation sector have resulted in the emergence of smart airports. Services and systems powered by the IoT enable smart airports to have enhanced robustness, efficiency and control, governed by real-time monitoring and analytics. Smart sensors control the environmental conditions inside the airport, automate passenger-related actions and support airport security. However, these augmentations and automation introduce security threats to network systems of smart airports. Cyber-attackers demonstrated the susceptibility of IoT systems and networks to Advanced Persistent Threats (APT), due to hardware constraints, software flaws or IoT misconfigurations. With the increasing complexity of attacks, it is imperative to safeguard IoT networks of smart airports and ensure reliability of services, as cyber-attacks can have tremendous consequences such as disrupting networks, cancelling travel, or stealing sensitive information. There is a need to adopt and develop new Artificial Intelligence (AI)-enabled cyber-defence techniques for smart airports, which will address the challenges brought about by the incorporation of IoT systems to the airport business processes, and the constantly evolving nature of contemporary cyber-attacks. In this study, we present a holistic review of existing smart airport applications and services enabled by IoT sensors and systems. Additionally, we investigate several types of cyber defence tools including AI and data mining techniques, and analyse their strengths and weaknesses in the context of smart airports. Furthermore, we provide a classification of smart airport sub-systems based on their purpose and criticality and address cyber threats that can affect the security of smart airport\u27s networks

A holistic review of cybersecurity and reliability perspectives in smart airports

Advances in the Internet of Things (IoT) and aviation sector have resulted in the emergence of smart airports. Services and systems powered by the IoT enable smart airports to have enhanced robustness, efficiency and control, governed by real-time monitoring and analytics. Smart sensors control the environmental conditions inside the airport, automate passenger-related actions and support airport security. However, these augmentations and automation introduce security threats to network systems of smart airports. Cyber-attackers demonstrated the susceptibility of IoT systems and networks to Advanced Persistent Threats (APT), due to hardware constraints, software flaws or IoT misconfigurations. With the increasing complexity of attacks, it is imperative to safeguard IoT networks of smart airports and ensure reliability of services, as cyber-attacks can have tremendous consequences such as disrupting networks, cancelling travel, or stealing sensitive information. There is a need to adopt and develop new Artificial Intelligence (AI)-enabled cyber-defence techniques for smart airports, which will address the challenges brought about by the incorporation of IoT systems to the airport business processes, and the constantly evolving nature of contemporary cyber-attacks. In this study, we present a holistic review of existing smart airport applications and services enabled by IoT sensors and systems. Additionally, we investigate several types of cyber defence tools including AI and data mining techniques, and analyse their strengths and weaknesses in the context of smart airports. Furthermore, we provide a classification of smart airport sub-systems based on their purpose and criticality and address cyber threats that can affect the security of smart airport\u27s networks

Machine learning and mixed reality for smart aviation: applications and challenges

The aviation industry is a dynamic and ever-evolving sector. As technology advances and becomes more sophisticated, the aviation industry must keep up with the changing trends. While some airlines have made investments in machine learning and mixed reality technologies, the vast majority of regional airlines continue to rely on inefficient strategies and lack digital applications. This paper investigates the state-of-the-art applications that integrate machine learning and mixed reality into the aviation industry. Smart aerospace engineering design, manufacturing, testing, and services are being explored to increase operator productivity. Autonomous systems, self-service systems, and data visualization systems are being researched to enhance passenger experience. This paper investigate safety, environmental, technological, cost, security, capacity, and regulatory challenges of smart aviation, as well as potential solutions to ensure future quality, reliability, and efficiency

Efficient Multi-Objective NeuroEvolution in Computer Vision and Applications for Threat Identification

Concealed threat detection is at the heart of critical security systems designed to en- sure public safety. Currently, methods for threat identification and detection are primarily manual, but there is a recent vision to automate the process. Problematically, developing computer vision models capable of operating in a wide range of settings, such as the ones arising in threat detection, is a challenging task involving multiple (and often conflicting) objectives. Automated machine learning (AutoML) is a flourishing field which endeavours to dis- cover and optimise models and hyperparameters autonomously, providing an alternative to classic, effort-intensive hyperparameter search. However, existing approaches typ- ically show significant downsides, like their (1) high computational cost/greediness in resources, (2) limited (or absent) scalability to custom datasets, (3) inability to provide competitive alternatives to expert-designed and heuristic approaches and (4) common consideration of a single objective. Moreover, most existing studies focus on standard classification tasks and thus cannot address a plethora of problems in threat detection and, more broadly, in a wide variety of compelling computer vision scenarios. This thesis leverages state-of-the-art convolutional autoencoders and semantic seg- mentation (Chapter 2) to develop effective multi-objective AutoML strategies for neural architecture search. These strategies are designed for threat detection and provide in- sights into some quintessential computer vision problems. To this end, the thesis first introduces two new models, a practical Multi-Objective Neuroevolutionary approach for Convolutional Autoencoders (MONCAE, Chapter 3) and a Resource-Aware model for Multi-Objective Semantic Segmentation (RAMOSS, Chapter 4). Interestingly, these ap- proaches reached state-of-the-art results using a fraction of computational resources re- quired by competing systems (0.33 GPU days compared to 3150), yet allowing for mul- tiple objectives (e.g., performance and number of parameters) to be simultaneously op- timised. This drastic speed-up was possible through the coalescence of neuroevolution algorithms with a new heuristic technique termed Progressive Stratified Sampling. The presented methods are evaluated on a range of benchmark datasets and then applied to several threat detection problems, outperforming previous attempts in balancing multiple objectives. The final chapter of the thesis focuses on thread detection, exploiting these two mod- els and novel components. It presents first a new modification of specialised proxy scores to be embedded in RAMOSS, enabling us to further accelerate the AutoML process even more drastically while maintaining avant-garde performance (above 85% precision for SIXray). This approach rendered a new automatic evolutionary Multi-objEctive method for cOncealed Weapon detection (MEOW), which outperforms state-of-the-art models for threat detection in key datasets: a gold standard benchmark (SixRay) and a security- critical, proprietary dataset. Finally, the thesis shifts the focus from neural architecture search to identifying the most representative data samples. Specifically, the Multi-objectIve Core-set Discovery through evolutionAry algorithMs in computEr vision approach (MIRA-ME) showcases how the new neural architecture search techniques developed in previous chapters can be adapted to operate on data space. MIRA-ME offers supervised and unsupervised ways to select maximally informative, compact sets of images via dataset compression. This operation can offset the computational cost further (above 90% compression), with a minimal sacrifice in performance (less than 5% for MNIST and less than 13% for SIXray). Overall, this thesis proposes novel model- and data-centred approaches towards a more widespread use of AutoML as an optimal tool for architecture and coreset discov- ery. With the presented and future developments, the work suggests that AutoML can effectively operate in real-time and performance-critical settings such as in threat de- tection, even fostering interpretability by uncovering more parsimonious optimal models. More widely, these approaches have the potential to provide effective solutions to chal- lenging computer vision problems that nowadays are typically considered unfeasible for AutoML settings