Object Detection in X-ray Images Using Transfer Learning with Data Augmentation

Object detection in X-ray images is an interesting problem in the field of machine vision. The reason is that images from an X-ray machine are usually obstructed with other objects and to itself, therefore object classification and localization is a challenging task. Furthermore, obtaining X-ray data is difficult due to an insufficient dataset available compared with photographic images from a digital camera. It is vital to easily detect objects in an X-ray image because it can be used as decision support in the detection of threat items such as improvised explosive devices (IED’s) in airports, train stations, and public places. Detection of IED components accurately requires an expert and can be achieved through extensive training. Also, manual inspection is tedious, and the probability of missed detection increases due to several pieces of baggage are scanned in a short period of time. As a solution, this paper used different object detection techniques (Faster R-CNN, SSD, R-FCN) and feature extractors (ResNet, MobileNet, Inception, Inception-ResNet) based on convolutional neural networks (CNN) in a novel IEDXray dataset in the detection of IED components. The IEDXray dataset is an X-ray image of IED replicas without the explosive material. Transfer learning with data augmentation was performed due to limited X-ray data available to train the whole network from scratch. Evaluation results showed that individual detection achieved 99.08% average precision (AP) in mortar detection and 77.29% mAP in three IED components

Tackling the X-ray cargo inspection challenge using machine learning

The current infrastructure for non-intrusive inspection of cargo containers cannot accommodate exploding com-merce volumes and increasingly stringent regulations. There is a pressing need to develop methods to automate parts of the inspection workflow, enabling expert operators to focus on a manageable number of high-risk images. To tackle this challenge, we developed a modular framework for automated X-ray cargo image inspection. Employing state-of-the-art machine learning approaches, including deep learning, we demonstrate high performance for empty container verification and specific threat detection. This work constitutes a significant step towards the partial automation of X-ray cargo image inspection

Automated X-ray image analysis for cargo security: Critical review and future promise

We review the relatively immature field of automated image analysis for X-ray cargo imagery. There is increasing demand for automated analysis methods that can assist in the inspection and selection of containers, due to the ever-growing volumes of traded cargo and the increasing concerns that customs- and security-related threats are being smuggled across borders by organised crime and terrorist networks. We split the field into the classical pipeline of image preprocessing and image understanding. Preprocessing includes: image manipulation; quality improvement; Threat Image Projection (TIP); and material discrimination and segmentation. Image understanding includes: Automated Threat Detection (ATD); and Automated Contents Verification (ACV). We identify several gaps in the literature that need to be addressed and propose ideas for future research. Where the current literature is sparse we borrow from the single-view, multi-view, and CT X-ray baggage domains, which have some characteristics in common with X-ray cargo

Meta-Transfer Learning Driven Tensor-Shot Detector for the Autonomous Localization and Recognition of Concealed Baggage Threats

Screening baggage against potential threats has become one of the prime aviation security concerns all over the world, where manual detection of prohibited items is a time-consuming and hectic process. Many researchers have developed autonomous systems to recognize baggage threats using security X-ray scans. However, all of these frameworks are vulnerable against screening cluttered and concealed contraband items. Furthermore, to the best of our knowledge, no framework possesses the capacity to recognize baggage threats across multiple scanner specifications without an explicit retraining process. To overcome this, we present a novel meta-transfer learning-driven tensor-shot detector that decomposes the candidate scan into dual-energy tensors and employs a meta-one-shot classification backbone to recognize and localize the cluttered baggage threats. In addition, the proposed detection framework can be well-generalized to multiple scanner specifications due to its capacity to generate object proposals from the unified tensor maps rather than diversified raw scans. We have rigorously evaluated the proposed tensor-shot detector on the publicly available SIXray and GDXray datasets (containing a cumulative of 1,067,381 grayscale and colored baggage X-ray scans). On the SIXray dataset, the proposed framework achieved a mean average precision (mAP) of 0.6457, and on the GDXray dataset, it achieved the precision and F1 score of 0.9441 and 0.9598, respectively. Furthermore, it outperforms state-of-the-art frameworks by 8.03% in terms of mAP, 1.49% in terms of precision, and 0.573% in terms of F1 on the SIXray and GDXray dataset, respectively

High-Level Information Fusion in Visual Sensor Networks

Information fusion techniques combine data from multiple sensors, along with additional information and knowledge, to obtain better estimates of the observed scenario than could be achieved by the use of single sensors or information sources alone. According to the JDL fusion process model, high-level information fusion is concerned with the computation of a scene representation in terms of abstract entities such as activities and threats, as well as estimating the relationships among these entities. Recent experiences confirm that context knowledge plays a key role in the new-generation high-level fusion systems, especially in those involving complex scenarios that cause the failure of classical statistical techniques –as it happens in visual sensor networks. In this chapter, we study the architectural and functional issues of applying context information to improve high-level fusion procedures, with a particular focus on visual data applications. The use of formal knowledge representations (e.g. ontologies) is a promising advance in this direction, but there are still some unresolved questions that must be more extensively researched.The UC3M Team gratefully acknowledges that this research activity is supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC, CAM CONTEXTS (S2009/TIC-1485) and DPS2008-07029-C02-02

Human factors in X-ray image inspection of passenger Baggage – Basic and applied perspectives

The X-ray image inspection of passenger baggage contributes substantially to aviation security and is best understood as a search and decision task: Trained security officers – so called screeners – search the images for threats among many harmless everyday objects, but the recognition of objects in X-ray images and therefore the decision between threats and harmless objects can be difficult. Because performance in this task depends on often difficult recognition, it is not clear to what extent basic research on visual search can be generalized to X-ray image inspection. Manuscript 1 of this thesis investigated whether X-ray image inspection and a traditional visual search task depend on the same visual-cognitive abilities. The results indicate that traditional visual search tasks and X-ray image inspection depend on different aspects of common visual-cognitive abilities. Another gap between basic research on visual search and applied research on X-ray image inspection is that the former is typically conducted with students and the latter with professional screeners. Therefore, these two populations were compared, revealing that professionals performed better in X-ray image inspection, but not the visual search task. However, there was no difference between students and professionals regarding the importance of the visual-cognitive abilities for either task. Because there is some freedom in the decision whether a suspicious object should be declared as a threat or as harmless, the results of X-ray image inspection in terms of hit and false alarm rate depend on the screeners’ response tendency. Manuscript 2 evaluated whether three commonly used detection measures – ${d}'$, ${A}'$, and ${d}_{a}$ – are a valid representation of detection performance that is independent from response tendency. The results were consistently in favor of da with a slope parameter of around 0.6. In Manuscript 3 it was further shown that screeners can change their response tendency to increase the detection of novel threats. Also, screeners with a high ability to recognize everyday objects detected more novel threats when their response tendency was manipulated. The thesis further addressed changes that screeners face due to technological developments. Manuscript 4 showed that screeners can inspect X-ray images for one hour straight without a decrease in performance under conditions of remote cabin baggage screening, which means that X-ray image inspection is performed in a quiet room remote from the checkpoint. These screeners did not show a lower performance, but reported more distress, compared to screeners who took a 10 min break after every 20 min of screening. Manuscript 5 evaluated detection systems for cabin baggage screening (EDSCB). EDSCB only increased the detection of improvised explosive devices (IEDs) for inexperienced screeners if alarms by the EDSCB were indicated on the image and the screeners had to decide whether a threat was present or not. The detection of mere explosives, which lack the triggering device of IEDs, was only increased if the screeners could not decide against an alarm by the EDSCB. Manuscript 6 used discrete event simulation to evaluate how EDSCB impacts the throughput of passenger baggage screening. Throughput decreased with increasing false alarm rate of the EDSCB. However, fast alarm resolution processes and screeners with a low false alarm rate increased throughput. Taken together, the present findings contribute to understanding X-ray image inspection as a task with a search and decision component. The findings provide insights into basic aspects like the required visual-cognitive abilities and valid measures of detection performance, but also into applied research questions like for how long X-ray image inspection can be performed and how automation can assist with the detection of explosives

Intriguing Properties of Adversarial ML Attacks in the Problem Space

Recent research efforts on adversarial ML have investigated problem-space attacks, focusing on the generation of real evasive objects in domains where, unlike images, there is no clear inverse mapping to the feature space (e.g., software). However, the design, comparison, and real-world implications of problem-space attacks remain underexplored. This paper makes two major contributions. First, we propose a novel formalization for adversarial ML evasion attacks in the problem-space, which includes the definition of a comprehensive set of constraints on available transformations, preserved semantics, robustness to preprocessing, and plausibility. We shed light on the relationship between feature space and problem space, and we introduce the concept of side-effect features as the byproduct of the inverse feature-mapping problem. This enables us to define and prove necessary and sufficient conditions for the existence of problem-space attacks. We further demonstrate the expressive power of our formalization by using it to describe several attacks from related literature across different domains. Second, building on our formalization, we propose a novel problem-space attack on Android malware that overcomes past limitations. Experiments on a dataset with 170K Android apps from 2017 and 2018 show the practical feasibility of evading a state-of-the-art malware classifier along with its hardened version. Our results demonstrate that "adversarial-malware as a service" is a realistic threat, as we automatically generate thousands of realistic and inconspicuous adversarial applications at scale, where on average it takes only a few minutes to generate an adversarial app. Our formalization of problem-space attacks paves the way to more principled research in this domain.Comment: This arXiv version (v2) corresponds to the one published at IEEE Symposium on Security & Privacy (Oakland), 202