Human factors in X-ray image inspection of passenger Baggage – Basic and applied perspectives

The X-ray image inspection of passenger baggage contributes substantially to aviation security and is best understood as a search and decision task: Trained security officers – so called screeners – search the images for threats among many harmless everyday objects, but the recognition of objects in X-ray images and therefore the decision between threats and harmless objects can be difficult. Because performance in this task depends on often difficult recognition, it is not clear to what extent basic research on visual search can be generalized to X-ray image inspection. Manuscript 1 of this thesis investigated whether X-ray image inspection and a traditional visual search task depend on the same visual-cognitive abilities. The results indicate that traditional visual search tasks and X-ray image inspection depend on different aspects of common visual-cognitive abilities. Another gap between basic research on visual search and applied research on X-ray image inspection is that the former is typically conducted with students and the latter with professional screeners. Therefore, these two populations were compared, revealing that professionals performed better in X-ray image inspection, but not the visual search task. However, there was no difference between students and professionals regarding the importance of the visual-cognitive abilities for either task. Because there is some freedom in the decision whether a suspicious object should be declared as a threat or as harmless, the results of X-ray image inspection in terms of hit and false alarm rate depend on the screeners’ response tendency. Manuscript 2 evaluated whether three commonly used detection measures – ${d}'$, ${A}'$, and ${d}_{a}$ – are a valid representation of detection performance that is independent from response tendency. The results were consistently in favor of da with a slope parameter of around 0.6. In Manuscript 3 it was further shown that screeners can change their response tendency to increase the detection of novel threats. Also, screeners with a high ability to recognize everyday objects detected more novel threats when their response tendency was manipulated. The thesis further addressed changes that screeners face due to technological developments. Manuscript 4 showed that screeners can inspect X-ray images for one hour straight without a decrease in performance under conditions of remote cabin baggage screening, which means that X-ray image inspection is performed in a quiet room remote from the checkpoint. These screeners did not show a lower performance, but reported more distress, compared to screeners who took a 10 min break after every 20 min of screening. Manuscript 5 evaluated detection systems for cabin baggage screening (EDSCB). EDSCB only increased the detection of improvised explosive devices (IEDs) for inexperienced screeners if alarms by the EDSCB were indicated on the image and the screeners had to decide whether a threat was present or not. The detection of mere explosives, which lack the triggering device of IEDs, was only increased if the screeners could not decide against an alarm by the EDSCB. Manuscript 6 used discrete event simulation to evaluate how EDSCB impacts the throughput of passenger baggage screening. Throughput decreased with increasing false alarm rate of the EDSCB. However, fast alarm resolution processes and screeners with a low false alarm rate increased throughput. Taken together, the present findings contribute to understanding X-ray image inspection as a task with a search and decision component. The findings provide insights into basic aspects like the required visual-cognitive abilities and valid measures of detection performance, but also into applied research questions like for how long X-ray image inspection can be performed and how automation can assist with the detection of explosives

Expertise, Automation and Trust in X-Ray Screening of Cabin Baggage

X-ray screening of passenger baggage is a key component in aviation security. The current study investigated how experts and novices performed in an X-ray baggage screening task while being assisted by an adaptable diagnostic aid. Furthermore, it examined how both groups operated and trusted this automated system. 30 experts (certified screeners) and 31 novices (students) had to indicate whether a target item (either a knife or a gun) was present in a series of X-ray images of cabin baggage. Half of the participants could choose between three different support levels of the diagnostic aid (DA): (1) no support, (2) a cue indicating the presence of a potential target without locating it, or (3) a cue indicating the presence of a potential target by surrounding it with a red frame. As expected, experts achieved higher detection performance (d’), were more self-confident and felt more competent in achieving the task than novices. Furthermore, experts experienced less time pressure and fatigue. Although both groups used the DA in a comparable way (in terms of support level used and frequency of level switches), results showed a performance increase for novices working with the DA compared to novices without support. This benefit of DA was not observed for experts. Interestingly, despite no difference in perceived trust ratings, experts were more compliant (i.e., following DA recommendations when it indicated the presence of a target) and reliant (i.e., following DA recommendations when it indicated the absence of a target) than novices. Altogether, the results of the present study suggested that novices benefited more from a DA than experts. Furthermore, compliance and reliance on DA seemed to depend on expertise with the task. Since experts should be better at assessing the reliability of the DA than novices, they may have used the DA as ‘back-up’ to confirm their decisions based on expertise (confirmatory function), while novices may have used it as a guide to base their decisions on (support function). Finally, trust towards a DA was associated with the degree to which participants found the DA useful

Contribution to the evaluation and optimization of passengers' screening at airports

Security threats have emerged in the past decades as a more and more critical issue for Air Transportation which has been one of the main ressource for globalization of economy. Reinforced control measures based on pluridisciplinary research and new technologies have been implemented at airports as a reaction to different terrorist attacks. From the scientific perspective, the efficient screening of passengers at airports remain a challenge and the main objective of this thesis is to open new lines of research in this field by developing advanced approaches using the resources of Computer Science. First this thesis introduces the main concepts and definitions of airport security and gives an overview of the passenger terminal control systems and more specifically the screening inspection positions are identified and described. A logical model of the departure control system for passengers at an airport is proposed. This model is transcribed into a graphical view (Controlled Satisfiability Graph-CSG) which allows to test the screening system with different attack scenarios. Then a probabilistic approach for the evaluation of the control system of passenger flows at departure is developped leading to the introduction of Bayesian Colored Petri nets (BCPN). Finally an optimization approach is adopted to organize the flow of passengers at departure as best as possible given the probabilistic performance of the elements composing the control system. After the establishment of a global evaluation model based on an undifferentiated serial processing of passengers, is analyzed a two-stage control structure which highlights the interest of pre-filtering and organizing the passengers into separate groups. The conclusion of this study points out for the continuation of this theme

A Risk-Based Optimization Framework for Security Systems Upgrades at Airports

Airports are fast-growing dynamic infrastructure assets. For example, the Canadian airport industry is growing by 5% annually and generates about $8 billion yearly. Since the 9/11 tragedy, airport security has been of paramount importance both in Canada and worldwide. Consequently, in 2002, in the wake of the attacks, the International Civil Aviation Organization (ICAO) put into force revised aviation security standards and recommended practices, and began a Universal Security Audit Program (USAP), in order to insure the worldwide safeguarding of civil aviation in general, and of airports in particular, against unlawful interference. To improve aviation security at both the national level and for individual airport, airport authorities in North America have initiated extensive programs to help quantify, detect, deter, and mitigate security risk. At the research level, a number of studies have examined scenarios involving threats to airports, the factors that contribute to airport vulnerability, and decision support systems for security management. However, more work is still required in the area of developing decision support tools that can assist airport officials in meeting the challenges associated with decision about upgrades; determining the status of their security systems and efficiently allocating financial resources to improve them to the level required. To help airport authorities make cost-effective decisions about airport security upgrades, this research has developed a risk-based optimization framework. The framework assists airport officials in quantitatively assessing the status of threats to their airports, the vulnerability to their security systems, and the consequences of security breaches. A key element of this framework is a new quantitative security metric ; the aim of which is to assist airport authorities self-assess the condition of their security systems, and to produce security risk indices that decision makers can use as prioritizing criteria and constraints when meeting decisions about security upgrades. These indices have been utilized to formulate an automated decision support system for upgrading security systems in airports. Because they represent one of the most important security systems in an airport, the research focuses on passenger and cabin baggage screening systems. Based on an analysis of the related threats, vulnerabilities and consequences throughout the flow of passengers, cabin baggage, and checked-in luggage, the proposed framework incorporates an optimization model for determining the most cost-effective countermeasures that can minimize security risks. For this purpose, the framework first calculates the level of possible improvement in security using a new risk metric. Among the important features of the framework is the fact that it allows airport officials to perform multiple “what-if” scenarios, to consider the limitations of security upgrade budgets, and to incorporate airport-specific requirements. Based on the received positive feedback from two actual airports, the framework can be extended to include other facets of security in airports, and to form a comprehensive asset management system for upgrading security at both single and multiple airports. From a broader perspective, this research contributes to the improvement of security in a major transportation sector that has an enormous impact on economic growth and on the welfare of regional, national and international societies

A Step toward Ending Long Airport Security Lines: The Modified Boarding Pass

Anyone who has traveled by air has most likely experienced long airport security lines. Yet not much is known about its cause because few have considered if passengers have created this problem for themselves. The present study attempts to fill this research gap by suggesting that when passengers are not well-prepared for security screening, they delay the process by making mistakes and not complying with procedures. This lack of preparedness can be attributed to several shortcomings of security signposts. This study proposes the use of a modified boarding pass as an alternative form of signage to help passengers better prepare for security screening. In a recall evaluation of the items to remove prior to security screening, the combination of the modified boarding pass and security signposts led to greater recall than when either stimuli were used alone. In an airport survey to gather public sentiment, three-quarters of the respondents saw value in the idea of the modified boarding pass. Although the majority of the respondents were receptive to it becoming an option for future travel, many also felt that the modified boarding pass would be more useful than security signposts or announcements at conveying helpful security screening information

Inference of resource-based simulation models from process event-log data

This research was focused on inferring resource-based simulation models from data. and has proven it is realistic to do so. The research has discovered a new Process Mining algorithm with superior performance and has developed methods to identify, quantify and discover resource attributes and resource-based decisions from data

Technology Against Terrorism: Structuring Security

This report is devoted primarily to three other topics: interagency coordination of efforts in counterterrorist research and development, integrated security systems, and the role of human factors in aviation security. In addition, it furnishes details on a number of technologies that play a role in counterterrorism

Towards Real-Time Anomaly Detection within X-ray Security Imagery: Self-Supervised Adversarial Training Approach

Automatic threat detection is an increasingly important area in X-ray security imaging since it is critical to aid screening operators to identify concealed threats. Due to the cluttered and occluded nature of X-ray baggage imagery and limited dataset availability, few studies in the literature have systematically evaluated the automated X-ray security screening. This thesis provides an exhaustive evaluation of the use of deep Convolutional Neural Networks (CNN) for the image classification and detection problems posed within the field. The use of transfer learning overcomes the limited availability of the object of interest data examples. A thorough evaluation reveals the superiority of the CNN features over conventional hand-crafted features. Further experimentation also demonstrates the capability of the supervised deep object detection techniques as object localization strategies within cluttered X-ray security imagery. By addressing the limitations of the current X-ray datasets such as annotation and class-imbalance, the thesis subsequently transitions the scope to- wards deep unsupervised techniques for the detection of anomalies based on the training on normal (benign) X-ray samples only. The proposed anomaly detection models within the thesis employ a conditional encoder-decoder generative adversarial network that jointly learns the generation of high-dimensional image space and the inference of latent space — minimizing the distance between these images and the latent vectors during training aids in learning the data distribution for the normal samples. As a result, a larger distance metric from this learned data distribution at inference time is indicative of an outlier from that distribution — an anomaly. Experimentation over several benchmark datasets, from varying domains, shows the model efficacy and superiority over previous state-of-the-art approaches. Based on the current approaches and open problems in deep learning, the thesis finally provides discussion and future directions for X-ray security imagery

Industry 4.0 : challenges and success factors for adopting digital technologies in airports

With the advent of Industry 4.0 technologies in the last decade, airports have undergone digitalisation to capitalise on the purported benefits of these technologies such as improved operational efficiency and passenger experience. The ongoing COVID-19 pandemic with emergence of its variants (e.g. Delta, Omicron) has exacerbated the need for airports to adopt new technologies such as contactless and robotic technologies to facilitate travel during this pandemic. However, there is limited knowledge of recent challenges and success factors for adoption of digital technologies in airports. Therefore, through an industry survey of airport operators and managers around the world (n=102, 0.75

Neurophysiological correlates of psychological attitudes of air traffic controllers during their work

The research proposed in this thesis is part of a European project called NINA (Neurometrics Indicators for Air Traffic Management) funded by Sesar Joint Undertaking, and it involves the participation of Sapienza University of Rome, École Nationale de l’Aviation Civile (ENAC), and Deep Blue srl (Human Factor and Safety Consultant Company). The main goal of the project is to elaborate neurophysiological measurements for real-time assessment and monitoring of the cognitive state in particular professional categories, such as Air Traffic Controllers (ATCOs). The evaluation is performed by using a combination of techniques such as Electroencephalography (EEG), Electrocardiography (EKG) and Electrooculography (EOG), during simulated and realistic working conditions. In the area of ATCOs, the Skill, Rule and Knowledge (S-R-K) taxonomy was developed by Rasmussen to describe the human performance under various circumstances and to integrate a variety of research results coming from human cognition studies (attention, memory, problem solving, decision-making, etc.) under a common framework. It provides a description of human cognition that is functional to the understanding and prediction of behaviour: it specifically deals with how people control their activity and behave in interaction with complex systems. Therefore, by considering the aspect of the cognitive processes in the framework of such taxonomy, it is possible to contextualise them in the work practices. Since to our knowledge there are no corresponding studies in the existing literature, another challenging objective of the project is to develop the SRK concept from a neurophysiological point of view. The focus of the proposed thesis is thus to verify the existence of identifiable neurophysiological features associated to the three levels of cognitive control of behaviour (Skill, Rule and Knowledge), in Air Traffic Management (ATM) context, by using a neurometric able to identify the behaviours of the original taxonomy from a different perspective. To map the neurophysiology of the SRK framework in ATM domain, and to use this methodology, could represent a promising step forward into the analysis of human behaviour, and furthermore, to develop new Human Factors tools able to discriminate the level of operators’ expertise during ecological tasks. In detail, the first part of this work illustrates a brief description of the brain and the Electroencephalographic technique, then an introduction of the NINA project and the literature related to the S-R-K levels of cognitive control are presented. The second section is focused on some additional brain features’ literature and on the experimental phase where several steps were performed as follows: a) the three categories of behaviours were associated with specific cognitive functions (e.g. attention, memory, decision making etc.) already investigated in literature with EEG measurements; b) a link between S-R-K behaviours and expected EEG frequency bands configurations were hypothesized; c) specific events were designed to trigger S, R and K behaviours and integrated into realistic ATM simulations; d) finally, the machine-learning algorithm automatic stop StepWise Linear Discriminant Analysis (asSWLDA) was trained to differentiate the three levels of cognitive control of behaviour by using brain features extracted from the EEG rhythms of different brain areas. Several professional ATCOs from the École Nationale de l’Aviation Civile (ENAC) of Toulouse (France) were involved in the study and the results showed that the classification algorithm was able to discriminate with high reliability the three levels of cognitive control of behaviour during simulated air-traffic scenarios in an ecological ATM environment