An Examination of the Calibration and Resolution Skills in Phishing Email Detection

This study examines individuals’ calibration and resolution skills in phishing email detection and tests the effects of several factors on both skills. It shows that calibration and resolution are two distinct capabilities of a person to detect phishing emails, and they are subject to the impacts of different factors: while calibration is influenced mostly by task factors such as familiarity with the emails, time to judgment, variability of time to judgment, and task easiness, resolution is influenced by both task factors such as variability of time to judgment and familiarity with the entity in the email, and individual characteristics such as online transaction experience and prior victimization of phishing attacks. The theoretical implication of the study is addressed, and the practical implication for designing effective training programs to improve one’s phishing detection ability is also discussed

Better Beware: Comparing Metacognition for Phishing and Legitimate Emails

Every electronic message poses some threat of being a phishing attack. If recipients underestimate that threat, they expose themselves, and those connected to them, to identity theft, ransom, malware, or worse. If recipients overestimate that threat, then they incur needless costs, perhaps reducing their willingness and ability to respond over time. In two experiments, we examined the appropriateness of individuals\u27 confidence in their judgments of whether email messages were legitimate or phishing, using calibration and resolution as metacognition metrics. Both experiments found that participants had reasonable calibration but poor resolution, reflecting a weak correlation between their confidence and knowledge. These patterns differed for legitimate and phishing emails, with participants being better calibrated for legitimate emails, except when expressing complete confidence in their judgments, but consistently overconfident for phishing emails. The second experiment compared performance on the laboratory task with individuals\u27 actual vulnerability, and found that participants with better resolution were less likely to have malicious files on their home computers. That comparison raised general questions about the design of anti-phishing training and of providing feedback essential to self-regulated learning

Overconfidence in Phishing Email Detection

This study examines overconfidence in phishing email detection. Researchers believe that overconfidence (i.e., where one’s judgmental confidence exceeds one’s actual performance in decision making) can lead to one’s adopting risky behavior in uncertain situations. This study focuses on what leads to overconfidence in phishing detection. We performed a survey experiment with 600 subjects to collect empirical data for the study. In the experiment, each subject judged a set of randomly selected phishing emails and authentic business emails. Specifically, we examined two metrics of overconfidence (i.e., overprecision and overestimation). Results show that cognitive effort decreased overconfidence, while variability in attention allocation, dispositional optimism, and familiarity with the business entities in the emails all increased overconfidence in phishing email detection. The effect of perceived self-efficacy of detecting phishing emails on overconfidence was marginal. In addition, all confidence beliefs poorly predicted detection accuracy and poorly explained its variance, which highlights the issue of relying on them to guide one’s behavior in detecting phishing. We discuss mechanisms to reduce overconfidence

Designing Digital Forensics Challenges for Multinational Cyber Defense Exercises

Töös püütakse kujundada ja hinnata digitaalse kohtuekspertiisi väljakutset, mida kasutada rahvusvahelisel küberkaitse õppusel. Eesmärk on fokusseerida põhioskustele, mida üks riiklik organisatsioon oma digitaalse kohtuekspertiisi ekspertidelt vajab ja disainida ning integreerida tehnilisi ülesandeid, mis adekvaatselt testivad neid oskusi suuremal küberkaitse õppusel. See töö kasutab NATO Locked Shields küberkaitse õppust test-näitena, mille jaoks väitekirja autor liitus digitaalse kohtuekspertiisi disainimeeskonnaga, NATO Cyber Defense Centre of Excellence juures, kui nad kavandasid ja rakendasid kolme-päevast digitaalse kohtuekspertiisi väljakutset. See lõputöö kehtestab rea tehnilisi ja protseduurilisi oskuseid, mida riiklikud organisatsioonid vajavad oma ekspertidelt, määrab viisid, kuidas testida neid oskusi ja arendab stsenaariumipõhist digitaalse kohtuekspertiisi väljakutset. Kasutades vahetult saadud tähelepanekuid, osaleja tagasisidet ja väljakutse tulemusi, et hinnata väljakutse efektiivsust, lõputöös leitakse, et stsenaarium testis piisavalt enamus oskusi õigel raskustasemel ja vajab parendamist ajastuses ning aruandlusstandardites. Lõpetuseks uuritakse erinevaid viise, kuidas parendada valitud meetodeid ja ülesandeid tuleviku õppusteks.This thesis seeks to design and evaluate a digital forensics challenge for inclusion in a multinational cyber defense exercise. The intent is to narrow down the key skills a state-based organization requires of its digital forensics experts and design and integrate technical tasks that adequately test these skills into a larger cyber defense exercise. It uses the NATO Locked Shields cyber defense exercise as a test case, for which the thesis author joined the digital forensics design team at the NATO Cyber Defense Centre of Excellence in designing and implementing a three day digital forensics challenge. This thesis establishes a series of technical and procedural skills state-based organizations require of their experts, determines ways to test these skills, and develops a scenario-based digital forensics challenge. Using first hand observations, participant feedback, and challenge scores to evaluate the effectiveness of the challenge, it finds that the scenario adequately tested a majority of the skills at the appropriate difficulty level and needs improvement in timing and reporting standards. Finally, it explores ways to improve upon the selected methods and tasks for future exercises

Measuring Information Security Awareness Efforts in Social Networking Sites – A Proactive Approach

For Social Network Sites to determine the effectiveness of their Information Security Awareness (ISA) techniques, many measurement and evaluation techniques are now in place to ensure controls are working as intended. While these techniques are inexpensive, they are all incident- driven as they are based on the occurrence of incident(s). Additionally, they do not present a true reflection of ISA since cyber-incidents are hardly reported. They are therefore adjudged to be post-mortem and risk permissive, the limitations that are inacceptable in industries where incident tolerance level is low. This paper aims at employing a non-incident statistic approach to measure ISA efforts. Using an object- oriented programming approach, PhP is employed as the coding language with MySQL database engine at the back-end to develop sOcialistOnline – a Social Network Sites (SNS) fully secured with multiple ISA techniques. Rather than evaluating the effectiveness of ISA efforts by success of attacks or occurrence of an event, password scanning is implemented to proactively measure the effects of ISA techniques in sOcialistOnline. Thus, measurement of ISA efforts is shifted from detective and corrective to preventive and anticipatory paradigms which are the best forms of information security approach

Human Computer Interaction and Emerging Technologies

The INTERACT Conferences are an important platform for researchers and practitioners in the field of human-computer interaction (HCI) to showcase their work. They are organised biennially by the International Federation for Information Processing (IFIP) Technical Committee on Human–Computer Interaction (IFIP TC13), an international committee of 30 member national societies and nine Working Groups. INTERACT is truly international in its spirit and has attracted researchers from several countries and cultures. With an emphasis on inclusiveness, it works to lower the barriers that prevent people in developing countries from participating in conferences. As a multidisciplinary field, HCI requires interaction and discussion among diverse people with different interests and backgrounds. The 17th IFIP TC13 International Conference on Human-Computer Interaction (INTERACT 2019) took place during 2-6 September 2019 in Paphos, Cyprus. The conference was held at the Coral Beach Hotel Resort, and was co-sponsored by the Cyprus University of Technology and Tallinn University, in cooperation with ACM and ACM SIGCHI. This volume contains the Adjunct Proceedings to the 17th INTERACT Conference, comprising a series of selected papers from workshops, the Student Design Consortium and the Doctoral Consortium. The volume follows the INTERACT conference tradition of submitting adjunct papers after the main publication deadline, to be published by a University Press with a connection to the conference itself. In this case, both the Adjunct Proceedings Chair of the conference, Dr Usashi Chatterjee, and the lead Editor of this volume, Dr Fernando Loizides, work at Cardiff University which is the home of Cardiff University Press

An Ensemble Self-Structuring Neural Network Approach to Solving Classification Problems with Virtual Concept Drift and its Application to Phishing Websites

Classification in data mining is one of the well-known tasks that aim to construct a classification model from a labelled input data set. Most classification models are devoted to a static environment where the complete training data set is presented to the classification algorithm. This data set is assumed to cover all information needed to learn the pertinent concepts (rules and patterns) related to how to classify unseen examples to predefined classes. However, in dynamic (non-stationary) domains, the set of features (input data attributes) may change over time. For instance, some features that are considered significant at time Ti might become useless or irrelevant at time Ti+j. This situation results in a phenomena called Virtual Concept Drift. Yet, the set of features that are dropped at time Ti+j might return to become significant again in the future. Such a situation results in the so-called Cyclical Concept Drift, which is a direct result of the frequently called catastrophic forgetting dilemma. Catastrophic forgetting happens when the learning of new knowledge completely removes the previously learned knowledge. Phishing is a dynamic classification problem where a virtual concept drift might occur. Yet, the virtual concept drift that occurs in phishing might be guided by some malevolent intelligent agent rather than occurring naturally. One reason why phishers keep changing the features combination when creating phishing websites might be that they have the ability to interpret the anti-phishing tool and thus they pick a new set of features that can circumvent it. However, besides the generalisation capability, fault tolerance, and strong ability to learn, a Neural Network (NN) classification model is considered as a black box. Hence, if someone has the skills to hack into the NN based classification model, he might face difficulties to interpret and understand how the NN processes the input data in order to produce the final decision (assign class value). In this thesis, we investigate the problem of virtual concept drift by proposing a framework that can keep pace with the continuous changes in the input features. The proposed framework has been applied to phishing websites classification problem and it shows competitive results with respect to various evaluation measures (Harmonic Mean (F1-score), precision, accuracy, etc.) when compared to several other data mining techniques. The framework creates an ensemble of classifiers (group of classifiers) and it offers a balance between stability (maintaining previously learned knowledge) and plasticity (learning knowledge from the newly offered training data set). Hence, the framework can also handle the cyclical concept drift. The classifiers that constitute the ensemble are created using an improved Self-Structuring Neural Networks algorithm (SSNN). Traditionally, NN modelling techniques rely on trial and error, which is a tedious and time-consuming process. The SSNN simplifies structuring NN classifiers with minimum intervention from the user. The framework evaluates the ensemble whenever a new data set chunk is collected. If the overall accuracy of the combined results from the ensemble drops significantly, a new classifier is created using the SSNN and added to the ensemble. Overall, the experimental results show that the proposed framework affords a balance between stability and plasticity and can effectively handle the virtual concept drift when applied to phishing websites classification problem. Most of the chapters of this thesis have been subject to publicatio

Artificial Intelligence as Evidence

This article explores issues that govern the admissibility of Artificial Intelligence (“AI”) applications in civil and criminal cases, from the perspective of a federal trial judge and two computer scientists, one of whom also is an experienced attorney. It provides a detailed yet intelligible discussion of what AI is and how it works, a history of its development, and a description of the wide variety of functions that it is designed to accomplish, stressing that AI applications are ubiquitous, both in the private and public sectors. Applications today include: health care, education, employment-related decision-making, finance, law enforcement, and the legal profession. The article underscores the importance of determining the validity of an AI application (i.e., how accurately the AI measures, classifies, or predicts what it is designed to), as well as its reliability (i.e., the consistency with which the AI produces accurate results when applied to the same or substantially similar circumstances), in deciding whether it should be admitted into evidence in civil and criminal cases. The article further discusses factors that can affect the validity and reliability of AI evidence, including bias of various types, “function creep,” lack of transparency and explainability, and the sufficiency of the objective testing of AI applications before they are released for public use. The article next provides an in-depth discussion of the evidentiary principles that govern whether AI evidence should be admitted in court cases, a topic which, at present, is not the subject of comprehensive analysis in decisional law. The focus of this discussion is on providing a step-by-step analysis of the most important issues, and the factors that affect decisions on whether to admit AI evidence. Finally, the article concludes with a discussion of practical suggestions intended to assist lawyers and judges as they are called upon to introduce, object to, or decide on whether to admit AI evidence

Jack Voltaic 3.0 Cyber Research Report

The Jack Voltaic (JV) Cyber Research project is an innovative, bottom-up approach to critical infrastructure resilience that informs our understanding of existing cybersecurity capabilities and identifies gaps. JV 3.0 contributed to a repeatable framework cities and municipalities nationwide can use to prepare. This report on JV 3.0 provides findings and recommendations for the military, federal agencies, and policy makers

An exploration into the use of webinjects by financial malware

As the number of computing devices connected to the Internet increases and the Internet itself becomes more pervasive, so does the opportunity for criminals to use these devices in cybercrimes. Supporting the increase in cybercrime is the growth and maturity of the digital underground economy with strong links to its more visible and physical counterpart. The digital underground economy provides software and related services to equip the entrepreneurial cybercriminal with the appropriate skills and required tools. Financial malware, particularly the capability for injection of code into web browsers, has become one of the more profitable cybercrime tool sets due to its versatility and adaptability when targeting clients of institutions with an online presence, both in and outside of the financial industry. There are numerous families of financial malware available for use, with perhaps the most prevalent being Zeus and SpyEye. Criminals create (or purchase) and grow botnets of computing devices infected with financial malware that has been configured to attack clients of certain websites. In the research data set there are 483 configuration files containing approximately 40 000 webinjects that were captured from various financial malware botnets between October 2010 and June 2012. They were processed and analysed to determine the methods used by criminals to defraud either the user of the computing device, or the institution of which the user is a client. The configuration files contain the injection code that is executed in the web browser to create a surrogate interface, which is then used by the criminal to interact with the user and institution in order to commit fraud. Demographics on the captured data set are presented and case studies are documented based on the various methods used to defraud and bypass financial security controls across multiple industries. The case studies cover techniques used in social engineering, bypassing security controls and automated transfers