ProtoExplorer: Interpretable Forensic Analysis of Deepfake Videos using Prototype Exploration and Refinement

In high-stakes settings, Machine Learning models that can provide predictions that are interpretable for humans are crucial. This is even more true with the advent of complex deep learning based models with a huge number of tunable parameters. Recently, prototype-based methods have emerged as a promising approach to make deep learning interpretable. We particularly focus on the analysis of deepfake videos in a forensics context. Although prototype-based methods have been introduced for the detection of deepfake videos, their use in real-world scenarios still presents major challenges, in that prototypes tend to be overly similar and interpretability varies between prototypes. This paper proposes a Visual Analytics process model for prototype learning, and, based on this, presents ProtoExplorer, a Visual Analytics system for the exploration and refinement of prototype-based deepfake detection models. ProtoExplorer offers tools for visualizing and temporally filtering prototype-based predictions when working with video data. It disentangles the complexity of working with spatio-temporal prototypes, facilitating their visualization. It further enables the refinement of models by interactively deleting and replacing prototypes with the aim to achieve more interpretable and less biased predictions while preserving detection accuracy. The system was designed with forensic experts and evaluated in a number of rounds based on both open-ended think aloud evaluation and interviews. These sessions have confirmed the strength of our prototype based exploration of deepfake videos while they provided the feedback needed to continuously improve the system.Comment: 15 pages, 6 figure

JPEG steganography with particle swarm optimization accelerated by AVX

Digital steganography aims at hiding secret messages in digital data transmitted over insecure channels. The JPEG format is prevalent in digital communication, and images are often used as cover objects in digital steganography. Optimization methods can improve the properties of images with embedded secret but introduce additional computational complexity to their processing. AVX instructions available in modern CPUs are, in this work, used to accelerate data parallel operations that are part of image steganography with advanced optimizations.Web of Science328art. no. e544

Performance modelling and optimization for video-analytic algorithms in a cloud-like environment using machine learning

CCTV cameras produce a large amount of video surveillance data per day, and analysing them require the use of significant computing resources that often need to be scalable. The emergence of the Hadoop distributed processing framework has had a significant impact on various data intensive applications as the distributed computed based processing enables an increase of the processing capability of applications it serves. Hadoop is an open source implementation of the MapReduce programming model. It automates the operation of creating tasks for each function, distribute data, parallelize executions and handles machine failures that reliefs users from the complexity of having to manage the underlying processing and only focus on building their application. It is noted that in a practical deployment the challenge of Hadoop based architecture is that it requires several scalable machines for effective processing, which in turn adds hardware investment cost to the infrastructure. Although using a cloud infrastructure offers scalable and elastic utilization of resources where users can scale up or scale down the number of Virtual Machines (VM) upon requirements, a user such as a CCTV system operator intending to use a public cloud would aspire to know what cloud resources (i.e. number of VMs) need to be deployed so that the processing can be done in the fastest (or within a known time constraint) and the most cost effective manner. Often such resources will also have to satisfy practical, procedural and legal requirements. The capability to model a distributed processing architecture where the resource requirements can be effectively and optimally predicted will thus be a useful tool, if available. In literature there is no clear and comprehensive modelling framework that provides proactive resource allocation mechanisms to satisfy a user's target requirements, especially for a processing intensive application such as video analytic. In this thesis, with the hope of closing the above research gap, novel research is first initiated by understanding the current legal practices and requirements of implementing video surveillance system within a distributed processing and data storage environment, since the legal validity of data gathered or processed within such a system is vital for a distributed system's applicability in such domains. Subsequently the thesis presents a comprehensive framework for the performance ii modelling and optimization of resource allocation in deploying a scalable distributed video analytic application in a Hadoop based framework, running on virtualized cluster of machines. The proposed modelling framework investigates the use of several machine learning algorithms such as, decision trees (M5P, RepTree), Linear Regression, Multi Layer Perceptron(MLP) and the Ensemble Classifier Bagging model, to model and predict the execution time of video analytic jobs, based on infrastructure level as well as job level parameters. Further in order to propose a novel framework for the allocate resources under constraints to obtain optimal performance in terms of job execution time, we propose a Genetic Algorithms (GAs) based optimization technique. Experimental results are provided to demonstrate the proposed framework's capability to successfully predict the job execution time of a given video analytic task based on infrastructure and input data related parameters and its ability determine the minimum job execution time, given constraints of these parameters. Given the above, the thesis contributes to the state-of-art in distributed video analytics, design, implementation, performance analysis and optimisation

Aikajanojen analysointiohjelmiston toteutus tietoturvapoikkeamien tutkintaan

Organizations today are trying to manage the many risks they percieve to be threatening the security of their valuable information assets, but often these risks realize into security incidents. Managing risks proactively is important, but equally important and challenging is to efficiently respond to the incidents that have already occurred, to minimize their impact on business processes. A part of managing security incidents is the technical analysis of any related computer systems, also known as digital forensic investigations. As a result of collecting evidence such as log files from these systems, the analysts end up with large amounts of data, which can form a timeline of events. These events describe different actions performed on the system in question. Analysing the timelines to find any events of interest is challenging due to the vast amount of data available on modern systems. The goal of this thesis is to create a software program to support the analysis of very large timelines as a part of digital forensic investigations. As a result, we have implemented software with an efficient query interface, which supports iterative exploration of the data and more complex analytical queries. Furthermore, we use a timeline visualization to compactly represent different properties of the data, which enables analysts to detect potential anomalies in an efficient way. This software also serves as a platform for future work, to experiment with more automated analysis techniques. We evaluated the software in a case study, in which it proved to show a great level of flexibility and performance compared to more traditional ways of working.Tärkeä osa nykypäivän organisaatioiden riskienhallintaa on tietopääoman turvaamiseen liittyvien riskien tunnistaminen. Näitä riskejä ei kuitenkaan usein oteta tarpeeksi vakavasti, sillä monesti ne myös realisoituvat tietoturvapoikkeamina. Kattava etukäteisvalmistautuminen on tärkeää, mutta poikkeamien vaikutusten minimoimisen kannalta oleellista on myös valmius tehokkaaseen poikkeamatilanteiden hallintaan. Osana tietoturvapoikkeamien hallintaa toteutetaan siihen liittyvien järjestelmien tekninen analyysi. Todistusaineiston, kuten erilaisten lokitiedostojen, keruun tuloksena tutkijat muodostavat aikajanan järjestelmässä suoritetuista toiminnoista. Koska modernien järjestelmien sisältämä tiedon määrä on poikkeuksetta suuri, on aikajanan analysointi mielenkiintoisten jälkien löytämiseksi erityisen haastavaa. Tämän diplomityön tavoitteena onkin luoda ohjelmisto tukemaan kooltaan erityisen suurten aikajanojen analysointia. Työn tuloksena luotiin ohjelmisto, joka tarjoaa tehokkaan kyselyrajapinnan, tukee tutkimukselle tyypillistä iteratiivista tiedon etsintää ja monimutkaisempia analyyttisia kyselyitä. Lisaksi ohjelmisto mahdollistaa monipuolisen aikajanan visualisoimisen, mikä helpottaa huomattavasti käytöspoikkeamien löytämistä. Tavoitteena oli myös tuottaa alusta, jota voidaan käyttää jatkossa uusien automaattisten analyysitekniikoiden kehittämisessä. Ohjelmiston toimivuus todennettiin tapaustutkimuksessa, joka osoitti ohjelmiston olevan erityisen joustava ja suorituskykyinen verrattuna aikaisempiin toimintatapoihin

Information Pooling Bias in Collaborative Cyber Forensics

abstract: Cyber threats are growing in number and sophistication making it important to continually study and improve all dimensions of cyber defense. Human teamwork in cyber defense analysis has been overlooked even though it has been identified as an important predictor of cyber defense performance. Also, to detect advanced forms of threats effective information sharing and collaboration between the cyber defense analysts becomes imperative. Therefore, through this dissertation work, I took a cognitive engineering approach to investigate and improve cyber defense teamwork. The approach involved investigating a plausible team-level bias called the information pooling bias in cyber defense analyst teams conducting the detection task that is part of forensics analysis through human-in-the-loop experimentation. The approach also involved developing agent-based models based on the experimental results to explore the cognitive underpinnings of this bias in human analysts. A prototype collaborative visualization tool was developed by considering the plausible cognitive limitations contributing to the bias to investigate whether a cognitive engineering-driven visualization tool can help mitigate the bias in comparison to off-the-shelf tools. It was found that participant teams conducting the collaborative detection tasks as part of forensics analysis, experience the information pooling bias affecting their performance. Results indicate that cognitive friendly visualizations can help mitigate the effect of this bias in cyber defense analysts. Agent-based modeling produced insights on internal cognitive processes that might be contributing to this bias which could be leveraged in building future visualizations. This work has multiple implications including the development of new knowledge about the science of cyber defense teamwork, a demonstration of the advantage of developing tools using a cognitive engineering approach, a demonstration of the advantage of using a hybrid cognitive engineering methodology to study teams in general and finally, a demonstration of the effect of effective teamwork on cyber defense performance.Dissertation/ThesisDoctoral Dissertation Applied Psychology 201

Anomalous behaviour detection using heterogeneous data

Anomaly detection is one of the most important methods to process and find abnormal data, as this method can distinguish between normal and abnormal behaviour. Anomaly detection has been applied in many areas such as the medical sector, fraud detection in finance, fault detection in machines, intrusion detection in networks, surveillance systems for security, as well as forensic investigations. Abnormal behaviour can give information or answer questions when an investigator is performing an investigation. Anomaly detection is one way to simplify big data by focusing on data that have been grouped or clustered by the anomaly detection method. Forensic data usually consists of heterogeneous data which have several data forms or types such as qualitative or quantitative, structured or unstructured, and primary or secondary. For example, when a crime takes place, the evidence can be in the form of various types of data. The combination of all the data types can produce rich information insights. Nowadays, data has become ‘big’ because it is generated every second of every day and processing has become time-consuming and tedious. Therefore, in this study, a new method to detect abnormal behaviour is proposed using heterogeneous data and combining the data using data fusion technique. Vast challenge data and image data are applied to demonstrate the heterogeneous data. The first contribution in this study is applying the heterogeneous data to detect an anomaly. The recently introduced anomaly detection technique which is known as Empirical Data Analytics (EDA) is applied to detect the abnormal behaviour based on the data sets. Standardised eccentricity (a newly introduced within EDA measure offering a new simplified form of the well-known Chebyshev Inequality) can be applied to any data distribution. Then, the second contribution is applying image data. The image data is processed using pre-trained deep learning network, and classification is done using a support vector machine (SVM). After that, the last contribution is combining anomaly result from heterogeneous data and image recognition using new data fusion technique. There are five types of data with three different modalities and different dimensionalities. The data cannot be simply combined and integrated. Therefore, the new data fusion technique first analyses the abnormality in each data type separately and determines the degree of suspicious between 0 and 1 and sums up all the degrees of suspicion data afterwards. This method is not intended to be a fully automatic system that resolves investigations, which would likely be unacceptable in any case. The aim is rather to simplify the role of the humans so that they can focus on a small number of cases to be looked in more detail. The proposed approach does simplify the processing of such huge amounts of data. Later, this method can assist human experts in their investigations and making final decisions

Interactive visualization of event logs for cybersecurity

Hidden cyber threats revealed with new visualization software Eventpa

Robotic equipment carrying RN detectors: requirements and capabilities for testing

77 pags., 32 figs., 5 tabs.-- ERNCIP Radiological and Nuclear Threats to Critical Infrastructure Thematic Group . -- This publication is a Technical report by the Joint Research Centre (JRC) . -- JRC128728 . -- EUR 31044 ENThe research leading to these results has received funding from the European Union as part of the European Reference Network for Critical Infrastructure Protection (ERNCIP) projec