ProtoExplorer: Interpretable Forensic Analysis of Deepfake Videos using Prototype Exploration and Refinement

In high-stakes settings, Machine Learning models that can provide predictions that are interpretable for humans are crucial. This is even more true with the advent of complex deep learning based models with a huge number of tunable parameters. Recently, prototype-based methods have emerged as a promising approach to make deep learning interpretable. We particularly focus on the analysis of deepfake videos in a forensics context. Although prototype-based methods have been introduced for the detection of deepfake videos, their use in real-world scenarios still presents major challenges, in that prototypes tend to be overly similar and interpretability varies between prototypes. This paper proposes a Visual Analytics process model for prototype learning, and, based on this, presents ProtoExplorer, a Visual Analytics system for the exploration and refinement of prototype-based deepfake detection models. ProtoExplorer offers tools for visualizing and temporally filtering prototype-based predictions when working with video data. It disentangles the complexity of working with spatio-temporal prototypes, facilitating their visualization. It further enables the refinement of models by interactively deleting and replacing prototypes with the aim to achieve more interpretable and less biased predictions while preserving detection accuracy. The system was designed with forensic experts and evaluated in a number of rounds based on both open-ended think aloud evaluation and interviews. These sessions have confirmed the strength of our prototype based exploration of deepfake videos while they provided the feedback needed to continuously improve the system

ProtoExplorer: Interpretable Forensic Analysis of Deepfake Videos using Prototype Exploration and Refinement

In high-stakes settings, Machine Learning models that can provide predictions that are interpretable for humans are crucial. This is even more true with the advent of complex deep learning based models with a huge number of tunable parameters. Recently, prototype-based methods have emerged as a promising approach to make deep learning interpretable. We particularly focus on the analysis of deepfake videos in a forensics context. Although prototype-based methods have been introduced for the detection of deepfake videos, their use in real-world scenarios still presents major challenges, in that prototypes tend to be overly similar and interpretability varies between prototypes. This paper proposes a Visual Analytics process model for prototype learning, and, based on this, presents ProtoExplorer, a Visual Analytics system for the exploration and refinement of prototype-based deepfake detection models. ProtoExplorer offers tools for visualizing and temporally filtering prototype-based predictions when working with video data. It disentangles the complexity of working with spatio-temporal prototypes, facilitating their visualization. It further enables the refinement of models by interactively deleting and replacing prototypes with the aim to achieve more interpretable and less biased predictions while preserving detection accuracy. The system was designed with forensic experts and evaluated in a number of rounds based on both open-ended think aloud evaluation and interviews. These sessions have confirmed the strength of our prototype based exploration of deepfake videos while they provided the feedback needed to continuously improve the system.Comment: 15 pages, 6 figure

ProtoExplorer: Interpretable forensic analysis of deepfake videos using prototype exploration and refinement

In high-stakes settings, Machine Learning models that can provide predictions that are interpretable for humans are crucial. This is even more true with the advent of complex deep learning based models with a huge number of tunable parameters. Recently, prototype-based methods have emerged as a promising approach to make deep learning interpretable. We particularly focus on the analysis of deepfake videos in a forensics context. Although prototype-based methods have been introduced for the detection of deepfake videos, their use in real-world scenarios still presents major challenges, in that prototypes tend to be overly similar and interpretability varies between prototypes. This paper proposes a Visual Analytics process model for prototype learning, and, based on this, presents ProtoExplorer, a Visual Analytics system for the exploration and refinement of prototype-based deepfake detection models. ProtoExplorer offers tools for visualizing and temporally filtering prototype-based predictions when working with video data. It disentangles the complexity of working with spatio-temporal prototypes, facilitating their visualization. It further enables the refinement of models by interactively deleting and replacing prototypes with the aim to achieve more interpretable and less biased predictions while preserving detection accuracy. The system was designed with forensic experts and evaluated in a number of rounds based on both open-ended think aloud evaluation and interviews. These sessions have confirmed the strength of our prototype-based exploration of deepfake videos while they provided the feedback needed to continuously improve the system

Kinetic model for Pd-based membranes coking/deactivation in propane dehydrogenation processes

This work aims at providing insight into the deactivation mechanism of Pd-based membranes in propane dehydrogenation processes. Thermogravimetric analysis (TGA) experiments were conducted to study the adsorption and coking of propylene over conventional thin layer (TL) and double-skinned (DS) Pd-based membranes under several operating conditions. A mechanistic monolayer-multilayer coke growth model was selected to mathematically describe the membrane coking observed during TGA experiments. In addition, the reaction rate of coke formation and its influence on membranes deactivation has been studied. The deactivation model able to describe the hydrogen flux decay over time suggests that monolayer coke is the main responsible for the membrane deactivation. Multilayer coke also causes deactivation but with a smaller order than monolayer coke, for both the TL and the DS membranes. Among the two membrane types, DS membrane deactivates faster, i.e. with a higher order than the TL membrane, which is equal to 1.55 for the former and 0.51 for the latter. This is related to the higher number of active sites available in the controlling step of the deactivation reaction, which are most probably given by the addition of the ceramic Al2O3 protective layer. XPS spectra further confirms that, in the presence of Pd, Al2O3 sites contribute to carbon formation by evidencing a different nature of carbon formed on the two membranes. Finally, the experimental results of hydrogen permeation over time conducted on different membranes types and operative conditions confirmed the validity of the derived and parametrized kinetic models for coke formation and membrane deactivation. The experimental findings and the kinetic model derived in this work provide essential tools for the design and optimization of membrane reactors for dehydrogenation processes.This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 814671 (BiZeolCat)

Effect of iron addition to the electrolyte on alkaline water electrolysis performance

Improvement of alkaline water electrolysis is a key enabler for quickly scaling up green hydrogen production. Fe is omnipresent within most industrial alkaline water electrolyzers and its effect on electrolyzer performance needs to be assessed. We conducted three-electrode and flow cell experiments with electrolyte Fe and Ni electrodes. Three-electrode cell experiments show that Fe ([Fe] = 6–357 μM; ICP-OES) promotes HER and OER by lowering both overpotentials by at least 100 mV at high current densities (T = 35°C–91°C). The overpotential of a zero-gap flow cell was decreased by 200 mV when increasing the Fe concentration ([Fe] = 13–549 μM, T = 21°C–75°C). HER benefits from the formation of Fe dendrite layers (SEM/EDX, XPS), which prevent NiHx formation and increase the overall active area. The OER benefits from the formation of mixed Ni/Fe oxyhydroxides leading to better catalytic activity and Tafel slope reduction

Image reconstruction in low-field MRI: A super-resolution approach

The quality of magnetic resonance images produced by conventional MRI scanners is guaranteed by the strength and homogeneity of the magnetic field. However, the superconducting magnets required to produce such a field make MRI scanners large and expensive and hence inaccessible to a large number of people in developing countries. Our partners are developing low-cost, portable MRI scanners that do not depend on superconducting magnets. In these scanners, the signal-to-noise ratio will be significantly lower due to the lower magnetic field strength. Additionally, inhomogeneities will be present, which means that the traditional way of obtaining the image, by inverse Fourier Transform, is no longer feasible. In this research, image reconstruction is done using an ill-posed system of equations of the form Ax = y, where A is the reconstruction matrix and x and y are vectors containing the image pixel values and the measured signals, respectively. Three different regularization techniques are considered, with total variation yielding the best results. Two methods for solving the regularized least-squares problem are considered: CGLS and CGNE. For the types of problems we are dealing with, CGNE is outperformed by CGLS: CGLS requires a lower number of iterations to converge and the computational cost per iteration is lower. The main focus of this research is on super-resolution: reconstructing a high resolution image from one or several low resolution images. Due to the low signal-to-noise ratios that are expected in the low-field MRI prototypes, it might be better to reconstruct images of a low resolution, and using these, create high resolution images, instead of opting for a direct high resolution reconstruction. In order to test this, the signal generation in a Halbach array based MRI scanner is simulated. Our simulations show that for very low (<1.5-2) signal-to-noise ratios, super-resolution can yield better results than direct high resolution reconstruction. Data obtained in a 7 T MRI scanner is used to validate our reconstruction model. Due to the type of gradient used and the low number of measurements in this experiment, the amount of available information is very limited. This makes it challenging to produce an image of good quality. However, in our final image, out of the four water bottles in the phantom, the three largest ones are clearly visible.Electrical Engineering, Mathematics and Computer ScienceDelft Institute of Applied Mathematic

Kristallisatie door eenzijdige warmteafvoer

Applied Science

Image Reconstruction for Low-Field MRI

Each year, hundreds of thousands of infants develop hydrocephalus ("water on the brain"). This is a disease that, if untreated, leads to brain damage and ultimately death. The prevalence of hydrocephalus is relatively high in children living in the Global South (in sub-Saharan countries, for example), but access to advanced imaging technology is usually limited in countries belonging to the Global South. This is especially problematic for hydrocephalus, since magnetic resonance imaging often is the diagnostic tool of choice for this disease, but MRI scanners are essentially out of reach due to their cost, size, and stringent infrastructure demands. Therefore, the introduction of an inexpensive, portable, low-field MRI scanner is clinically relevant. An interdisciplinary team of researchers at the Leiden University Medical Center, Pennsylvania State University, Mbarara University of Science and Technology and Delft University of Technology has been working on the development of such low-fieldMRI scanners, with the first goal being to aid in the diagnosis of hydrocephalus in infants in sub-Saharan Africa. Within this project, several prototypes and various dedicated image reconstruction techniques have been developed. This dissertation focuses on the latter. High-field MRI scanners have very strong and homogeneous static magnetic background fields, due to the superconducting magnets they are equipped with. To significantly reduce production costs, the low-field scanners considered in this work use permanent magnets to realize their static background fields. Obviously, such background fields are much weaker than in a high-field MRI scanner, leading to measured signals with a significantly lower signal-to-noise ratio, since this ratio scales with the magnitude of the background field. For spatial encoding (i.e., to distinguish what part of the signal originates from what part of the body or object inside the scanner), high-field scanners depend on gradient coils which superimpose a linearly varying magnetic field on the background field. The first prototype we consider does not have any gradient coils. Instead, spatial encoding is carried out by making use of the inhomogeneities in the static magnetic background field. Due to the nonbijective nature of the field, a single measurement does not yield enough information for a reconstruction. However, by carrying out several measurements and rotating the field between subsequent measurements, image reconstruction should be possible. The second prototype follows the design of high-field scannersmore closely: it was designed such that the static magnetic field is as homogeneous as possible and the scanner is equipped with three gradient coils to allow for spatial encoding in three directions. In this case, the relationship between signal and image can be described by a Fourier Transform...Numerical Analysi

Spin swap and exchange coupling in a quantum dot array

Quantum TransportApplied Mathematics/Applied PhysicsApplied Science