Media Forensics and DeepFakes: an overview

With the rapid progress of recent years, techniques that generate and manipulate multimedia content can now guarantee a very advanced level of realism. The boundary between real and synthetic media has become very thin. On the one hand, this opens the door to a series of exciting applications in different fields such as creative arts, advertising, film production, video games. On the other hand, it poses enormous security threats. Software packages freely available on the web allow any individual, without special skills, to create very realistic fake images and videos. So-called deepfakes can be used to manipulate public opinion during elections, commit fraud, discredit or blackmail people. Potential abuses are limited only by human imagination. Therefore, there is an urgent need for automated tools capable of detecting false multimedia content and avoiding the spread of dangerous false information. This review paper aims to present an analysis of the methods for visual media integrity verification, that is, the detection of manipulated images and videos. Special emphasis will be placed on the emerging phenomenon of deepfakes and, from the point of view of the forensic analyst, on modern data-driven forensic methods. The analysis will help to highlight the limits of current forensic tools, the most relevant issues, the upcoming challenges, and suggest future directions for research

Automatic architecture selection for hierarchical mixture of experts models

Hierarchical mixture of experts (HME) is a powerful tree-structured modeling technique based on the divide and conquer principle. HME model trees consist of two types of nodes - gate nodes, which are responsible for splitting a large complex problem into several smaller subproblems, and expert nodes, which perform the corresponding subproblemsolving. Selecting the number of such nodes as well as the order in which they are arranged is, however, a non-trivial task. A commonly used approach involves fitting several architectures and using methods such as cross-validation to pick the best one. As well as being computationally intensive, this method first requires one to pick the set of architectures to consider. For complex models with a large number of architectural elements, this leads to an unmanageable number of potential options. Pre-setting model architecture also requires choosing initial parameter values, which becomes progressively more challenging as parameter dimensionality increases. The latter challenges could be addressed by growing trees during the model fitting process instead of selecting the architecture in advance. It is thus evident that HME models suffer from a lack of a flexible and adaptive way of performing automatic architecture selection. The work presented in this thesis proposes automatic architecture selection methods for HME models, which allow for adding and removing tree nodes as well as adjusting the order in which they are arranged. As part of the development, three Bayesian parameter sampling strategies are proposed and systematically evaluated resulting in a recommended strategy. An adaptation of the Reversible Jump (RJ) algorithm is then used to grow and prune HME model trees. The main downfall of the RJ, which lies in low acceptance rates, is addressed by the addition of a novel reversible jump proposal algorithm. A new Gate Swaps (GS) algorithm is then proposed to tackle the problem of changing the order in which the existing tree nodes are arranged. Both algorithms are evaluated on two real-life problems with a particular focus on the Glasgow rental property prices data. It is shown that HME models fitted using the proposed RJ GS MCMC yield accurate predictions as well as provide an exceptionally high level of model interpretability, which is unusual amongst other machine learning methods

Cooperative Training in Multiple Classifier Systems

Multiple classifier system has shown to be an effective technique for classification. The success of multiple classifiers does not entirely depend on the base classifiers and/or the aggregation technique. Other parameters, such as training data, feature attributes, and correlation among the base classifiers may also contribute to the success of multiple classifiers. In addition, interaction of these parameters with each other may have an impact on multiple classifiers performance. In the present study, we intended to examine some of these interactions and investigate further the effects of these interactions on the performance of classifier ensembles. The proposed research introduces a different direction in the field of multiple classifiers systems. We attempt to understand and compare ensemble methods from the cooperation perspective. In this thesis, we narrowed down our focus on cooperation at training level. We first developed measures to estimate the degree and type of cooperation among training data partitions. These evaluation measures enabled us to evaluate the diversity and correlation among a set of disjoint and overlapped partitions. With the aid of properly selected measures and training information, we proposed two new data partitioning approaches: Cluster, De-cluster, and Selection (CDS) and Cooperative Cluster, De-cluster, and Selection (CO-CDS). In the end, a comprehensive comparative study was conducted where we compared our proposed training approaches with several other approaches in terms of robustness of their usage, resultant classification accuracy and classification stability. Experimental assessment of CDS and CO-CDS training approaches validates their robustness as compared to other training approaches. In addition, this study suggests that: 1) cooperation is generally beneficial and 2) classifier ensembles that cooperate through sharing information have higher generalization ability compared to the ones that do not share training information

Special Libraries, Winter 1986

Volume 77, Issue 1https://scholarworks.sjsu.edu/sla_sl_1986/1000/thumbnail.jp

Situating language and consciousness

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Linguistics and Philosophy, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 98-104).Language and consciousness enrich our lives. But they are rare commodities; most creatures are language-less and unconscious. This dissertation is about the conditions that distinguish the haves from the have-nots. The semantic properties of a natural language expression are determined by conventions governing the way speakers use the expression to communicate information. The capacity to speak a language involves highly specialized (perhaps even modular) cognition. Some authors think that one cannot consistently accept both views. In Chapter 1 ('Content and Competence') I explain why one can. According to the convention-based theory of content determination, propositions are fit to be the contents of both thought and speech. Recently, this view has been challenged. The challenge exploits a series of observations about what it takes to understand semantically incomplete sentences. In Chapter 2 ('Speaker Meaning in Context'), I explain how the challenge can be met. Physicalists seem to owe an explanatory debt. Why should psychophysical relations appear contingent? In Chapter 3 ('There Couldn't Have Been Zombies, but it's a Lucky Coincidence That There Aren't') I pay the debt on their behalf. My explanation proceeds in three steps. First, I observe that there are necessary coincidences, or accidents. Second, I show that traditional epistemological arguments for dualism merely establish that phenomenal states and corresponding physical states are accidentally, or coincidentally, related. Finally, I suggest that inattention to the distinction between coincidence/accidentality and contingency results in frequent equivocation. Thus the disposition to (correctly) judge that psychophysical relations are coincidences manifests itself as a disposition to (incorrectly) judge that psychophysical relations are contingent. In Chapter 4 ('Zombies are Inconceivable') I deny that psychophysical relations appear contingent. The chapter begins with an argument to the effect that zombies cannot be coherently conceived. I then consider and reject various ways of resisting the argument.by Mahrad Almotahari.Ph.D

Collaborative adaptive accessibility and human capabilities

This thesis discusses the challenges and opportunities facing the field of accessibility, particularly as computing becomes ubiquitous. It is argued that a new approach is needed that centres around adaptations (specific, atomic changes) to user interfaces and content in order to improve their accessibility for a wider range of people than targeted by present Assistive Technologies (ATs). Further, the approach must take into consideration the capabilities of people at the human level and facilitate collaboration, in planned and ad-hoc environments. There are two main areas of focus: (1) helping people experiencing minor-to-moderate, transient and potentially-overlapping impairments, as may be brought about by the ageing process and (2) supporting collaboration between people by reasoning about the consequences, from different users perspectives, of the adaptations they may require. A theoretical basis for describing these problems and a reasoning process for the semi-automatic application of adaptations is developed. Impairments caused by the environment in which a device is being used are considered. Adaptations are drawn from other research and industry artefacts. Mechanical testing is carried out on key areas of the reasoning process, demonstrating fitness for purpose. Several fundamental techniques to extend the reasoning process in order to take temporal factors (such as fluctuating user and device capabilities) into account are broadly described. These are proposed to be feasible, though inherently bring compromises (which are defined) in interaction stability and the needs of different actors (user, device, target level of accessibility). This technical work forms the basis of the contribution of one work-package of the Sustaining ICT use to promote autonomy (Sus-IT) project, under the New Dynamics of Ageing (NDA) programme of research in the UK. Test designs for larger-scale assessment of the system with real-world participants are given. The wider Sus-IT project provides social motivations and informed design decisions for this work and is carrying out longitudinal acceptance testing of the processes developed here

A framework for Adaptive Capability Profiling

This thesis documents research providing improvements in the field of accessibility modelling, which will be of particular interest as computing becomes increasingly ubiquitous. It is argued that a new approach is required that takes into account the dynamic relationship between users, their technology (both hardware and software) and any additional Assistive Technologies (ATs) that may be required. In addition, the approach must find a balance between fidelity and transportability. A theoretical framework has been developed that is able to represent both users and technology in symmetrical (hierarchical) recursive profiles, using a vocabulary that moves from device-specific to device-agnostic capabilities. The research has resulted in the development of a single unified solution that is able to functionally assess the accessibility of interactions through the use of pattern matching between graph-based profiles. A self-efficacy study was also conducted, which identified the inability of older people to provide the data necessary to drive a system based on the framework. Subsequently, the ethical considerations surrounding the use of automated data collection agents were discussed and a mechanism for representing contextual information was also included. Finally, real user data was collected and processed using a practically implemented prototype to provide an evaluation of the approach. The thesis represents a contribution through its ability to both: (1) accommodate the collection of data from a wide variety of sources, and (2) support accessibility assessments at varying levels of abstraction in order to identify if/where assistance may be necessary. The resulting approach has contributed to a work-package of the Sus-IT project, under the New Dynamics of Ageing (NDA) programme of research in the UK. It has also been presented to a W3C Research and Development Working Group symposium on User Modelling for Accessibility (UM4A). Finally, dissemination has been taken forward through its inclusion as an invited paper presented during a subsequent parallel session within the 8th International Conference on Universal Access in Human-Computer Interaction