Towards Human Explainable Digital Forensics: Generating Human Interpretable Evidence for Semantic Understanding in Manipulated Images and Text

Abstract

Detecting and characterizing manipulations in digital media continues to pose a significant challenge within the field of digital forensics. Despite notable advancements, the discipline often remains in a reactive stance against emerging threats. Current state-of-the-art methods, typically evaluated within academic settings, fails to mirror the complexities of real-world disinformation scenarios. These methods generally prioritize high performance based on quantitative metrics, yet they demonstrate a considerable dependency on training data and lack adaptability to new novel attack signatures. With the rapid evolution of attack methodologies, the dependency on highly accurate models that do not generalize or adapt well to unseen threats proves inadequate for safeguarding today\u27s media landscape. We propose that the forensic community reevaluate methodological design from a threat assessment perspective. Approaches that prioritize explainability, adaptability, and independence from specific data sets are crucial as a primary line of defense. If new disinformation signatures remain undetected, defending against them becomes a formidable challenge. We advocate for a shift within the community towards developing models capable of producing actionable evidence that semantically explains the rationale behind identified manipulations. This shift is imperative as machine learning models become increasingly accessible to the public and all future media inevitably incorporates some synthetic elements. Identifying whether such alterations pose a semantic threat through a comprehensive threat assessment framework is essential to maintaining the integrity of the media we consume