Intrinsically Embedded Signatures for Multimedia Forensics

This dissertation examines the use of signatures that are intrinsically embedded in media recordings for studies and applications in multimedia forensics. These near-invisible signatures are fingerprints that are captured unintentionally in a recording due to influences from the environment in which it was made and the recording device that was used to make it. We focus on two types of such signatures: the Electric Network Frequency (ENF) signal and the flicker signal. The ENF is the frequency of power distribution networks and has a nominal value of 50Hz or 60Hz. The ENF fluctuates around its nominal value due to load changes in the grid. It is particularly relevant to multimedia forensics because ENF variations captured intrinsically in a media recording reflect the time and location related properties of the respective area in which it was made. This has led to a number of applications in information forensics and security, such as time-of-recording authentication/estimation and ENF-based detection of tampering in a recording. The first part of this dissertation considers the extraction and detection of the ENF signal. We discuss our proposed spectrum combining approach for ENF estimation that exploits the presence of ENF traces at several harmonics within the same recording to produce more accurate and robust ENF signal estimates. We also explore possible factors that can promote or hinder the capture of ENF traces in recordings, which is important for a better understanding of the real-world applicability of ENF signals. Next, we discuss novel real-world ENF-based applications proposed through this dissertation research. We discuss using the embedded ENF signal to identify the region-of-recording of a media signal through a pattern analysis and learning framework that distinguishes between ENF signals coming from different power grids. We also discuss the use of the ENF traces embedded in a video to characterize the video camera that had originally produced the video, an application that was inspired by our work on flicker forensics. The last part of the dissertation considers the flicker signal and its use in forensics. We address problems in the entertainment industry pertaining to movie piracy related investigations, where a pirated movie is formed by camcording media content shown on an LCD screen. The flicker signature can be inherently created in such a scenario due to the interplay between the back-light of an LCD screen and the recording mechanism of the video camera. We build an analytic model of the flicker, relating it to inner parameters of the video camera and the screen producing the video. We then demonstrate that solely analyzing such a pirated video can lead to the identification of the video camera and the screen that produced the video, which can be used as corroborating evidence in piracy investigations

Three Dimensional Denoising Filter For Effective Source Smartphone Video Identification and Verification

The field of digital image and video forensics has recently seen significant advances and has attracted attention from a growing number of researchers given the availability of imaging functionalities in most current multimedia devices at no cost including smartphones and tablets. Photo response non-uniformity (PRNU) noise is a sensor pattern noise characterizing the imaging device. However, estimating the PRNU from smartphone videos can be a challenging process because of the lossy compression that digital videos normally undergo for various reasons in addition to other non-unique noise components that interfere with the video data. This paper presents a new filtering technique for PRNU estimation based on the three-dimensional discrete wavelet transform followed by a 3D wiener filter. The rationale is that the 3D filter can filter out the compression artifacts along the temporal dimension in a more effective way than simple averaging. Experimental results on a public video dataset captured by various smartphone devices have shown a significant gain obtained with the proposed approach over the well-known two-dimensional wavelet-based Wiener approach

The New Reflexivity: Puzzle Films, Found Footage, and Cinematic Narration in the Digital Age

“The New Reflexivity” tracks two narrative styles of contemporary Hollywood production that have yet to be studied in tandem: the puzzle film and the found footage horror film. In early August 1999, near the end of what D.N. Rodowick refers to as “the summer of digital paranoia,” two films entered the wide-release U.S. theatrical marketplace and enjoyed surprisingly massive financial success, just as news of the “death of film” circulated widely. Though each might typically be classified as belonging to the horror genre, both the unreliable “puzzle film” The Sixth Sense and the fake-documentary “found footage film” The Blair Witch Project stood as harbingers of new narrative currents in global cinema. This dissertation looks closely at these two films, reading them as illustrative of two decidedly millennial narrative styles, styles that stepped out strikingly from the computer-generated shadows cast by big-budget Hollywood. The industrial shift to digital media that coincides with the rise of these films in the late 90s reframed the cinematic image as inherently manipulable, no longer a necessary index of physical reality. Directors become image-writers, constructing photorealistic imagery from scratch. Meanwhile, DVDs and online paratexts encourage cinephiles to digitize, to attain and interact with cinema in novel ways. “The New Reflexivity” reads The Sixth Sense and The Blair Witch Project as reflexive allegories of cinema’s and society’s encounters with new digital media. The most basic narrative tricks and conceits of puzzle films and found footage films produce an unusually intense and ludic engagement with narrative boundaries and limits, thus undermining the naturalized practices of classical Hollywood narration. Writers and directors of these films treat recorded events and narrative worlds as reviewable, remixable, and upgradeable, just as Hollywood digitizes and tries to keep up with new media. Though a great deal of critical attention has been paid to both puzzle and found footage films separately, no lengthy critical survey has yet been undertaken that considers these movies in terms of their shared formal and thematic concerns. Rewriting the rules of popular cinematic narration, these films encourage viewers to be suspicious of what they see onscreen, to be aware of the possibility of unreliable narration, or CGI and the “Photoshopped.” Urgent to film and cultural studies, “The New Reflexivity” suggests that these genres’ complicitous critique of new media is decidedly instructive for a networked society struggling with what it means to be digital

Toward Robust Video Event Detection and Retrieval Under Adversarial Constraints

The continuous stream of videos that are uploaded and shared on the Internet has been leveraged by computer vision researchers for a myriad of detection and retrieval tasks, including gesture detection, copy detection, face authentication, etc. However, the existing state-of-the-art event detection and retrieval techniques fail to deal with several real-world challenges (e.g., low resolution, low brightness and noise) under adversary constraints. This dissertation focuses on these challenges in realistic scenarios and demonstrates practical methods to address the problem of robustness and efficiency within video event detection and retrieval systems in five application settings (namely, CAPTCHA decoding, face liveness detection, reconstructing typed input on mobile devices, video confirmation attack, and content-based copy detection). Specifically, for CAPTCHA decoding, I propose an automated approach which can decode moving-image object recognition (MIOR) CAPTCHAs faster than humans. I showed that not only are there inherent weaknesses in current MIOR CAPTCHA designs, but that several obvious countermeasures (e.g., extending the length of the codeword) are not viable. More importantly, my work highlights the fact that the choice of underlying hard problem selected by the designers of a leading commercial solution falls into a solvable subclass of computer vision problems. For face liveness detection, I introduce a novel approach to bypass modern face authentication systems. More specifically, by leveraging a handful of pictures of the target user taken from social media, I show how to create realistic, textured, 3D facial models that undermine the security of widely used face authentication solutions. My framework makes use of virtual reality (VR) systems, incorporating along the way the ability to perform animations (e.g., raising an eyebrow or smiling) of the facial model, in order to trick liveness detectors into believing that the 3D model is a real human face. I demonstrate that such VR-based spoofing attacks constitute a fundamentally new class of attacks that point to a serious weaknesses in camera-based authentication systems. For reconstructing typed input on mobile devices, I proposed a method that successfully transcribes the text typed on a keyboard by exploiting video of the user typing, even from significant distances and from repeated reflections. This feat allows us to reconstruct typed input from the image of a mobile phone’s screen on a user’s eyeball as reflected through a nearby mirror, extending the privacy threat to include situations where the adversary is located around a corner from the user. To assess the viability of a video confirmation attack, I explored a technique that exploits the emanations of changes in light to reveal the programs being watched. I leverage the key insight that the observable emanations of a display (e.g., a TV or monitor) during presentation of the viewing content induces a distinctive flicker pattern that can be exploited by an adversary. My proposed approach works successfully in a number of practical scenarios, including (but not limited to) observations of light effusions through the windows, on the back wall, or off the victim’s face. My empirical results show that I can successfully confirm hypotheses while capturing short recordings (typically less than 4 minutes long) of the changes in brightness from the victim’s display from a distance of 70 meters. Lastly, for content-based copy detection, I take advantage of a new temporal feature to index a reference library in a manner that is robust to the popular spatial and temporal transformations in pirated videos. My technique narrows the detection gap in the important area of temporal transformations applied by would-be pirates. My large-scale evaluation on real-world data shows that I can successfully detect infringing content from movies and sports clips with 90.0% precision at a 71.1% recall rate, and can achieve that accuracy at an average time expense of merely 5.3 seconds, outperforming the state of the art by an order of magnitude.Doctor of Philosoph

Personal imaging

Thesis (Ph.D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts & Sciences, 1997.Includes bibliographical references (p. 217-223).In this thesis, I propose a new synergy between humans and computers, called "Humanistic Intelligence" (HI), and provide a precise definition of this new form of human-computer interaction. I then present a means and apparatus for reducing this principle to practice. The bulk of this thesis concentrates on a specific embodiment of this invention, called Personal Imaging, most notably, a system which I show attains new levels of creativity in photography, defines a new genre of documentary video, and goes beyond digital photography/video to define a new renaissance in imaging, based on simple principles of projective geometry combined with linearity and superposition properties of light. I first present a mathematical theory of imaging which allows the apparatus to measure, to within a single unknown constant, the quantity of light arriving from each direction, to a fixed point in space, using a collection of images taken from a sensor array having a possibly unknown nonlinearity. Within the context of personal imaging, this theory is a contribution in and of itself (in the sense that it was an unsolved problem previously), but when also combined with the proposed apparatus, it allows one to construct environment maps by simply looking around. I then present a new form of connected humanistic intelligence in which individuals can communicate, across boundaries of time and space, using shared environment maps, and the resulting computer-mediated reality that arises out of long-term adaptation in a personal imaging environment. Finally, I present a new philosophical framework for cultural criticism which arises out of a new concept called 'humanistic property'. This new philosophical framework has two axes, a 'reflectionist' axis and a 'diffusionist' axis. In particular, I apply the new framework to personal imaging, thus completing a body of work that lies at the intersection of art, science, and technology.by Steve Mann.Ph.D

Strategies for Unbridled Data Dissemination: An Emergency Operations Manual

This project is a study of free data dissemination and impediments to it. Drawing upon post-structuralism, Actor Network Theory, Participatory Action Research, and theories of the political stakes of the posthuman by way of Stirnerian egoism and illegalism, the project uses a number of theoretical, technical and legal texts to develop a hacker methodology that emphasizes close analysis and disassembly of existent systems of content control. Specifically, two tiers of content control mechanisms are examined: a legal tier, as exemplified by Intellectual Property Rights in the form of copyright and copyleft licenses, and a technical tier in the form of audio, video and text-based watermarking technologies. A series of demonstrative case studies are conducted to further highlight various means of content distribution restriction. A close reading of a copyright notice is performed in order to examine its internal contradictions. Examples of watermarking employed by academic e-book and journal publishers and film distributors are also examined and counter-forensic techniques for removing such watermarks are developed. The project finds that both legal and technical mechanisms for restricting the flow of content can be countervailed, which in turn leads to the development of different control mechanisms and in turn engenders another wave of evasion procedures. The undertaken methodological approach thus leads to the discovery of on-going mutation and adaptation of in-between states of resistance. Finally, an analysis of various existent filesharing applications is performed, and a new Tor-based BitTorrent tracker is set up to strengthen the anonymization of established filesharing methods. It is found that there exist potential de-anonymization attacks against all analyzed file-sharing tools, with potentially more secure filesharing options also seeing less user adoption

Variable Format: Media Poetics and the Little Database

This dissertation explores the situation of twentieth-century art and literature becoming digital. Focusing on relatively small online collections, I argue for materially invested readings of works of print, sound, and cinema from within a new media context. With bibliographic attention to the avant-garde legacy of media specificity and the little magazine, I argue that the “films,” “readings,” “magazines,” and “books” indexed on a series of influential websites are marked by meaningful transformations that continue to shape the present through a dramatic reconfiguration of the past. I maintain that the significance of an online version of a work is not only transformed in each instance of use, but that these versions fundamentally change our understanding of each historical work in turn. Here, I offer the analogical coding of these platforms as “little databases” after the little magazines that served as the vehicle of modernism and the historical avant-garde. Like the study of the full run of a magazine, these databases require a bridge between close and distant reading. Rather than contradict each other as is often argued, in this instance a combined macro- and microscopic mode of analysis yields valuable information not readily available by either method in isolation. In both directions, the social networks and technical protocols of database culture inscribe the limits of potential readings. Bridging the material orientation of bibliographic study with the format theory of recent media scholarship, this work constructs a media poetics for reading analog works situated within the windows, consoles, and networks of the twenty-first century

Globalizing Pakistani Identity Across The Border: The Politics of Crossover Stardom in the Hindi Film Industry

Few studies have examined how the changing landscape of cultural production in India shapes the negotiation of religious and national identities onscreen in an increasingly integrated media environment. This dissertation explores the representation and reception of three crossover stars in India: Ali Zafar, Fawad Khan, and Mahira Khan. These stars embody a new global imaginary for Pakistan that contradicts the ghettoized depictions of Pakistan as a terrorist state that continue to be perpetuated in both India and the West. What does it mean to be a Pakistani star in India, and which ‘identity’ takes precedence in popular discourse – national or religious, if any such distinction can be made? Pakistani crossover stars blur audiovisual boundaries between India, Pakistan and the West that problematize these questions of identity on multiple levels, a process mediated by globalization. At the same time, they expose conflicts over identity and belonging in an age when political nationalisms and global capitalism are colliding at an unprecedented rate

From rituals to films: a case study of the visual rhetoric of Igbo culture in Nolywood films

Many reasons have been advanced as to why the video film industry in Nigeria has been so successful financially and in building loyal audiences among Africans around the world. The present thesis argues that Nollywood films help to provide a time and a place for resolving deep-felt tensions in an increasingly modern world while affirming an authentic African [Igbo] identity. The way contemporary video films are produced brings these films close to the dominant emotional and identity questions posed by the Igbos, Nigerians and Africans alike. Particularly in Nigeria, the Nollywood film industry has brought familiar symbolic rituals of cultures on to the screen for audiences’ pleasure. Exploring the recurrent themes of these films raises consciousness about Nollywood as a new and special site where cultures are generated and regenerated. Here, major questions of values and meanings of life are explored, which raise awareness of the Igbo’s journey as a people. This thesis uses textual analysis as well as indigenous audience focus-group analysis to explore cultural representations in Nollywood. A wide range of participants were interviewed in the eight focus-group sessions that were conducted. Two in-depth interview sessions were also carried out on some Nollywood actors. Broadly, this research objectives were: • to identify a conceptual framework for understanding the culture of Africa and Nigeria, in particular, using the concept of ‘communalism’. • to determine the range of reception and consumption modalities of Nollywood products in Nigeria by means of focus-group interviews. • to explore the impact of Nollywood as an industry in the wake of globalization and in the context of current global trends. In pursuing these goals, this study looked at selected key films including, Things Fall Apart (1986), Coronation (2004), Bless Me (2005), Igodo: The Land of the Living Dead (1999), Living in Bondage (1992), My Best Friend (2003), Oil Village (2001), Widow (2007), Last Ofala (2002), Fool at 40 (2006), Festival of Fire (1999) and a lot more as listed in this study’s filmography. At the end this research found that the experience of Nollywood films is something of a centripetal process of communication for the Igbo and Nigerian viewers who believe that these texts help build their societies, culturally from below

Video Vortex reader : responses to Youtube

The Video Vortex Reader is the first collection of critical texts to deal with the rapidly emerging world of online video – from its explosive rise in 2005 with YouTube, to its future as a significant form of personal media. After years of talk about digital convergence and crossmedia platforms we now witness the merger of the Internet and television at a pace no-one predicted. These contributions from scholars, artists and curators evolved from the first two Video Vortex conferences in Brussels and Amsterdam in 2007 which focused on responses to YouTube, and address key issues around independent production and distribution of online video content. What does this new distribution platform mean for artists and activists? What are the alternatives