Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Digital Watermarking for Verification of Perception-based Integrity of Audio Data

In certain application fields digital audio recordings contain sensitive content. Examples are historical archival material in public archives that preserve our cultural heritage, or digital evidence in the context of law enforcement and civil proceedings. Because of the powerful capabilities of modern editing tools for multimedia such material is vulnerable to doctoring of the content and forgery of its origin with malicious intent. Also inadvertent data modification and mistaken origin can be caused by human error. Hence, the credibility and provenience in terms of an unadulterated and genuine state of such audio content and the confidence about its origin are critical factors. To address this issue, this PhD thesis proposes a mechanism for verifying the integrity and authenticity of digital sound recordings. It is designed and implemented to be insensitive to common post-processing operations of the audio data that influence the subjective acoustic perception only marginally (if at all). Examples of such operations include lossy compression that maintains a high sound quality of the audio media, or lossless format conversions. It is the objective to avoid de facto false alarms that would be expectedly observable in standard crypto-based authentication protocols in the presence of these legitimate post-processing. For achieving this, a feasible combination of the techniques of digital watermarking and audio-specific hashing is investigated. At first, a suitable secret-key dependent audio hashing algorithm is developed. It incorporates and enhances so-called audio fingerprinting technology from the state of the art in contentbased audio identification. The presented algorithm (denoted as ”rMAC” message authentication code) allows ”perception-based” verification of integrity. This means classifying integrity breaches as such not before they become audible. As another objective, this rMAC is embedded and stored silently inside the audio media by means of audio watermarking technology. This approach allows maintaining the authentication code across the above-mentioned admissible post-processing operations and making it available for integrity verification at a later date. For this, an existent secret-key ependent audio watermarking algorithm is used and enhanced in this thesis work. To some extent, the dependency of the rMAC and of the watermarking processing from a secret key also allows authenticating the origin of a protected audio. To elaborate on this security aspect, this work also estimates the brute-force efforts of an adversary attacking this combined rMAC-watermarking approach. The experimental results show that the proposed method provides a good distinction and classification performance of authentic versus doctored audio content. It also allows the temporal localization of audible data modification within a protected audio file. The experimental evaluation finally provides recommendations about technical configuration settings of the combined watermarking-hashing approach. Beyond the main topic of perception-based data integrity and data authenticity for audio, this PhD work provides new general findings in the fields of audio fingerprinting and digital watermarking. The main contributions of this PhD were published and presented mainly at conferences about multimedia security. These publications were cited by a number of other authors and hence had some impact on their works

Description-driven Adaptation of Media Resources

The current multimedia landscape is characterized by a significant diversity in terms of available media formats, network technologies, and device properties. This heterogeneity has resulted in a number of new challenges, such as providing universal access to multimedia content. A solution for this diversity is the use of scalable bit streams, as well as the deployment of a complementary system that is capable of adapting scalable bit streams to the constraints imposed by a particular usage environment (e.g., the limited screen resolution of a mobile device). This dissertation investigates the use of an XML-driven (Extensible Markup Language) framework for the format-independent adaptation of scalable bit streams. Using this approach, the structure of a bit stream is first translated into an XML description. In a next step, the resulting XML description is transformed to reflect a desired adaptation of the bit stream. Finally, the transformed XML description is used to create an adapted bit stream that is suited for playback in the targeted usage environment. The main contribution of this dissertation is BFlavor, a new tool for exposing the syntax of binary media resources as an XML description. Its development was inspired by two other technologies, i.e. MPEG-21 BSDL (Bitstream Syntax Description Language) and XFlavor (Formal Language for Audio-Visual Object Representation, extended with XML features). Although created from a different point of view, both languages offer solutions for translating the syntax of a media resource into an XML representation for further processing. BFlavor (BSDL+XFlavor) harmonizes the two technologies by combining their strengths and eliminating their weaknesses. The expressive power and performance of a BFlavor-based content adaptation chain, compared to tool chains entirely based on either BSDL or XFlavor, were investigated by several experiments. One series of experiments targeted the exploitation of multi-layered temporal scalability in H.264/AVC, paying particular attention to the use of sub-sequences and hierarchical coding patterns, as well as to the use of metadata messages to communicate the bit stream structure to the adaptation logic. BFlavor was the only tool to offer an elegant and practical solution for XML-driven adaptation of H.264/AVC bit streams in the temporal domain

Real-time video streaming using peer-to-peer for video distribution

The growth of the Internet has led to research and development of several new and useful applications including video streaming. Commercial experiments are underway to determine the feasibility of multimedia broadcasting using packet based data networks alongside traditional over-the-air broadcasting. Broadcasting companies are offering low cost or free versions of video content online to both guage and at the same time generate popularity. In addition to television broadcasting, video streaming is used in a number of application areas including video conferencing, telecommuting and long distance education. Large scale video streaming has not become as widespread or widely deployed as could be expected. The reason for this is the high bandwidth requirement (and thus high cost) associated with video data. Provision of a constant stream of video data on a medium to large scale typically consumes a significant amount of bandwidth. An effect of this is that encoding bit rates are lowered and consequently video quality is degraded resulting in even slower uptake rates for video streaming services. The aim of this dissertation is to investigate peer-to-peer streaming as a potential solution to this bandwidth problem. The proposed peer-to-peer based solution relies on end user co-operation for video data distribution. This approach is highly effective in reducing the outgoing bandwidth requirement for the video streaming server. End users redistribute received video chunks amongst their respective peers and in so doing increase the potential capacity of the entire network for supporting more clients. A secondary effect of such a system is that encoding capabilities (including higher encoding bit rates or encoding of additional sub-channels) can be enhanced. Peer-to-peer distribution enables any regular user to stream video to large streaming networks with many viewers. This research includes a detailed overview of the fields of video streaming and peer-to-peer networking. Techniques for optimal video preparation and data distribution were investigated. A variety of academic and commercial peer-to-peer based multimedia broadcasting systems were analysed as a means to further define and place the proposed implementation in context with respect to other peercasting implementations. A proof-of-concept of the proposed implementation was developed, mathematically analyzed and simulated in a typical deployment scenario. Analysis was carried out to predict simulation performance and as a form of design evaluation and verification. The analysis highlighted some critical areas which resulted in adaptations to the initial design as well as conditions under which performance can be guaranteed. A simulation of the proof-of-concept system was used to determine the extent of bandwidth savings for the video server. The aim of the simulations was to show that it is possible to encode and deliver video data in real time over a peer-to-peer network. The proposed system achieved expectations and showed significant bandwidth savings for a sustantially large video streaming audience. The implementation was able to encode video in real time and continually stream video packets on time to connected peers while continually supporting network growth by connecting additional peers (or stream viewers). The system performed well and showed good performance under typical real world restrictions on available bandwith capacity.Dissertation (MEng)--University of Pretoria, 2009.Electrical, Electronic and Computer Engineeringunrestricte

Performance and Energy Consumption Characterization and Modeling of Video Decoding on Multi-core Heterogenous SoC and their Applications

To meet the increasing complexity of mobile multimedia applications, the System on Chip (SoC) equipping modern mobile devices integrate powerful heterogeneous processing elements among which General Purpose Processors (GPP), Digital Signal Processors (DSP), hardware accelerator are the most common ones.Due to the ever-growing gap between battery lifetime and hardware/software complexity in addition to application computing power needs, the energy saving issue becomes crucial in the design of such systems. In this context, we propose a study aiming to enhance the understanding of the energy consumption behavior of video decoding on these kinds of systems. Accordingly, an end-to-end methodology for characterizing and modeling the performance and the energy consumption of video decoding on GPP and DSP is proposed. The characterization step is based on an exhaustive experimental methodology for evaluating, at different abstraction levels, the performance and the energy consumption of video decoding. It was achieved on embedded platforms on which were executed a wide range of video decoding configurations. This step highlighted the importance to consider different parameters which may pertain to different abstraction levels in evaluating the overall energy efficiency of a given system. The measurements obtained in this step were used to build empirically performance and energy models for video decoding on both GPP and DSP. The proposed models gave very accurate estimation (R 2 = 97%) of both the performance and the energy consumption of video decoding in terms of a rich set of parameters including the video quality and the processor frequency. Moreover, based on a multi-level characterization and sub-model decomposition approaches, we show how the developed models, unlike classic empirical models, are easily and rapidly generalizable to other platforms.Some possible applications using the developed models, in the context of adaptive video decoding, were proposed. In general, it consists to use the capability of the proposed performance model to predict the decoding time of a given video quality in dimensioning/scheduling the processing resources. Due to the increasing demand on High Definition (HD), the characterization methodology was extended to consider HD video decoding on both parallel multi-cores and hardware video accelerator. This part highlighted the potential of parallelism video decoding to increase the energy efficiency of video decoding and point out some open issues in this domain.Pour répondre à la complexité croissante des applications multimédia mobiles, les systèmes sur puce équipant les appareils mobiles modernes intègrent des unités de calcul puissantes et hétérogène. Parmi ces units de calcul, on peut trouver des processeurs à usage général, des processeur de traitement de signal et des accélérateurs matériels. En raison de l’écart toujours croissant entre la durée de vie des batteries et la demande de plus en plus importante en puissance de calcul, l’économie d’énergie devient un enjeu crucial dans la conception des systèmes mobiles. Cette problématique est accentuée par l’augmentation de la complexité des logiciels et architectures matériels utilisés. Dans ce contexte, nous proposons une étude visant à améliorer la compréhension des considérations énergétiques du décodage vidéo sur ce genre de systèmes. Nous proposerons ainsi une méthodologie pour la caractérisation et la modélisation des performances et de la consommation d’énergie du décodage vidéo, aussi bien sur des processeurs à usage général de type ARM que sur un processeurde traitement de signal. L’étape de caractérisation est basée sur une méthodologie expérimentale pour évaluer de façon exhaustive et à différents niveaux d’abstraction, les performances et la consommation d’énergie du décodage vidéo. Cette caractérisation a été réalisée sur des plates-formes embarquées sur lesquels ont été exécutés un large éventail de configurations du décodage vidéo. Cette étape a souligné l’importance d’examiner différents paramètres qui peuvent se rapporter à différents niveaux d’abstraction dans l’évaluation de l’efficacité énergétique globale d’un système donné. Les mesures obtenues dans cette étape ont été utilisées pour construire empiriquement des modèles de performance et de consommation d’énergie pour le décodage vidéo à la fois sur des processeurs à usage général type ARM et sur un processeur de traitement de signal. Les modèles proposés peuvent estimer avec une grande précision (R 2 = 97%) la performance et la consommation d’énergie de décodage vidéo en fonction d’un nombre de paramètres comprenant la qualité de la vidéo et la fréquence du processeur. En plus, en se basant sur une caractérisation multi-niveaux et une approches de modélisation par décomposition en sous-modèles, nous montrons comment les modèles développés, contrairement aux modèles empiriques classiques, sont facilement et rapidement généralisables à d’autres plates-formes. Nous proposerons également certaines applications possibles des modèles développés, dans le cadre du décodage vidéo adaptatif. En général, cela consiste à exploiter la capacité du modèle de performance proposé pour prédire le temps de décodage d’une qualité vidéo donnée afin de mieux dimensionner les ressources de calculs dans un but de réduire leur consommationd’énergie

Building information modeling – A game changer for interoperability and a chance for digital preservation of architectural data?

Digital data associated with the architectural design-andconstruction process is an essential resource alongside -and even past- the lifecycle of the construction object it describes. Despite this, digital architectural data remains to be largely neglected in digital preservation research – and vice versa, digital preservation is so far neglected in the design-and-construction process. In the last 5 years, Building Information Modeling (BIM) has seen a growing adoption in the architecture and construction domains, marking a large step towards much needed interoperability. The open standard IFC (Industry Foundation Classes) is one way in which data is exchanged in BIM processes. This paper presents a first digital preservation based look at BIM processes, highlighting the history and adoption of the methods as well as the open file format standard IFC (Industry Foundation Classes) as one way to store and preserve BIM data

Conceptual metaphor in English popular technology and Greek translation

This research project studies the metaphorical conceptualisation of technology in English popular technology magazines and in translation in the respective Greek editions. The focus is on the cognitive linguistic view of metaphor initially presented by Lakoff and Johnson (1980), on the metaphor identification procedure (Pragglejaz Group 2007), and critical metaphor analysis (Charteris-Black 2004). The analysis of the English data identifies 14 main metaphors and 29 submetaphors which contribute to the structure of the target domain of technology. It distinguishes between conventional and novel metaphors, and common and original metaphorical expressions, motivated by correlations in experience between diverse source domains and by the widespread diffusion and impact of technology. The English data also provide insight into the functions of these metaphors in popular technology discourse and reveal evidence to thinking, values and attitudes about technology in the English language. The analysis of the Greek data examines similarities and differences in the conceptualisations between the English and Greek languages and cultures, and finds similarities in the categories of metaphors, frequency of and preference for metaphor use in the source and target languages, and in the majority of metaphorical expressions. Similarities are based on common experiences stemming from experiential co-occurrence or experiential similarity, and on translated experience. Differences are restricted to specific-level metaphors and expressions, motivated by alternative conceptualisations of terminology, cultural specificity and preferential conceptualisations. A set of translation strategies and a number of possible translation effects are also identified. These strategies and effects add to the possibilities of translation variations and the range of translation options, and are used to draw conclusions regarding the similarities and differences between the English and Greek languages and cultures. Consequently, through the identification and description of metaphors in technology magazines and in translation, the study attempts to highlight aspects of the culture of technology, which views technology as a cultural artefact and a producer of its own culture.EThOS - Electronic Theses Online ServiceGreek State Scholarship Foundation (IKY)GBUnited Kingdo

Teacher roles during amusement park visits – insights from observations, interviews and questionnaires

Amusement parks offer rich possibilities for physics learning, through observations and experiments that illustrate important physical principles and often involve the whole body. Amusement parks are also among the most popular school excursions, but very often the learning possibilities are underused. In this work we have studied different teacher roles and discuss how universities, parks or event managers can encourage and support teachers and schools in their efforts to make amusement park visits true learning experiences for their students