Bhowmik D

Caldwell S

Ebrahimi T

Temmermans F

English

Newcastle University E-Prints

      JPEG Trust: an international standard facilitating the assessment of trustworthiness of digital media assets  Frederik Temmermans1,a,b, Sabrina Caldwellc, Deepayan Bhowmikd, Touradj Ebrahimie aVrije Universiteit Brussel, Belgium; bimec, Belgium; cThe Australian National University, Australia; dUniversity of Stirling, United Kingdom; eEcole Polytechnique Fédérale de Lausanne (EPFL), Switzerland ABSTRACT JPEG Trust is a new international standard that responds to the pressing need of assessing trust in digital media assets. JPEG Trust provides a comprehensive framework addressing key elements such as provenance, authenticity, integrity, and copyright declaration. Built on top of established JPEG and other industry standards, the framework ensures compatibility across digital media ecosystems. This paper provides an overview of the JPEG Trust framework and demonstrates its potential to enhance the reliability and trustworthiness of digital media content. Keywords: JPEG Trust, media authenticity, provenance, trust, interoperability, JPEG, standardization  1. INTRODUCTION Nowadays, images originate from a wide variety of sources, including but not limited to digital cameras, scanned film photos, photo editing software, artificial intelligence algorithms (e.g., Generative AI), and various combinations thereof. These technological advancements enable the creation of innovative imagery and even contribute to new insights derived from our media assets. However, they also pose the risk of causing confusion and deception. To take advantage of the beneficial outcomes of modern digital media production technologies while mitigating the associated threats, it is crucial to be able to assess if the media we encounter can be trusted. In this context, the JPEG standardization committee (ISO/IEC JTC 1/SC 29/WG 1) initiated the development of a new international standard: JPEG Trust (ISO/IEC 21617). JPEG Trust arises from an exploration that started five years ago around requirements for addressing mis- and dis- information in digital media. JPEG Trust provides a comprehensive framework for individuals, organizations, and governing institutions interested in establishing an environment of trust for the media that they use, and supporting trust in the media they share online. This framework addresses aspects of provenance, authenticity, integrity, copyright declaration, and identification of assets and stakeholders. The JPEG Trust framework is built in compliance with well-established JPEG standards as well as other widely adopted industry standards to ensure a smooth integration into existing digital media ecosystems. The framework can be integrated into ecosystems that use any of the JPEG family of standards (including JPEG 1, JPEG 2000, JPEG XS, JPEG XL and JPEG AI). In addition, due to its generic nature, many aspects of the framework can also be applied to other image file formats or other media modalities such as video or audio. This paper introduces the three pillars of the first part of JPEG Trust, the Core Foundation: annotating provenance information, extracting and evaluating trust indicators, and handling privacy and security concerns. The usage of the standard and its components is illustrated with concrete examples and scenarios. Since JPEG Trust is an ongoing standardization effort, the paper concludes with an overview of the next steps and their associated timeline.  1 frederik.temmermans@vub.be        2. BACKGROUND The increasing sophistication in digital media content creation and manipulation, facilitated by advancements in technology, poses significant challenges in assessing media authenticity [1]. Traditional media forensics methods have been employed to evaluate authenticity across various modalities such as images, audio, and video [2]. These methods, including identifying camera models and detecting splicing and copy-move forgeries, are effective in certain scenarios, however they are not readily available and/or timely for most users, and do not address the need for proactive assertion of media integrity. The advent of AI-based synthetic media, including deepfakes and Generative Adversarial Networks (GANs), has further exacerbated these challenges [3]. These technologies enable seamless manipulation and creation of media content, complicating the detection of the origin and subsequent alterations [4][5]. Despite continuing evolution of detection methods, high false positive rates and the complexity of interpreting results remain significant obstacles [6][7]. In response to these challenges, the JPEG standardization committee initiated the “JPEG Fake Media” exploration in October 2020 [8]. This effort aimed to develop a secure and interoperable standard for annotating the provenance and modifications of media assets [9]. In April 2022, the Call for Proposals on JPEG Fake Media resulted in the initiation of the JPEG Trust standard, which aims to create a comprehensive framework for establishing trust in media production, distribution, and consumption [10]. 3. ANNOTATING PROVENANCE INFORMATION Given the proliferation of generated and manipulated digital media, it is crucial that end-users are offered tools that allow them to assess if the media they encounter can be trusted. These tools may be reactive or proactive. Reactive tools include algorithms that analyze the content, such as deepfake or forgery detection methods. A proactive approach, on the other hand, relies on the secure and precise signaling of the provenance information. While reactive approaches have been proven effective in certain scenarios, they always suffer from their inherent cat-and-mouse nature. While the JPEG Trust framework supports reactive methods, as further detailed in the following section, the long-term vision is more oriented towards proactive methods empowering provenance annotations from the moment a media asset is created until the moment it is presented to the end-user. This approach is also aligned with the trust model of the Coalition for Content Provenance and Authenticity (C2PA) [11]. C2PA is an industry initiative that specifies the technical means for combining statements of fact with a digital signature. Some of these facts include cryptographic bindings, information about human, non-human, and AI-based actors, and the actions (creation, editing, type of editing, etc.) performed in the creation or modification of the media asset. Given the aligned vision and the industry adoption that has already been achieved by C2PA, JPEG Trust adopts and builds on top of the C2PA specification for those aspects related to provenance signaling. In practice, this means that media assets that have C2PA-compliant provenance information are fully compatible with the JPEG Trust framework. 4. EXTRACTING AND EVALUATING TRUST INDICATORS Framework Within the JPEG Trust framework, trust indicators are defined as “pieces of information derived from a media asset that indicate a level of trustworthiness of that media asset in a given context”. The final words “in a given context” are crucial in this definition. Trustworthiness cannot be universally determined. Whether something can be considered trustworthy or not always depends on a given context and may even be subjective. For example, consider a person involved in a car accident who shares a picture of the damage to their car with family members. Since those family members can verify that the picture was taken and sent by their relatives, they will not question its veracity. However, when this same picture is shared with an insurance company to be used as evidence, additional trust indicators would be required. The insurance company might want guarantees that the time and location are correct and that the picture was not edited after its creation. Within the JPEG Trust framework, such rules that determine whether a certain asset can be trusted in a certain context are expressed in a trust profile. The result of the assessment of a trust profile against a specific media asset is called the trust report. This pipeline is illustrated in Figure 1 and further detailed in the subsequent sections.        Figure 1: Illustration of the JPEG Trust framework. Trust indicators As shown in Figure 1, trust indicators can be extracted from various elements of the media asset: the media content, the metadata or the provenance information. Here are some examples of trust indicators for each of these elements: • Media content: result of the application of a deepfake detection algorithm, e.g. probability that the image was generated or modified by AI. • Metadata: presence of EXIF metadata or values of specific fields (note that EXIF information can easily be modified, however, such a change can be identified and as explained before, the value of a specific indicator depends on the given scenario). • Provenance information: was the image captured and digitally signed by a camera? It is important to notice that while many trust indicators are specified within the JPEG Trust Core Foundation, anyone can define their own trust indicators. This open approach enables the support of a wide range of usage scenarios which can be very domain specific. Trust credential To facilitate the processing or evaluation of trust indicators, the trust indicators extracted from a certain media asset can be expressed in a trust credential. The trust credential is a JSON-LD serialization of the extracted indicators. Note that since there can be an indefinite number of trust indicators, the trust credential can never be complete. When a trust credential is used for the evaluation of a specific trust profile it is important that the trust credential contains all the indicators utilized within the profile. In other words, the trust credential should be created with a trust indicator extractor that is compliant with the trust profile. Multiple trust indicator extractors can be used to combine their results into a single trust credential. Trust profile Trust profiles are specified by individuals or organizations to define the rules for assessing the trustworthiness of media assets in their specific context. These profiles provide a customizable and context-sensitive approach to media verification, tailored to the diverse needs of different stakeholders. For instance, individuals might create a trust profile to identify and validate images that originate from their relatives, ensuring personal and familial authenticity. A social network provider could develop a trust profile with rules designed to distinguish between images that have been verified to originate from cameras and those generated by AI. Governments could establish trust profiles that enable end users to verify whether media assets comply with local regulations, enhancing regulatory adherence and public trust. Technically, trust profiles are expressed in the YAML markup language and contain several statements that provide contextual information as well as the expressions that define the rules of the profile. These expressions are defined as JSON formulas that operate on the trust indicators from a trust credential. The results of these expressions can be binary, numerical, textual, or referential. In addition to the direct machine-processable results, a human-readable and multilingual description of the results can be specified for inclusion in the trust report.       By allowing for the creation of tailored trust profiles, the JPEG Trust framework accommodates a wide range of scenarios and requirements, ensuring that the assessment of media authenticity is both flexible and robust. This approach not only supports diverse use cases but also enhances the reliability and transparency of media verification processes across different contexts. Trust report A trust report is the result of evaluating a trust profile against a specific media asset. The primary aim of the trust report is to provide end-users with insights regarding the trustworthiness of a media asset within a given context, as specified by the provider of the specific profile. This process ensures that users can make informed decisions about the media they encounter based on standardized and/or contextually relevant criteria. The trust report can be embedded as additional provenance information within the original media asset. This embedding allows end-users to access and evaluate the report without needing to generate it themselves. The embedding of the trust report also ensures that the trustworthiness information travels with the media asset, maintaining its integrity and providing continuous verification capability. The generation of the trust report, along with its signing and embedding, can be performed by the provider of the trust profile. This practice adds a layer of trust, as the report is not self-generated but rather issued by a trusted entity. The provider’s signature serves as a certification of the report’s authenticity and reliability. In addition to providing immediate insights, trust reports can also include a detailed breakdown of the evaluation process. This breakdown can specify the criteria used, the results of individual trust indicator assessments, and any additional context necessary for understanding the trustworthiness assessment. Such detailed information supports transparency and allows end-users to understand the basis of the trust evaluation. Since trust profiles are not only intended for machine processing but also for human interpretation, trust reports can be multilingual. By providing human-readable descriptions in multiple languages, trust reports can cater to users from different regions and linguistic backgrounds, while maintaining the same semantic interpretation of trust. Trust reports play a crucial role in the JPEG Trust framework by delivering clear, reliable, and context-specific assessments of media authenticity. This capability not only aids in combating misinformation but also supports the integrity and reliability of digital media across various platforms and use cases. 5. HANDLING PRIVACY AND SECURITY CONCERNS The JPEG Trust framework provides mechanisms to associate provenance information with the media asset content and its metadata. However, in certain scenarios, this provenance information might contain sensitive information that should not be exposed when the asset is shared with a broader audience. To this end, JPEG Trust defines means to protect privacy-sensitive information by adopting well-established cryptographic encryption methods. This is facilitated by building on top of the JPEG Privacy and Security (ISO/IEC 19566-4) standard [12], as illustrated in Figure 2.  Figure 2. Integration of JPEG Privacy and Security with JPEG Trust.       The privacy provisions allow for the protection of certain information when appropriate. For example, the identity of a field photographer capturing an image in a conflict zone can be protected. In this example, the agency distributing the photograph could sign the media asset instead of the photographer. End users would not be able to identify the photographer, and it is up to them to decide whether they consider this particular image trustworthy based on the reputation of the agency. By implementing these privacy and security measures, JPEG Trust ensures that sensitive information is safeguarded without compromising the overall trustworthiness of the media asset. This approach balances the need for transparency with the necessity of protecting individuals’ privacy and security in various contexts 6. CONCLUSIONS AND NEXT STEPS The JPEG Trust Part 1 - Core Foundation (ISO/IEC 21617-1) is expected to be published as an International Standard later in 2024. In order to facilitate the assessment of media trustworthiness in various scenarios, this Core Foundation specifies three fundamental pillars: annotating provenance information, extracting and evaluating trust indicators, and handling privacy and security concerns. The Core Foundation is the starting point of the JPEG Trust framework that will evolve over time and be extended with additional functionalities in the future. Currently the JPEG Committee is working on features to better support media tokenization such as declaration of authorship, ownership and rights. In addition, a new Part 2 “Trust Profiles Catalogue” was initiated to provide suggested defined Trust Profiles for specific common usage scenarios. A Part 3 “Media Asset Watermarking” was requested to exploit the use of watermarking as one of the available components of the JPEG Trust framework to support usage scenarios for content authenticity, provenance, integrity, labeling, and binding between JPEG Trust metadata and corresponding media assets. The work of this Part 3 will focus on various types of watermarking, including explicit or visible watermarking, invisible watermarking, and implicit watermarking in the media asset metadata. Finally, reference software is being developed that will be published as a new Part of JPEG Trust in the future. ACKNOWLEDGMENT The authors acknowledge valuable exchanges with members of the JPEG Trust Ad Hoc Group. REFERENCES [1] Aengus C. & Touradj E., "Risk governance and the rise of deepfakes", Swiss Federal Institute of Technology Lausanne (EPFL), May 2021, [Online: posted 12-May-2020]. [2] Afchar, D., Nozic, V., Yamagishi, J., and Echizen, I., “MesoNet: a Compact Facial Video Forgery Detection Network,” in 2018 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–7, iSSN: 2157-4774 (2018). [3] Agarwal, S., Farid, H., Gu, Y., He, M., Nagano, K. and Li, H., "Protecting world leaders against deep fakes.", in CVPR Workshops, pp. 38–45 (2019). [4] Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., and Ferrer, C., “The DeepFake Detection Challenge (DFDC) Dataset,” arXiv:2006.07397 [cs], Oct. 2020, arXiv: 2006.07397. [Online]. Available: http://arxiv.org/abs/2006.07397 [5] Farid, H., “Digital doctoring: how to tell the real from the fake”, Significance, 3(4), 162-166 (2006). [6] Gray, C., “Using Machine Learning to Identify Fake Images”, Bachelor's thesis, University of Stirling (2019). [7] Guarnera, L., Giudice, O., and Battiato, S., "Deepfake detection by analyzing convolutional traces." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2020). [8] ISO/IEC JTC 1/SC29/WG1 N100157, REQ "Final Call for Proposal for JPEG Fake Media", 95th Meeting, Online, April 2022. [9] F. Temmermans, D. Bhowmik, F, Pereira, T. Ebrahimi, "JPEG Fake Media: a provenance-based sustainable approach to secure and trustworthy media annotation," Proc. SPIE 11842, Applications of Digital Image Processing XLIV, 118420L (1 August 2021); https://doi.org/10.1117/12.2597556       [10] F. Temmermans, S. Caldwell, S. Papadopoulos, F. Pereira and P. Rixhon, "Towards an international standard to establish trust in media production, distribution and consumption," 2023 24th International Conference on Digital Signal Processing (DSP), Rhodes (Rodos), Greece, 2023, pp. 1-5, doi: 10.1109/DSP58604.2023.10167884. [11] Rosenthol, L., Parsons, A., Scouten, E., Aythora, J., MacCormack, B., England, P., Levallee, M., Dotan, J., Hanna, S., Farid, H. and Gregory, S., “The Content Authenticity Initiative, Setting the Standard for Digital Content Attribution”, (2020). [12] Temmermans, F., Ebrahimi, T., Foessel, S., Delgado, J., Ishikawa, T., Natu, A., Schelkens, P., “JPEG Privacy and Security framework for social networking and GLAM services”, EURASIP Journal on Image and Video Processing, 2017, 68 (2017). 

JPEG Trust: an international standard facilitating the assessment of trustworthiness of digital media assets

https://eprints.ncl.ac.uk/file_store/production/301694/FDBD7896-0D74-4CCD-9793-6FD7AE277938.pdf

JPEG Trust: an international standard facilitating the assessment of trustworthiness of digital media assets

Abstract

Similar works

Full text

Available Versions

Newcastle University E-Prints