Measuring trustworthiness of image data in the internet of things environment

Internet of Things (IoT) image sensors generate huge volumes of digital images every day. However, easy availability and usability of photo editing tools, the vulnerability in communication channels and malicious software have made forgery attacks on image sensor data effortless and thus expose IoT systems to cyberattacks. In IoT applications such as smart cities and surveillance systems, the smooth operation depends on sensors’ sharing data with other sensors of identical or different types. Therefore, a sensor must be able to rely on the data it receives from other sensors; in other words, data must be trustworthy. Sensors deployed in IoT applications are usually limited to low processing and battery power, which prohibits the use of complex cryptography and security mechanism and the adoption of universal security standards by IoT device manufacturers. Hence, estimating the trust of the image sensor data is a defensive solution as these data are used for critical decision-making processes. To our knowledge, only one published work has estimated the trustworthiness of digital images applied to forensic applications. However, that study’s method depends on machine learning prediction scores returned by existing forensic models, which limits its usage where underlying forensics models require different approaches (e.g., machine learning predictions, statistical methods, digital signature, perceptual image hash). Multi-type sensor data correlation and context awareness can improve the trust measurement, which is absent in that study’s model. To address these issues, novel techniques are introduced to accurately estimate the trustworthiness of IoT image sensor data with the aid of complementary non-imagery (numeric) data-generating sensors monitoring the same environment. The trust estimation models run in edge devices, relieving sensors from computationally intensive tasks. First, to detect local image forgery (splicing and copy-move attacks), an innovative image forgery detection method is proposed based on Discrete Cosine Transformation (DCT), Local Binary Pattern (LBP) and a new feature extraction method using the mean operator. Using Support Vector Machine (SVM), the proposed method is extensively tested on four well-known publicly available greyscale and colour image forgery datasets and on an IoT-based image forgery dataset that we built. Experimental results reveal the superiority of our proposed method over recent state-of-the-art methods in terms of widely used performance metrics and computational time and demonstrate robustness against low availability of forged training samples. Second, a robust trust estimation framework for IoT image data is proposed, leveraging numeric data-generating sensors deployed in the same area of interest (AoI) in an indoor environment. As low-cost sensors allow many IoT applications to use multiple types of sensors to observe the same AoI, the complementary numeric data of one sensor can be exploited to measure the trust value of another image sensor’s data. A theoretical model is developed using Shannon’s entropy to derive the uncertainty associated with an observed event and Dempster-Shafer theory (DST) for decision fusion. The proposed model’s efficacy in estimating the trust score of image sensor data is analysed by observing a fire event using IoT image and temperature sensor data in an indoor residential setup under different scenarios. The proposed model produces highly accurate trust scores in all scenarios with authentic and forged image data. Finally, as the outdoor environment varies dynamically due to different natural factors (e.g., lighting condition variations in day and night, presence of different objects, smoke, fog, rain, shadow in the scene), a novel trust framework is proposed that is suitable for the outdoor environments with these contextual variations. A transfer learning approach is adopted to derive the decision about an observation from image sensor data, while also a statistical approach is used to derive the decision about the same observation from numeric data generated from other sensors deployed in the same AoI. These decisions are then fused using CertainLogic and compared with DST-based fusion. A testbed was set up using Raspberry Pi microprocessor, image sensor, temperature sensor, edge device, LoRa nodes, LoRaWAN gateway and servers to evaluate the proposed techniques. The results show that CertainLogic is more suitable for measuring the trustworthiness of image sensor data in an outdoor environment.Doctor of Philosoph

Image and Video Forensics

Nowadays, images and videos have become the main modalities of information being exchanged in everyday life, and their pervasiveness has led the image forensics community to question their reliability, integrity, confidentiality, and security. Multimedia contents are generated in many different ways through the use of consumer electronics and high-quality digital imaging devices, such as smartphones, digital cameras, tablets, and wearable and IoT devices. The ever-increasing convenience of image acquisition has facilitated instant distribution and sharing of digital images on digital social platforms, determining a great amount of exchange data. Moreover, the pervasiveness of powerful image editing tools has allowed the manipulation of digital images for malicious or criminal ends, up to the creation of synthesized images and videos with the use of deep learning techniques. In response to these threats, the multimedia forensics community has produced major research efforts regarding the identification of the source and the detection of manipulation. In all cases (e.g., forensic investigations, fake news debunking, information warfare, and cyberattacks) where images and videos serve as critical evidence, forensic technologies that help to determine the origin, authenticity, and integrity of multimedia content can become essential tools. This book aims to collect a diverse and complementary set of articles that demonstrate new developments and applications in image and video forensics to tackle new and serious challenges to ensure media authenticity

Towards Learning Representations in Visual Computing Tasks

abstract: The performance of most of the visual computing tasks depends on the quality of the features extracted from the raw data. Insightful feature representation increases the performance of many learning algorithms by exposing the underlying explanatory factors of the output for the unobserved input. A good representation should also handle anomalies in the data such as missing samples and noisy input caused by the undesired, external factors of variation. It should also reduce the data redundancy. Over the years, many feature extraction processes have been invented to produce good representations of raw images and videos. The feature extraction processes can be categorized into three groups. The first group contains processes that are hand-crafted for a specific task. Hand-engineering features requires the knowledge of domain experts and manual labor. However, the feature extraction process is interpretable and explainable. Next group contains the latent-feature extraction processes. While the original feature lies in a high-dimensional space, the relevant factors for a task often lie on a lower dimensional manifold. The latent-feature extraction employs hidden variables to expose the underlying data properties that cannot be directly measured from the input. Latent features seek a specific structure such as sparsity or low-rank into the derived representation through sophisticated optimization techniques. The last category is that of deep features. These are obtained by passing raw input data with minimal pre-processing through a deep network. Its parameters are computed by iteratively minimizing a task-based loss. In this dissertation, I present four pieces of work where I create and learn suitable data representations. The first task employs hand-crafted features to perform clinically-relevant retrieval of diabetic retinopathy images. The second task uses latent features to perform content-adaptive image enhancement. The third task ranks a pair of images based on their aestheticism. The goal of the last task is to capture localized image artifacts in small datasets with patch-level labels. For both these tasks, I propose novel deep architectures and show significant improvement over the previous state-of-art approaches. A suitable combination of feature representations augmented with an appropriate learning approach can increase performance for most visual computing tasks.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Graphonomics and your Brain on Art, Creativity and Innovation : Proceedings of the 19th International Graphonomics Conference (IGS 2019 – Your Brain on Art)

[Italiano]: “Grafonomia e cervello su arte, creatività e innovazione”. Un forum internazionale per discutere sui recenti progressi nell'interazione tra arti creative, neuroscienze, ingegneria, comunicazione, tecnologia, industria, istruzione, design, applicazioni forensi e mediche. I contributi hanno esaminato lo stato dell'arte, identificando sfide e opportunità, e hanno delineato le possibili linee di sviluppo di questo settore di ricerca. I temi affrontati includono: strategie integrate per la comprensione dei sistemi neurali, affettivi e cognitivi in ambienti realistici e complessi; individualità e differenziazione dal punto di vista neurale e comportamentale; neuroaesthetics (uso delle neuroscienze per spiegare e comprendere le esperienze estetiche a livello neurologico); creatività e innovazione; neuro-ingegneria e arte ispirata dal cervello, creatività e uso di dispositivi di mobile brain-body imaging (MoBI) indossabili; terapia basata su arte creativa; apprendimento informale; formazione; applicazioni forensi. / [English]: “Graphonomics and your brain on art, creativity and innovation”. A single track, international forum for discussion on recent advances at the intersection of the creative arts, neuroscience, engineering, media, technology, industry, education, design, forensics, and medicine. The contributions reviewed the state of the art, identified challenges and opportunities and created a roadmap for the field of graphonomics and your brain on art. The topics addressed include: integrative strategies for understanding neural, affective and cognitive systems in realistic, complex environments; neural and behavioral individuality and variation; neuroaesthetics (the use of neuroscience to explain and understand the aesthetic experiences at the neurological level); creativity and innovation; neuroengineering and brain-inspired art, creative concepts and wearable mobile brain-body imaging (MoBI) designs; creative art therapy; informal learning; education; forensics