Rotation-invariant binary representation of sensor pattern noise for source-oriented image and video clustering

Most existing source-oriented image and video clustering algorithms based on sensor pattern noise (SPN) rely on the pairwise similarities, whose calculation usually dominates the overall computational time. The heavy computational burden is mainly incurred by the high dimensionality of SPN, which typically goes up to millions for delivering plausible clustering performance. This problem can be further aggravated by the uncertainty of the orientation of images or videos because the spatial correspondence between data with uncertain orientations needs to be reestablished in a brute-force search manner. In this work, we propose a rotation-invariant binary representation of SPN to address the issue of rotation and reduce the computational cost of calculating the pairwise similarities. Results on two public multimedia forensics databases have shown that the proposed approach is effective in overcoming the rotation issue and speeding up the calculation of pairwise SPN similarities for source-oriented image and video clustering

Provenance analysis for instagram photos

As a feasible device fingerprint, sensor pattern noise (SPN) has been proven to be effective in the provenance analysis of digital images. However, with the rise of social media, millions of images are being uploaded to and shared through social media sites every day. An image downloaded from social networks may have gone through a series of unknown image manipulations. Consequently, the trustworthiness of SPN has been challenged in the provenance analysis of the images downloaded from social media platforms. In this paper, we intend to investigate the effects of the pre-defined Instagram images filters on the SPN-based image provenance analysis. We identify two groups of filters that affect the SPN in quite different ways, with Group I consisting of the filters that severely attenuate the SPN and Group II consisting of the filters that well preserve the SPN in the images. We further propose a CNN-based classifier to perform filter-oriented image categorization, aiming to exclude the images manipulated by the filters in Group I and thus improve the reliability of the SPN-based provenance analysis. The results on about 20, 000 images and 18 filters are very promising, with an accuracy higher than 96% in differentiating the filters in Group I and Group II

User profiles’ image clustering for digital investigations

Sharing images on Social Network (SN) platforms is one of the most widespread behaviors which may cause privacy-intrusive and illegal content to be widely distributed. Clustering the images shared through SN platforms according to the acquisition cameras embedded in smartphones is regarded as a significant task in forensic investigations of cybercrimes. The Sensor Pattern Noise (SPN) caused by camera sensor imperfections due to the manufacturing process has been proved to be an effective and robust camera fingerprint that can be used for several tasks, such as digital evidence analysis, smartphone fingerprinting and user profile linking as well. Clustering the images uploaded by users on their profiles is a way of fingerprinting the camera sources and it is considered a challenging task since users may upload different types of images, i.e., the images taken by users’ smartphones (taken images) and single images from different sources, cropped images, or generic images from the Web (shared images). The shared images make a perturbation in the clustering task, as they do not usually present sufficient characteristics of SPN of their related sources. Moreover, they are not directly referable to the user’s device so they have to be detected and removed from the clustering process. In this paper, we propose a user profiles’ image clustering method without prior knowledge about the type and number of the camera sources. The hierarchical graph-based method clusters both types of images, taken images and shared images. The strengths of our method include overcoming large-scale image datasets, the presence of shared images that perturb the clustering process and the loss of image details caused by the process of content compression on SN platforms. The method is evaluated on the VISION dataset, which is a public benchmark including images from 35 smartphones. The dataset is perturbed by 3000 images, simulating the shared images from different sources except for users’ smartphones. Experimental results confirm the robustness of the proposed method against perturbed datasets and its effectiveness in the image clustering

Classification and Clustering of Shared Images on Social Networks and User Profile Linking

The ever increasing prevalence of smartphones and the popularity of social network platforms have facilitated instant sharing of multimedia content through social networks. However, the ease in taking and sharing photos and videos through social networks also allows privacy-intrusive and illegal content to be widely distributed. As such, images captured and shared by users on their profiles are considered as significant digital evidence for social network data analysis. The Sensor Pattern Noise (SPN) caused by camera sensor imperfections during the manufacturing process mainly consists of the Photo-Response Non-Uniformity (PRNU) noise that can be extracted from taken images without hacking the device. It has been proven to be an effective and robust device fingerprint that can be used for different important digital image forensic tasks, such as image forgery detection, source device identification and device linking. Particularly, by fingerprinting the camera sources captured a set of shared images on social networks, User Profile Linking (UPL) can be performed on social network platforms. The aim of this thesis is to present effective and robust methods and algorithms for better fulfilling shared image analysis based on SPN. We propose clustering and classification based methods to achieve Smartphone Identification (SI) and UPL tasks, given a set of images captured by a known number of smartphones and shared on a set of known user profiles. The important outcome of the proposed methods is UPL across different social networks where the clustered images from one social network are applied to fingerprint the related smartphones and link user profiles on the other social network. Also, we propose two methods for large-scale image clustering of different types of the shared images by users, without prior knowledge about the types and number of the smartphones

Microphone smart device fingerprinting from video recordings

This report aims at summarizing the on-going research activity carried out by DG-JRC in the framework of the institutional project Authors and Victims Identification of Child Abuse on-line, concerning the use of microphone fingerprinting for source device classification. Starting from an exhaustive study of the State of Art regarding the matter, this report describes a feasibility study about the adoption of microphone fingerprinting for source identification of video recordings. A set of operational scenarios have been established in collaboration with EUROPOL law enforcers, according to investigators needs. A critical analysis of the obtained results has demonstrated the feasibility of microphone fingerprinting and it has suggested a set of recommendations, both in terms of usability and future researches in the field.JRC.E.3-Cyber and Digital Citizens' Securit

Comparative analysis of computer-vision and BLE technology based indoor navigation systems for people with visual impairments

Background: Considerable number of indoor navigation systems has been proposed to augment people with visual impairments (VI) about their surroundings. These systems leverage several technologies, such as computer-vision, Bluetooth low energy (BLE), and other techniques to estimate the position of a user in indoor areas. Computer-vision based systems use several techniques including matching pictures, classifying captured images, recognizing visual objects or visual markers. BLE based system utilizes BLE beacons attached in the indoor areas as the source of the radio frequency signal to localize the position of the user. Methods: In this paper, we examine the performance and usability of two computer-vision based systems and BLE-based system. The first system is computer-vision based system, called CamNav that uses a trained deep learning model to recognize locations, and the second system, called QRNav, that utilizes visual markers (QR codes) to determine locations. A field test with 10 blindfolded users has been conducted while using the three navigation systems. Results: The obtained results from navigation experiment and feedback from blindfolded users show that QRNav and CamNav system is more efficient than BLE based system in terms of accuracy and usability. The error occurred in BLE based application is more than 30% compared to computer vision based systems including CamNav and QRNav. Conclusions: The developed navigation systems are able to provide reliable assistance for the participants during real time experiments. Some of the participants took minimal external assistance while moving through the junctions in the corridor areas. Computer vision technology demonstrated its superiority over BLE technology in assistive systems for people with visual impairments. - 2019 The Author(s).Scopu

Are Social Networks Watermarking Us or Are We (Unawarely) Watermarking Ourself?

In the last decade, Social Networks (SNs) have deeply changed many aspects of society, and one of the most widespread behaviours is the sharing of pictures. However, malicious users often exploit shared pictures to create fake profiles, leading to the growth of cybercrime. Thus, keeping in mind this scenario, authorship attribution and verification through image watermarking techniques are becoming more and more important. In this paper, we firstly investigate how thirteen of the most popular SNs treat uploaded pictures in order to identify a possible implementation of image watermarking techniques by respective SNs. Second, we test the robustness of several image watermarking algorithms on these thirteen SNs. Finally, we verify whether a method based on the Photo-Response Non-Uniformity (PRNU) technique, which is usually used in digital forensic or image forgery detection activities, can be successfully used as a watermarking approach for authorship attribution and verification of pictures on SNs. The proposed method is sufficiently robust, in spite of the fact that pictures are often downgraded during the process of uploading to the SNs. Moreover, in comparison to conventional watermarking methods the proposed method can successfully pass through different SNs, solving related problems such as profile linking and fake profile detection. The results of our analysis on a real dataset of 8400 pictures show that the proposed method is more effective than other watermarking techniques and can help to address serious questions about privacy and security on SNs. Moreover, the proposed method paves the way for the definition of multi-factor online authentication mechanisms based on robust digital features