A Survey of Deep Learning Solutions for Multimedia Visual Content Analysis

The increasing use of social media networks on handheld devices, especially smartphones with powerful built-in cameras, and the widespread availability of fast and high bandwidth broadband connections, added to the popularity of cloud storage, is enabling the generation and distribution of massive volumes of digital media, including images and videos. Such media is full of visual information and holds immense value in today's world. The volume of data involved calls for automated visual content analysis systems able to meet the demands of practice in terms of efficiency and effectiveness. Deep learning (DL) has recently emerged as a prominent technique for visual content analysis. It is data-driven in nature and provides automatic end-to-end learning solutions without the need to rely explicitly on predefined handcrafted feature extractors. Another appealing characteristic of DL solutions is the performance they can achieve, once the network is trained, under practical constraints. This paper identifies eight problem domains which require analysis of visual artifacts in multimedia. It surveys the recent, authoritative, and the best performing DL solutions and lists the datasets used in the development of these deep methods for the identified types of visual analysis problems. This paper also discusses the challenges that the DL solutions face which can compromise their reliability, robustness, and accuracy for visual content analysis

WVD: A New Synthetic Dataset for Video-based Violence Detection

Violence detection is becoming increasingly relevant in many areas such as for automatic content filtering, video surveillance and law enforcement. Existing datasets and methods discriminate between violent and non-violent scenes based on very abstract definitions of violence. Available datasets, such as "Hockey Fight" and "Movies", only contain fight versus non-fight videos; no weapons are discriminated in them. In this paper, we focus explicitly on weapon-based fighting sequences and propose a new dataset based on the popular action-adventure video game Grand Theft Auto-V (GTA-V). This new dataset is called "Weapon Violence Dataset" (WVD). The choice for a virtual dataset follows a trend which allows creating and labelling as sophisticated and large volume, yet realistic, datasets as possible. Furthermore, WVD also avoids the drawbacks of access to real data and potential implications. To the best of our knowledge no similar dataset, that captures weapon-based violence, exists. The paper evaluates the proposed dataset by utilising local feature descriptors using an SVM classifier. The extracted features are aggregated using the Bag of Visual Word (BoVW) technique to classify weapon-based violence videos. Our results indicate that SURF achieves the best performance

Weapon Violence Dataset 2.0: A synthetic dataset for violence detection

In the current era, satisfying the appetite of data hungry models is becoming an increasingly challenging task. This challenge is particularly magnified in research areas characterised by sensitivity, where the quest for genuine data proves to be elusive. The study of violence serves as a poignant example, entailing ethical considerations and compounded by the scarcity of authentic, real-world data that is predominantly accessible only to law enforcement agencies. Existing datasets in this field often resort to using content from movies or open-source video platforms like YouTube, further emphasising the scarcity of authentic data. To address this, our dataset aims to pioneer a new approach by creating the first synthetic virtual dataset for violence detection, named the Weapon Violence Dataset (WVD). The dataset is generated by creating virtual violence scenarios inside the photo-realistic video game namely: Grand Theft Auto-V (GTA-V). This dataset includes carefully selected video clips of person-to-person fights captured from a frontal view, featuring various weapons—both hot and cold across different times of the day. Specifically, WVD contains three categories: Hot violence and Cold violence (representing the violence category) as well as No violence (constituting the control class). The dataset is designed and created in a way that will enable the research community to train deep models on such synthetic data with the ability to increase the data corpus if the needs arise. The dataset is publicly available on Kaggle and comprises normal RGB and optic flow videos

Deep labeller: automatic bounding box generation for synthetic violence detection datasets

Manually labelling datasets for training violence detection systems is time-consuming, expensive, and labor-intensive. Mind wandering, boredom, and short attention span can also cause labelling errors. Moreover, collecting and distributing sensitive images containing violence has ethical implications. Automation is the future for labelling sensitive image datasets. Deep labeller is a two-stage Deep Learning (DL) method that uses pre-trained DL object detection methods on MS-COCO for automatic labelling. The Deep Labeller method labels violent and nonviolent images in WVD and USI. In stage 1, WVD generates weak labels using synthetic images. In stage 2, the Deep labeller method is retrained on weak labels. USI dataset is used to test our method on real-world violence. Deep labeller generated weak and strong labels with an IoU of 0.80036 in stage 1 and 0.95 in stage 2 on the WVD. Automatically generated labels. To test our method’s generalisation power, violent and nonviolent image labels on USI dataset had a mean IoU of 0.7450

Privacy Verification of PhotoDNA Based on Machine Learning

PhotoDNA is a perceptual fuzzy hash technology designed and developed by Microsoft. It is deployed by all major big data service providers to detect Indecent Images of Children (IIOC). Protecting the privacy of individuals is of paramount importance in such images. Microsoft claims that a PhotoDNA hash cannot be reverse engineered into the original image; therefore, it is not possible to identify individuals or objects depicted in the image. In this chapter, we evaluate the privacy protection capability of PhotoDNA by testing it against machine learning. Specifically, our aim is to detect the presence of any structural information that might be utilized to compromise the privacy of the individuals via classification. Due to the widespread usage of PhotoDNA as a deterrent to IIOC by big data companies, ensuring its ability to protect privacy would be crucial. In our experimentation, we achieved a classification accuracy of 57.20%. This result indicates that PhotoDNA is resistant to machine-learning-based classification attacks

COVID-19 digital contact tracing applications and techniques: A review post initial deployments

The coronavirus disease 2019 (COVID-19) is a severe global pandemic that has claimed millions of lives and continues to overwhelm public health systems in many countries. The spread of COVID-19 pandemic has negatively impacted the human mobility patterns such as daily transportation-related behavior of the public. There is a requirement to understand the disease spread patterns and its routes among neighboring individuals for the timely implementation of corrective measures at the required placement. To increase the effectiveness of contact tracing, countries across the globe are leveraging advancements in mobile technology and Internet of Things (IoT) to aid traditional manual contact tracing to track individuals who have come in close contact with identified COVID-19 patients. Even as the first administration of vaccines begins in 2021, the COVID-19 management strategy will continue to be multi-pronged for the foreseeable future with digital contact tracing being a vital component of the response along with the use of preventive measures such as social distancing and the use of face masks. After some months of deployment of digital contact tracing technology, deeper insights into the merits of various approaches and the usability, privacy, and ethical trade-offs involved are emerging. In this paper, we provide a comprehensive analysis of digital contact tracing solutions in terms of their methodologies and technologies in the light of the new data emerging about international experiences of deployments of digital contact tracing technology. We also provide a discussion on open challenges such as scalability, privacy, adaptability and highlight promising directions for future work