38 research outputs found
On Face Segmentation, Face Swapping, and Face Perception
We show that even when face images are unconstrained and arbitrarily paired,
face swapping between them is actually quite simple. To this end, we make the
following contributions. (a) Instead of tailoring systems for face
segmentation, as others previously proposed, we show that a standard fully
convolutional network (FCN) can achieve remarkably fast and accurate
segmentations, provided that it is trained on a rich enough example set. For
this purpose, we describe novel data collection and generation routines which
provide challenging segmented face examples. (b) We use our segmentations to
enable robust face swapping under unprecedented conditions. (c) Unlike previous
work, our swapping is robust enough to allow for extensive quantitative tests.
To this end, we use the Labeled Faces in the Wild (LFW) benchmark and measure
the effect of intra- and inter-subject face swapping on recognition. We show
that our intra-subject swapped faces remain as recognizable as their sources,
testifying to the effectiveness of our method. In line with well known
perceptual studies, we show that better face swapping produces less
recognizable inter-subject results. This is the first time this effect was
quantitatively demonstrated for machine vision systems
AFFECT-PRESERVING VISUAL PRIVACY PROTECTION
The prevalence of wireless networks and the convenience of mobile cameras enable many new video applications other than security and entertainment. From behavioral diagnosis to wellness monitoring, cameras are increasing used for observations in various educational and medical settings. Videos collected for such applications are considered protected health information under privacy laws in many countries. Visual privacy protection techniques, such as blurring or object removal, can be used to mitigate privacy concern, but they also obliterate important visual cues of affect and social behaviors that are crucial for the target applications. In this dissertation, we propose to balance the privacy protection and the utility of the data by preserving the privacy-insensitive information, such as pose and expression, which is useful in many applications involving visual understanding.
The Intellectual Merits of the dissertation include a novel framework for visual privacy protection by manipulating facial image and body shape of individuals, which: (1) is able to conceal the identity of individuals; (2) provide a way to preserve the utility of the data, such as expression and pose information; (3) balance the utility of the data and capacity of the privacy protection.
The Broader Impacts of the dissertation focus on the significance of privacy protection on visual data, and the inadequacy of current privacy enhancing technologies in preserving affect and behavioral attributes of the visual content, which are highly useful for behavior observation in educational and medical settings. This work in this dissertation represents one of the first attempts in achieving both goals simultaneously
Visual Content Privacy Protection: A Survey
Vision is the most important sense for people, and it is also one of the main
ways of cognition. As a result, people tend to utilize visual content to
capture and share their life experiences, which greatly facilitates the
transfer of information. Meanwhile, it also increases the risk of privacy
violations, e.g., an image or video can reveal different kinds of
privacy-sensitive information. Researchers have been working continuously to
develop targeted privacy protection solutions, and there are several surveys to
summarize them from certain perspectives. However, these surveys are either
problem-driven, scenario-specific, or technology-specific, making it difficult
for them to summarize the existing solutions in a macroscopic way. In this
survey, a framework that encompasses various concerns and solutions for visual
privacy is proposed, which allows for a macro understanding of privacy concerns
from a comprehensive level. It is based on the fact that privacy concerns have
corresponding adversaries, and divides privacy protection into three
categories, based on computer vision (CV) adversary, based on human vision (HV)
adversary, and based on CV \& HV adversary. For each category, we analyze the
characteristics of the main approaches to privacy protection, and then
systematically review representative solutions. Open challenges and future
directions for visual privacy protection are also discussed.Comment: 24 pages, 13 figure