StyleGAN Encoder-Based Attack for Block Scrambled Face Images

In this paper, we propose an attack method to block scrambled face images, particularly Encryption-then-Compression (EtC) applied images by utilizing the existing powerful StyleGAN encoder and decoder for the first time. Instead of reconstructing identical images as plain ones from encrypted images, we focus on recovering styles that can reveal identifiable information from the encrypted images. The proposed method trains an encoder by using plain and encrypted image pairs with a particular training strategy. While state-of-the-art attack methods cannot recover any perceptual information from EtC images, the proposed method discloses personally identifiable information such as hair color, skin color, eyeglasses, gender, etc. Experiments were carried out on the CelebA dataset, and results show that reconstructed images have some perceptual similarities compared to plain images.Comment: To appear in APSIPA ASC 202

Combined Use of Federated Learning and Image Encryption for Privacy-Preserving Image Classification with Vision Transformer

In recent years, privacy-preserving methods for deep learning have become an urgent problem. Accordingly, we propose the combined use of federated learning (FL) and encrypted images for privacy-preserving image classification under the use of the vision transformer (ViT). The proposed method allows us not only to train models over multiple participants without directly sharing their raw data but to also protect the privacy of test (query) images for the first time. In addition, it can also maintain the same accuracy as normally trained models. In an experiment, the proposed method was demonstrated to well work without any performance degradation on the CIFAR-10 and CIFAR-100 datasets

Easy Encryption for Email, Photo, and Other Cloud Services

Modern users carry mobile devices with them at nearly all times, and this likely has contributed to the rapid growth of private user data—such as emails, photos, and more—stored online in the cloud. Unfortunately, the security of many cloud services for user data is lacking, and the vast amount of user data stored in the cloud is an attractive target for adversaries. Even a single compromise of a user’s account yields all its data to attackers. A breach of an unencrypted email account gives the attacker full access to years, even decades, of emails. Ideally, users would encrypt their data to prevent this. However, encrypting data at rest has long been considered too difficult for users, even technical ones, mainly due to the confusing nature of managing cryptographic keys. My thesis is that strong security can be made easy to use through client-side encryption using self-generated per-device cryptographic keys, such that user data in cloud services is well protected, encryption is transparent and largely unnoticeable to users even on multiple devices, and encryption can be used with existing services without any server-side modifications. This dissertation introduces a new paradigm for usable cryptographic key management, Per-Device Keys (PDK), and explores how self-generated keys unique to every device can enable new client-side encryption schemes that are compatible with existing online services yet are transparent to users. PDK’s design based on self-generated keys allows them to stay on each device and never leave them. Management of these self-generated keys can be shown to users as a device management abstraction which looks like pairing devices with each other, and not any form of cryptographic key management. I design, implement, and evaluate three client-side encryption schemes supported by PDK, with a focus on designing around usability to bring transparent encryption to users. First, I introduce Easy Email Encryption (E3), a secure email solution that is easy to use. Usersstruggle with using end-to-end encrypted email, such as PGP and S/MIME, because it requires users to understand cryptographic key exchanges to send encrypted emails. E3 eliminates this key exchange by focusing on storing encrypting emails instead of sending them. E3 transparently encrypts emails on receipt, ensuring that all emails received before a compromise are protected from attack, and relies on widely-used TLS connections to protect in-flight emails. Emails are encrypted using self-generated keys, which are completely hidden from the user and do not need to be exchanged with other users, alleviating the burden of users having to know how to use and manage them. E3 encrypts on the client, making it easy to deploy because it requires no server or protocol changes and is compatible with any existing email service. Experimental results show that E3 is compatible with existing IMAP email services, including Gmail and Yahoo!, and has good performance for common email operations. Results of a user study show that E3 provides much stronger security guarantees than current practice yet is much easier to use than end-to-end encrypted email such as PGP. Second, I introduce Easy Secure Photos (ESP), an easy-to-use system that enables photos tobe encrypted and stored using existing cloud photo services. Users cannot store encrypted photos in services like Google Photos because these services only allow users to upload valid images such as JPEG images, but typical encryption methods do not retain image file formats for the encrypted versions and are not compatible with image processing such as image compression. ESP introduces a new image encryption technique that outputs valid encrypted JPEG files which are accepted by cloud photo services, and are robust against compression. The photos are encrypted using self-generated keys before being uploaded to cloud photo services, and are decrypted when downloaded to users’ devices. Similar to E3, ESP hides all the details of encryption/decryption and key management from the user. Since all crypto operations happen in the user’s photo app, ESP requires no changes to existing cloud photo services, making it easy to deploy. Experimental results and user studies show that ESP encryption is robust against attack techniques, exhibits acceptable performance overheads, and is simple for users to set up and use. Third, I introduce Easy Device-based Passwords (EDP), a password manager with improvedsecurity guarantees over existing ones while maintaining their familiar usage models. To encrypt and decrypt user passwords, existing password managers rely on weak, human-generated master passwords which are easy to use but easily broken. EDP introduces a new approach using self-generated keys to encrypt passwords, and an easy-to-use pairing mechanism to allow users to access passwords across multiple devices. Keys are not exposed to users and users do not need to know anything about key management. EDP is the first password manager that secures passwords even with untrusted servers, protecting against server break-ins and password database leaks. Experimental results and a user study show that EDP ensures password security with untrusted servers and infrastructure, has comparable performance to existing password managers, and is considered usable by users

Privacy-Friendly Photo Sharing and Relevant Applications Beyond

Popularization of online photo sharing brings people great convenience, but has also raised concerns for privacy. Researchers proposed various approaches to enable image privacy, most of which focus on encrypting or distorting image visual content. In this thesis, we investigate novel solutions to protect image privacy with a particular emphasis on online photo sharing. To this end, we propose not only algorithms to protect visual privacy in image content but also design of architectures for privacy-preserving photo sharing. Beyond privacy, we also explore additional impacts and potentials of employing daily images in other three relevant applications. First, we propose and study two image encoding algorithms to protect visual content in image, within a Secure JPEG framework. The first method scrambles a JPEG image by randomly changing the signs of its DCT coefficients based on a secret key. The second method, named JPEG Transmorphing, allows one to protect arbitrary image regions with any obfuscation, while secretly preserving the original image regions in application segments of the obfuscated JPEG image. Performance evaluations reveal a good degree of storage overhead and privacy protection capability for both methods, and particularly a good level of pleasantness for JPEG Transmorphing, if proper manipulations are applied. Second, we investigate the design of two architectures for privacy-preserving photo sharing. The first architecture, named ProShare, is built on a public key infrastructure (PKI) integrated with a ciphertext-policy attribute-based encryption (CP-ABE), to enable the secure and efficient access to user-posted photos protected by Secure JPEG. The second architecture is named ProShare S, in which a photo sharing service provider helps users make photo sharing decisions automatically based on their past decisions using machine learning. The photo sharing service analyzes not only the content of a user's photo, but also context information about the image capture and a prospective requester, and finally makes decision whether or not to share a particular photo to the requester, and if yes, at which granularity. A user study along with extensive evaluations were performed to validate the proposed architecture. In the end, we research into three relevant topics in regard to daily photos captured or shared by people, but beyond their privacy implications. In the first study, inspired by JPEG Transmorphing, we propose an animated JPEG file format, named aJPEG. aJPEG preserves its animation frames as application markers in a JPEG image and provides smaller file size and better image quality than conventional GIF. In the second study, we attempt to understand the impact of popular image manipulations applied in online photo sharing on evoked emotions of observers. The study reveals that image manipulations indeed influence people's emotion, but such impact also depends on the image content. In the last study, we employ a deep convolutional neural network (CNN), the GoogLeNet model, to perform automatic food image detection and categorization. The promising results obtained provide meaningful insights in design of automatic dietary assessment system based on multimedia techniques, e.g. image analysis