Rethinking Image Forgery Detection via Contrastive Learning and Unsupervised Clustering

Image forgery detection aims to detect and locate forged regions in an image. Most existing forgery detection algorithms formulate classification problems to classify pixels into forged or pristine. However, the definition of forged and pristine pixels is only relative within one single image, e.g., a forged region in image A is actually a pristine one in its source image B (splicing forgery). Such a relative definition has been severely overlooked by existing methods, which unnecessarily mix forged (pristine) regions across different images into the same category. To resolve this dilemma, we propose the FOrensic ContrAstive cLustering (FOCAL) method, a novel, simple yet very effective paradigm based on contrastive learning and unsupervised clustering for the image forgery detection. Specifically, FOCAL 1) utilizes pixel-level contrastive learning to supervise the high-level forensic feature extraction in an image-by-image manner, explicitly reflecting the above relative definition; 2) employs an on-the-fly unsupervised clustering algorithm (instead of a trained one) to cluster the learned features into forged/pristine categories, further suppressing the cross-image influence from training data; and 3) allows to further boost the detection performance via simple feature-level concatenation without the need of retraining. Extensive experimental results over six public testing datasets demonstrate that our proposed FOCAL significantly outperforms the state-of-the-art competing algorithms by big margins: +24.3% on Coverage, +18.6% on Columbia, +17.5% on FF++, +14.2% on MISD, +13.5% on CASIA and +10.3% on NIST in terms of IoU. The paradigm of FOCAL could bring fresh insights and serve as a novel benchmark for the image forgery detection task. The code is available at https://github.com/HighwayWu/FOCAL

Generalizable Synthetic Image Detection via Language-guided Contrastive Learning

The heightened realism of AI-generated images can be attributed to the rapid development of synthetic models, including generative adversarial networks (GANs) and diffusion models (DMs). The malevolent use of synthetic images, such as the dissemination of fake news or the creation of fake profiles, however, raises significant concerns regarding the authenticity of images. Though many forensic algorithms have been developed for detecting synthetic images, their performance, especially the generalization capability, is still far from being adequate to cope with the increasing number of synthetic models. In this work, we propose a simple yet very effective synthetic image detection method via a language-guided contrastive learning and a new formulation of the detection problem. We first augment the training images with carefully-designed textual labels, enabling us to use a joint image-text contrastive learning for the forensic feature extraction. In addition, we formulate the synthetic image detection as an identification problem, which is vastly different from the traditional classification-based approaches. It is shown that our proposed LanguAge-guided SynThEsis Detection (LASTED) model achieves much improved generalizability to unseen image generation models and delivers promising performance that far exceeds state-of-the-art competitors by +22.66% accuracy and +15.24% AUC. The code is available at https://github.com/HighwayWu/LASTED

Structual variation detection in the human genome

Thesis advisor: Gabor T. MarthStructural variations (SVs), like single nucleotide polymorphisms (SNPs) and short insertion-deletion polymorphisms (INDELs), are a ubiquitous feature of genomic sequences and are major contributors to human genetic diversity and disease. Due to technical difficulties, i.e. the high data-acquisition cost and/or low detection resolution of previous genome-scanning technologies, this source of genetic variation has not been well studied until the completion of the Human Genome Project and the emergence of next-generation sequencing (NGS) technologies. The assembly of the human genome and economical high-throughput sequencing technologies enable the development of numerous new SV detection algorithms with unprecedented accuracy, sensitivity and precision. Although a number of SV detection programs have been developed for various SV types, such as copy number variations, deletions, tandem duplications, inversions and translocations, some types of SVs, e.g. copy number variations (CNVs) in capture sequencing data and mobile element insertions (MEIs) have undergone limited study. This is a result of the lack of suitable statistical models and computational approaches, e.g. efficient mapping method to handle multiple aligned reads from mobile element (ME) sequences. The focus of my dissertation was to identify and characterize CNVs in capture sequencing data and MEI from large-scale whole-genome sequencing data. This was achieved by building sophisticated statistical models and developing efficient algorithms and analysis methods for NGS data. In Chapter 2, I present a novel algorithm that uses the read depth (RD) signal to detect CNVs in deep-coverage exon capture sequencing data that are originally designed for SNPs discovery. We were one of the early pioneers to tackle this problem. In Chapter 3, I present a fast, convenient and memory-efficient program, Tangram, that integrates read-pair (RP) and split-read (SR) signals to detect and genotype MEI events. Based on the results from both simulated and experimental data, Tangram has superior sensitivity, specificity, breakpoint resolution and genotyping accuracy, when compared to other recently published MEI detection methods. Lastly, Chapter 4 summarizes my work for SV detection in human genomes during my PhD study and describes the future direction of genetic variant researches.Thesis (PhD) — Boston College, 2013.Submitted to: Boston College. Graduate School of Arts and Sciences.Discipline: Biology

RESEARCH ON THE ASEISMIC BEHAVIOR OF LONG-SPAN CABLE-STAYED BRIDGE WITH DAMPING EFFECT

The main beam of a cable-stayed bridge with a floating system may have a larger longitudinal displacement subject to earthquake effect. Thus, seismic control and isolation are crucial to bridge safety. This paper takes Huai’an Bridge, which has elastic coupling devices and viscous dampers set at the joint of the tower and the beam, as the research background. Its finite element model is established, and the elastic stiffness of elastic coupling devices and damper parameters are analyzed. Viscous damper and elastic coupling devices are simulated using Maxwell model and spring elements, and their damping effects are analyzed and compared through structural dynamic time-history analysis. Results show that viscous damper and elastic coupling device furnished at the joint of tower and beam of a cable-stayed bridge tower beam can effectively reduce the longitudinal displacement of the key part of the construction subject to earthquake effect, perfect the internal force distribution, and improve the aseismic performance. Between the two, viscous damper has better damping effects

Multi-Interval Rolling-Window Joint Dispatch and Pricing of Energy and Reserve under Uncertainty

In this paper, the intra-day multi-interval rolling-window joint dispatch and pricing of energy and reserve is studied under increasing volatile and uncertain renewable generations. A look-ahead energy-reserve co-optimization model is proposed for the rolling-window dispatch, where possible contingencies and load/renewable forecast errors over the look-ahead window are modeled as several scenario trajectories, while generation, especially its ramp, is jointly scheduled with reserve to minimize the expected system cost considering these scenarios. Based on the proposed model, marginal prices of energy and reserve are derived, which incorporate shadow prices of generators' individual ramping capability limits to eliminate their possible ramping-induced opportunity costs or arbitrages. We prove that under mild conditions, the proposed market design provides dispatch-following incentives to generators without the need for out-of-the-market uplifts, and truthful-bidding incentives of price-taking generators can be guaranteed as well. Some discussions are also made on how to fit the proposed framework into current market practice. These findings are validated in numerical simulations

Generating Robust Adversarial Examples against Online Social Networks (OSNs)

Online Social Networks (OSNs) have blossomed into prevailing transmission channels for images in the modern era. Adversarial examples (AEs) deliberately designed to mislead deep neural networks (DNNs) are found to be fragile against the inevitable lossy operations conducted by OSNs. As a result, the AEs would lose their attack capabilities after being transmitted over OSNs. In this work, we aim to design a new framework for generating robust AEs that can survive the OSN transmission; namely, the AEs before and after the OSN transmission both possess strong attack capabilities. To this end, we first propose a differentiable network termed SImulated OSN (SIO) to simulate the various operations conducted by an OSN. Specifically, the SIO network consists of two modules: 1) a differentiable JPEG layer for approximating the ubiquitous JPEG compression and 2) an encoder-decoder subnetwork for mimicking the remaining operations. Based upon the SIO network, we then formulate an optimization framework to generate robust AEs by enforcing model outputs with and without passing through the SIO to be both misled. Extensive experiments conducted over Facebook, WeChat and QQ demonstrate that our attack methods produce more robust AEs than existing approaches, especially under small distortion constraints; the performance gain in terms of Attack Success Rate (ASR) could be more than 60%. Furthermore, we build a public dataset containing more than 10,000 pairs of AEs processed by Facebook, WeChat or QQ, facilitating future research in the robust AEs generation. The dataset and code are available at https://github.com/csjunjun/RobustOSNAttack.git.Comment: 26 pages, 9 figure

Distributed Spectrum and Power Allocation for D2D-U Networks: A Scheme based on NN and Federated Learning

In this paper, a Device-to-Device communication on unlicensed bands (D2D-U) enabled network is studied. To improve the spectrum efficiency (SE) on the unlicensed bands and fit its distributed structure while ensuring the fairness among D2D-U links and the harmonious coexistence with WiFi networks, a distributed joint power and spectrum scheme is proposed. In particular, a parameter, named as price, is defined, which is updated at each D2D-U pair by a online trained Neural network (NN) according to the channel state and traffic load. In addition, the parameters used in the NN are updated by two ways, unsupervised self-iteration and federated learning, to guarantee the fairness and harmonious coexistence. Then, a non-convex optimization problem with respect to the spectrum and power is formulated and solved on each D2D-U link to maximize its own data rate. Numerical simulation results are demonstrated to verify the effectiveness of the proposed scheme