Exploiting Low-confidence Pseudo-labels for Source-free Object Detection

Source-free object detection (SFOD) aims to adapt a source-trained detector to an unlabeled target domain without access to the labeled source data. Current SFOD methods utilize a threshold-based pseudo-label approach in the adaptation phase, which is typically limited to high-confidence pseudo-labels and results in a loss of information. To address this issue, we propose a new approach to take full advantage of pseudo-labels by introducing high and low confidence thresholds. Specifically, the pseudo-labels with confidence scores above the high threshold are used conventionally, while those between the low and high thresholds are exploited using the Low-confidence Pseudo-labels Utilization (LPU) module. The LPU module consists of Proposal Soft Training (PST) and Local Spatial Contrastive Learning (LSCL). PST generates soft labels of proposals for soft training, which can mitigate the label mismatch problem. LSCL exploits the local spatial relationship of proposals to improve the model's ability to differentiate between spatially adjacent proposals, thereby optimizing representational features further. Combining the two components overcomes the challenges faced by traditional methods in utilizing low-confidence pseudo-labels. Extensive experiments on five cross-domain object detection benchmarks demonstrate that our proposed method outperforms the previous SFOD methods, achieving state-of-the-art performance

SimpleNet: A Simple Network for Image Anomaly Detection and Localization

We propose a simple and application-friendly network (called SimpleNet) for detecting and localizing anomalies. SimpleNet consists of four components: (1) a pre-trained Feature Extractor that generates local features, (2) a shallow Feature Adapter that transfers local features towards target domain, (3) a simple Anomaly Feature Generator that counterfeits anomaly features by adding Gaussian noise to normal features, and (4) a binary Anomaly Discriminator that distinguishes anomaly features from normal features. During inference, the Anomaly Feature Generator would be discarded. Our approach is based on three intuitions. First, transforming pre-trained features to target-oriented features helps avoid domain bias. Second, generating synthetic anomalies in feature space is more effective, as defects may not have much commonality in the image space. Third, a simple discriminator is much efficient and practical. In spite of simplicity, SimpleNet outperforms previous methods quantitatively and qualitatively. On the MVTec AD benchmark, SimpleNet achieves an anomaly detection AUROC of 99.6%, reducing the error by 55.5% compared to the next best performing model. Furthermore, SimpleNet is faster than existing methods, with a high frame rate of 77 FPS on a 3080ti GPU. Additionally, SimpleNet demonstrates significant improvements in performance on the One-Class Novelty Detection task. Code: https://github.com/DonaldRR/SimpleNet.Comment: Accepted to CVPR 202

AdaptGuard: Defending Against Universal Attacks for Model Adaptation

Model adaptation aims at solving the domain transfer problem under the constraint of only accessing the pretrained source models. With the increasing considerations of data privacy and transmission efficiency, this paradigm has been gaining recent popularity. This paper studies the vulnerability to universal attacks transferred from the source domain during model adaptation algorithms due to the existence of malicious providers. We explore both universal adversarial perturbations and backdoor attacks as loopholes on the source side and discover that they still survive in the target models after adaptation. To address this issue, we propose a model preprocessing framework, named AdaptGuard, to improve the security of model adaptation algorithms. AdaptGuard avoids direct use of the risky source parameters through knowledge distillation and utilizes the pseudo adversarial samples under adjusted radius to enhance the robustness. AdaptGuard is a plug-and-play module that requires neither robust pretrained models nor any changes for the following model adaptation algorithms. Extensive results on three commonly used datasets and two popular adaptation methods validate that AdaptGuard can effectively defend against universal attacks and maintain clean accuracy in the target domain simultaneously. We hope this research will shed light on the safety and robustness of transfer learning. Code is available at https://github.com/TomSheng21/AdaptGuard.Comment: ICCV202

Improving Zero-Shot Generalization for CLIP with Synthesized Prompts

With the growing interest in pretrained vision-language models like CLIP, recent research has focused on adapting these models to downstream tasks. Despite achieving promising results, most existing methods require labeled data for all classes, which may not hold in real-world applications due to the long tail and Zipf's law. For example, some classes may lack labeled data entirely, such as emerging concepts. To address this problem, we propose a plug-and-play generative approach called \textbf{S}ynt\textbf{H}es\textbf{I}zed \textbf{P}rompts~(\textbf{SHIP}) to improve existing fine-tuning methods. Specifically, we follow variational autoencoders to introduce a generator that reconstructs the visual features by inputting the synthesized prompts and the corresponding class names to the textual encoder of CLIP. In this manner, we easily obtain the synthesized features for the remaining label-only classes. Thereafter, we fine-tune CLIP with off-the-shelf methods by combining labeled and synthesized features. Extensive experiments on base-to-new generalization, cross-dataset transfer learning, and generalized zero-shot learning demonstrate the superiority of our approach. The code is available at \url{https://github.com/mrflogs/SHIP}.Comment: Accepted by ICCV 202

Efficient Characterizations of Multiphoton States with Ultra-thin Integrated Photonics

Metasurface enables the generation and manipulation of multiphoton entanglement with flat optics, providing a more efficient platform for large-scale photonic quantum information processing. Here, we show that a single metasurface optical chip would allow more efficient characterizations of multiphoton entangled states, such as shadow tomography, which generally requires fast and complicated control of optical setups to perform projective measurements in different bases, a demanding task using conventional optics. The compact and stable device here allows implementations of general positive observable value measures with a reduced sample complexity and significantly alleviates the experimental complexity to implement shadow tomography. Integrating self-learning and calibration algorithms, we observe notable advantages in the reconstruction of multiphoton entanglement, including using fewer measurements, having higher accuracy, and being robust against optical loss. Our work unveils the feasibility of metasurface as a favorable integrated optical device for efficient characterization of multiphoton entanglement, and sheds light on scalable photonic quantum technologies with ultra-thin integrated optics.Comment: 15 pages, 9 figure

First Look at z > 1 Bars in the Rest-Frame Near-Infrared with JWST Early CEERS Imaging

Stellar bars are key drivers of secular evolution in galaxies and can be effectively studied using rest-frame near-infrared (NIR) images, which trace the underlying stellar mass and are less impacted by dust and star formation than rest-frame UV or optical images. We leverage the power of {\it{JWST}} CEERS NIRCam images to present the first quantitative identification and characterization of stellar bars at $z>1$ based on rest-frame NIR F444W images of high resolution (~1.3 kpc at z ~ 1-3). We identify stellar bars in these images using quantitative criteria based on ellipse fits. For this pilot study, we present six examples of robustly identified bars at $z>1$ with spectroscopic redshifts, including the two highest redshift bars at ~2.136 and 2.312 quantitatively identified and characterized to date. The stellar bars at $z$ ~ 1.1-2.3 presented in our study have projected semi-major axes of ~2.9-4.3 kpc and projected ellipticities of ~0.41-0.53 in the rest-frame NIR. The barred host galaxies have stellar masses ~ $1 \times 10^{10}$ to $2 \times 10^{11}$ $M_{\odot}$, star formation rates of ~ 21-295 $M_{\odot}$ yr$^{-1}$, and several have potential nearby companions. Our finding of bars at $z$ ~1.1-2.3 demonstrates the early onset of such instabilities and supports simulations where bars form early in massive dynamically cold disks. It also suggests that if these bars at lookback times of 8-10 Gyr survive out to present epochs, bar-driven secular processes may operate over a long time and have a significant impact on some galaxies by z ~ 0.Comment: 16 pages, 5 figures. Accepted for Publication in Astrophysical Journal Letter

First Look at z > 1 Bars in the Rest-frame Near-infrared with JWST Early CEERS Imaging

Stellar bars are key drivers of secular evolution in galaxies and can be effectively studied using rest-frame near-infrared (NIR) images, which trace the underlying stellar mass and are less impacted by dust and star formation than rest-frame UV or optical images. We leverage the power of JWST CEERS NIRCam images to present the first quantitative identification and characterization of stellar bars at z &gt; 1 based on rest-frame NIR F444W images of high resolution (∼1.3 kpc at z ∼ 1-3). We identify stellar bars in these images using quantitative criteria based on ellipse fits. For this pilot study, we present six examples of robustly identified bars at z &gt; 1 with spectroscopic redshifts, including the two highest-redshift bars at z ∼ 2.136 and 2.312 quantitatively identified and characterized to date. The stellar bars at z ∼ 1.1-2.3 presented in our study have projected semimajor axes of ∼2.9-4.3 kpc and projected ellipticities of ∼0.41-0.53 in the rest-frame NIR. The barred host galaxies have stellar masses ∼1 × 10 10 to 2 × 10 11 M ⊙ and star formation rates of ∼21-295 M ⊙ yr −1, and several have potential nearby companions. Our finding of bars at z ∼ 1.1-2.3 demonstrates the early onset of such instabilities and supports simulations where bars form early in massive dynamically cold disks. It also suggests that if these bars at lookback times of 8-11 Gyr survive out to present epochs, bar-driven secular processes may operate over a long time and have a significant impact on some galaxies by z ∼ 0.</p