143 research outputs found
Securing the Spike: On the Transferabilty and Security of Spiking Neural Networks to Adversarial Examples
Spiking neural networks (SNNs) have attracted much attention for their high
energy efficiency and for recent advances in their classification performance.
However, unlike traditional deep learning approaches, the analysis and study of
the robustness of SNNs to adversarial examples remains relatively
underdeveloped. In this work we advance the field of adversarial machine
learning through experimentation and analyses of three important SNN security
attributes. First, we show that successful white-box adversarial attacks on
SNNs are highly dependent on the underlying surrogate gradient technique.
Second, we analyze the transferability of adversarial examples generated by
SNNs and other state-of-the-art architectures like Vision Transformers and Big
Transfer CNNs. We demonstrate that SNNs are not often deceived by adversarial
examples generated by Vision Transformers and certain types of CNNs. Lastly, we
develop a novel white-box attack that generates adversarial examples capable of
fooling both SNN models and non-SNN models simultaneously. Our experiments and
analyses are broad and rigorous covering two datasets (CIFAR-10 and CIFAR-100),
five different white-box attacks and twelve different classifier models
Cooperative Spin Amplification
Quantum amplification is recognized as a key resource for precision
measurements. However, most conventional paradigms employ an ensemble of
independent particles that usually limit the performance of quantum
amplification in gain, spectral linewidth, etc. Here we demonstrate a new
signal amplification using cooperative 129Xe nuclear spins embedded within a
feedback circuit, where the noble-gas spin coherence time is enhanced by at
least one order of magnitude. Using such a technique, magnetic field can be
substantially pre-enhanced by more than three orders and is in situ readout
with an embedded 87Rb magnetometer. We realize an ultrahigh magnetic
sensitivity of 4.0 fT/Hz that surpasses the photon-shot noise and even
below the spin-projection noise of the embedded atomic magnetometer, allowing
for exciting applications including searches for dark matter with sensitivity
well beyond supernova constraints. Our findings extend the physics of quantum
amplification to cooperative spin systems and can be generalized to a wide
variety of existing sensors, enabling a new class of cooperative quantum
sensors.Comment: 7 pages, 4 figure
Web-Based Engine For Discovery Of Observations Using Landscape Units
Investigations of natural resources processes realted to the water cycle are best studied using a commensurate landscape unit for the spatial extent of the process. Consequently, the capability to efficiently delineate the watershed extent along with the main hydrological characteristics and behavior of the river network and its drainage area is essential. The watershed search engine discussed in the present paper is designed to identify various observations acquired in the upstream drainage area of the watershed from a point specified by the user. The point can be selected on or outside the stream network using a web mapping interface. The discovered variables and attributes are those stored in the geodatabase associated with the application (e.g., stream flow gages, water quality observations points, weather stations, etc). The base map for the search engine is the National Hydrography Dataset Plus V2.0 (NHD Plus) and Geometric Network analysis are applied to develop the model on the GIS platform. In the application, the user input is defined as the point of interest for the search. Subsequently, the drainage area upstream from the point of interest is identified and visualized using mapping functions available in the NHD Plus library. Ancillary information provided by the NHD database and other relevant attributes of the data for the discovered point of observations are also provided. Given that the variety of activities in the drainage area upstream from the specified point of interests have direct impacts at the location of interest the engine would enhance the information available for efficiently documenting various aspects of water quantity and quality. The application holds promise to benefit users pertaining to watershed management communities and watershed resources researchers
RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion
The raw depth image captured by indoor depth sensors usually has an extensive
range of missing depth values due to inherent limitations such as the inability
to perceive transparent objects and the limited distance range. The incomplete
depth map with missing values burdens many downstream vision tasks, and a
rising number of depth completion methods have been proposed to alleviate this
issue. While most existing methods can generate accurate dense depth maps from
sparse and uniformly sampled depth maps, they are not suitable for
complementing large contiguous regions of missing depth values, which is common
and critical in images captured in indoor environments. To overcome these
challenges, we design a novel two-branch end-to-end fusion network named
RDFC-GAN, which takes a pair of RGB and incomplete depth images as input to
predict a dense and completed depth map. The first branch employs an
encoder-decoder structure, by adhering to the Manhattan world assumption and
utilizing normal maps from RGB-D information as guidance, to regress the local
dense depth values from the raw depth map. In the other branch, we propose an
RGB-depth fusion CycleGAN to transfer the RGB image to the fine-grained
textured depth map. We adopt adaptive fusion modules named W-AdaIN to propagate
the features across the two branches, and we append a confidence fusion head to
fuse the two outputs of the branches for the final depth map. Extensive
experiments on NYU-Depth V2 and SUN RGB-D demonstrate that our proposed method
clearly improves the depth completion performance, especially in a more
realistic setting of indoor environments, with the help of our proposed pseudo
depth maps in training.Comment: Haowen Wang and Zhengping Che are with equal contributions. Under
review. An earlier version has been accepted by CVPR 2022 (arXiv:2203.10856
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
Multi-modality (MM) image fusion aims to render fused images that maintain
the merits of different modalities, e.g., functional highlight and detailed
textures. To tackle the challenge in modeling cross-modality features and
decomposing desirable modality-specific and modality-shared features, we
propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse)
network. Firstly, CDDFuse uses Restormer blocks to extract cross-modality
shallow features. We then introduce a dual-branch Transformer-CNN feature
extractor with Lite Transformer (LT) blocks leveraging long-range attention to
handle low-frequency global features and Invertible Neural Networks (INN)
blocks focusing on extracting high-frequency local information. A
correlation-driven loss is further proposed to make the low-frequency features
correlated while the high-frequency features uncorrelated based on the embedded
information. Then, the LT-based global fusion and INN-based local fusion layers
output the fused image. Extensive experiments demonstrate that our CDDFuse
achieves promising results in multiple fusion tasks, including infrared-visible
image fusion and medical image fusion. We also show that CDDFuse can boost the
performance in downstream infrared-visible semantic segmentation and object
detection in a unified benchmark. The code is available at
https://github.com/Zhaozixiang1228/MMIF-CDDFuse.Comment: Accepted by CVPR 202
Focusing light through scattering media by transmission matrix inversion
Focusing light through scattering media has broad applications in optical imaging, manipulation and therapy. The contrast of the focus can be quantified by peak-to-background intensity ratio (PBR). Here, we theoretically and numerically show that by using a transmission matrix inversion method to achieve focusing, within a limited field of view and under a low noise condition in transmission matrix measurements, the PBR of the focus can be higher than that achieved by conventional methods such as optical phase conjugation or feedback-based wavefront shaping. Experimentally, using a phase-modulation spatial light modulator, we increase the PBR by 66% over that achieved by conventional methods based on phase conjugation. In addition, we demonstrate that, within a limited field of view and under a low noise condition in transmission matrix measurements, our matrix inversion method enables light focusing to multiple foci with greater fidelity than those of conventional methods
DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field
Estimating 6D poses and reconstructing 3D shapes of objects in open-world
scenes from RGB-depth image pairs is challenging. Many existing methods rely on
learning geometric features that correspond to specific templates while
disregarding shape variations and pose differences among objects in the same
category. As a result, these methods underperform when handling unseen object
instances in complex environments. In contrast, other approaches aim to achieve
category-level estimation and reconstruction by leveraging normalized geometric
structure priors, but the static prior-based reconstruction struggles with
substantial intra-class variations. To solve these problems, we propose the
DTF-Net, a novel framework for pose estimation and shape reconstruction based
on implicit neural fields of object categories. In DTF-Net, we design a
deformable template field to represent the general category-wise shape latent
features and intra-category geometric deformation features. The field
establishes continuous shape correspondences, deforming the category template
into arbitrary observed instances to accomplish shape reconstruction. We
introduce a pose regression module that shares the deformation features and
template codes from the fields to estimate the accurate 6D pose of each object
in the scene. We integrate a multi-modal representation extraction module to
extract object features and semantic masks, enabling end-to-end inference.
Moreover, during training, we implement a shape-invariant training strategy and
a viewpoint sampling method to further enhance the model's capability to
extract object pose features. Extensive experiments on the REAL275 and CAMERA25
datasets demonstrate the superiority of DTF-Net in both synthetic and real
scenes. Furthermore, we show that DTF-Net effectively supports grasping tasks
with a real robot arm.Comment: The first two authors are with equal contributions. Paper accepted by
ACM MM 202
- …