452 research outputs found
Large-scale global optimization of ultra-high dimensional non-convex landscapes based on generative neural networks
We present a non-convex optimization algorithm metaheuristic, based on the
training of a deep generative network, which enables effective searching within
continuous, ultra-high dimensional landscapes. During network training,
populations of sampled local gradients are utilized within a customized loss
function to evolve the network output distribution function towards one peak at
high-performing optima. The deep network architecture is tailored to support
progressive growth over the course of training, which allows the algorithm to
manage the curse of dimensionality characteristic of high-dimensional
landscapes. We apply our concept to a range of standard optimization problems
with dimensions as high as one thousand and show that our method performs
better with fewer function evaluations compared to state-of-the-art algorithm
benchmarks. We also discuss the role of deep network over-parameterization,
loss function engineering, and proper network architecture selection in
optimization, and why the required batch size of sampled local gradients is
independent of problem dimension. These concepts form the foundation for a new
class of algorithms that utilize customizable and expressive deep generative
networks to solve non-convex optimization problems
Balanced Multi-modal Federated Learning via Cross-Modal Infiltration
Federated learning (FL) underpins advancements in privacy-preserving
distributed computing by collaboratively training neural networks without
exposing clients' raw data. Current FL paradigms primarily focus on uni-modal
data, while exploiting the knowledge from distributed multimodal data remains
largely unexplored. Existing multimodal FL (MFL) solutions are mainly designed
for statistical or modality heterogeneity from the input side, however, have
yet to solve the fundamental issue,"modality imbalance", in distributed
conditions, which can lead to inadequate information exploitation and
heterogeneous knowledge aggregation on different modalities.In this paper, we
propose a novel Cross-Modal Infiltration Federated Learning (FedCMI) framework
that effectively alleviates modality imbalance and knowledge heterogeneity via
knowledge transfer from the global dominant modality. To avoid the loss of
information in the weak modality due to merely imitating the behavior of
dominant modality, we design the two-projector module to integrate the
knowledge from dominant modality while still promoting the local feature
exploitation of weak modality. In addition, we introduce a class-wise
temperature adaptation scheme to achieve fair performance across different
classes. Extensive experiments over popular datasets are conducted and give us
a gratifying confirmation of the proposed framework for fully exploring the
information of each modality in MFL.Comment: 10 pages, 5 figures 4 table
Dual-mode adaptive-SVD ghost imaging
In this paper, we present a dual-mode adaptive singular value decomposition
ghost imaging (A-SVD GI), which can be easily switched between the modes of
imaging and edge detection. It can adaptively localize the foreground pixels
via a threshold selection method. Then only the foreground region is
illuminated by the singular value decomposition (SVD) - based patterns,
consequently retrieving high-quality images with fewer sampling ratios. By
changing the selecting range of foreground pixels, the A-SVD GI can be switched
to the mode of edge detection to directly reveal the edge of objects, without
needing the original image. We investigate the performance of these two modes
through both numerical simulations and experiments. We also develop a
single-round scheme to halve measurement numbers in experiments, instead of
separately illuminating positive and negative patterns in traditional methods.
The binarized SVD patterns, generated by the spatial dithering method, are
modulated by a digital micromirror device (DMD) to speed up the data
acquisition. This dual-mode A-SVD GI can be applied in various applications,
such as remote sensing or target recognition, and could be further extended for
multi-modality functional imaging/detection
DualMix: Unleashing the Potential of Data Augmentation for Online Class-Incremental Learning
Online Class-Incremental (OCI) learning has sparked new approaches to expand
the previously trained model knowledge from sequentially arriving data streams
with new classes. Unfortunately, OCI learning can suffer from catastrophic
forgetting (CF) as the decision boundaries for old classes can become
inaccurate when perturbated by new ones. Existing literature have applied the
data augmentation (DA) to alleviate the model forgetting, while the role of DA
in OCI has not been well understood so far. In this paper, we theoretically
show that augmented samples with lower correlation to the original data are
more effective in preventing forgetting. However, aggressive augmentation may
also reduce the consistency between data and corresponding labels, which
motivates us to exploit proper DA to boost the OCI performance and prevent the
CF problem. We propose the Enhanced Mixup (EnMix) method that mixes the
augmented samples and their labels simultaneously, which is shown to enhance
the sample diversity while maintaining strong consistency with corresponding
labels. Further, to solve the class imbalance problem, we design an Adaptive
Mixup (AdpMix) method to calibrate the decision boundaries by mixing samples
from both old and new classes and dynamically adjusting the label mixing ratio.
Our approach is demonstrated to be effective on several benchmark datasets
through extensive experiments, and it is shown to be compatible with other
replay-based techniques.Comment: 10 pages, 7 figures and 3 table
- …