3,560 research outputs found
Personalized Federated Learning via ADMM with Moreau Envelope
Personalized federated learning (PFL) is an approach proposed to address the
issue of poor convergence on heterogeneous data. However, most existing PFL
frameworks require strong assumptions for convergence. In this paper, we
propose an alternating direction method of multipliers (ADMM) for training PFL
models with Moreau envelope (FLAME), which achieves a sublinear convergence
rate, relying on the relatively weak assumption of gradient Lipschitz
continuity. Moreover, due to the gradient-free nature of ADMM, FLAME alleviates
the need for hyperparameter tuning, particularly in avoiding the adjustment of
the learning rate when training the global model. In addition, we propose a
biased client selection strategy to expedite the convergence of training of PFL
models. Our theoretical analysis establishes the global convergence under both
unbiased and biased client selection strategies. Our experiments validate that
FLAME, when trained on heterogeneous data, outperforms state-of-the-art methods
in terms of model performance. Regarding communication efficiency, it exhibits
an average speedup of 3.75x compared to the baselines. Furthermore,
experimental results validate that the biased client selection strategy speeds
up the convergence of both personalized and global models.Comment: 15 page
Efficient Spatial Dataset Search over Multiple Data Sources
In this paper, we investigate a novel spatial dataset search paradigm over
multiple spatial data sources, which enables users to conduct join and union
searches seamlessly. Specifically, we define two search problems called Maximum
Intersection Query (MIQ) and Maximum Coverage Query with a Connection
constraint (MCQC). To address these problems, we propose a unified Multi-source
Spatial Dataset Search (MSDS) framework. In MSDS, we design a multi-layer index
to accelerate the MIQ and MCQC. In addition, we prove that the MCQC is NP-hard
and design two greedy algorithms to solve the problem. To deal with the
constant update of spatial datasets in each data source, we design a dynamic
index updating strategy and optimize search algorithms to reduce communication
costs and improve search efficiency. We evaluate the efficiency of MSDS on five
real-world data sources, and the experimental results show that our framework
is able to achieve a significant reduction in running time and communication
cost
Recommended from our members
Visualization of facet-dependent pseudo-photocatalytic behavior of TiO2 nanorods for water splitting using In situ liquid cell TEM
We report an investigation of the pseudo-photocatalytic behavior of rutile TiO2 nanorods for water splitting observed with liquid cell transmission electron microscopy (TEM). The electron beam serves as a “light” source to initiate the catalytic reaction and a “water-in-salt” aqueous solution is employed as the electrolyte. The observation reveals that bubbles are generated preferentially residing near the {110} facet of a rutile TiO2 nanorod under a low electron dose rate (9.3–18.6 e-/Å2·s). These bubbles are ascribed to hydrogen gas generated from the pseudo-photocatalytic water splitting. As the electron beam current density increases to 93 e-/Å2 ·s, bubbles are also found at the {001} and {111} facets as well as in the bulk liquid solution, demonstrating the dominant effects of water electrolysis by electron beam under higher dose rates. The facet-dependent pseudo-photocatalytic behavior of rutile TiO2 nanorods is further validated using density functional theory (DFT)calculation. Our work establishes a facile liquid cell TEM setup for the study of pseudo-photocatalytic water splitting and it may also be applied to investigation of other photo-activated phenomena occurring at the solid-liquid interfaces
Cavity locking with spatial modulation of optical phase front for laser stabilization
We study optical cavity locking for laser stabilization through spatial
modulation of the phase front of a light beam. A theoretical description of the
underlying principle is developed for this method and special attention is paid
to residual amplitude modulation (RAM) caused by experimental imperfections,
especially the manufacture errors of the spatial phase modulator. The studied
locking method owns the common advantages of the Pound-Drever-Hall method and
the tilt-locking one, and it can provide a more artful way to eliminate RAM
noise in phase modulation for the ultimate stability of lasers. In situations
where cost and portability are a practical issue, the studied method allows one
to realize compact laser stabilization systems locked to
Fabry-P\acute{\mbox{e}}rot cavities without use of expensive bulky devices,
such as signal generators and electro-optic modulators.Comment: 3 figure
Exploring Post-Training Quantization of Protein Language Models
Recent advancements in unsupervised protein language models (ProteinLMs),
like ESM-1b and ESM-2, have shown promise in different protein prediction
tasks. However, these models face challenges due to their high computational
demands, significant memory needs, and latency, restricting their usage on
devices with limited resources. To tackle this, we explore post-training
quantization (PTQ) for ProteinLMs, focusing on ESMFold, a simplified version of
AlphaFold based on ESM-2 ProteinLM. Our study is the first attempt to quantize
all weights and activations of ProteinLMs. We observed that the typical uniform
quantization method performs poorly on ESMFold, causing a significant drop in
TM-Score when using 8-bit quantization. We conducted extensive quantization
experiments, uncovering unique challenges associated with ESMFold, particularly
highly asymmetric activation ranges before Layer Normalization, making
representation difficult using low-bit fixed-point formats. To address these
challenges, we propose a new PTQ method for ProteinLMs, utilizing piecewise
linear quantization for asymmetric activation values to ensure accurate
approximation. We demonstrated the effectiveness of our method in protein
structure prediction tasks, demonstrating that ESMFold can be accurately
quantized to low-bit widths without compromising accuracy. Additionally, we
applied our method to the contact prediction task, showcasing its versatility.
In summary, our study introduces an innovative PTQ method for ProteinLMs,
addressing specific quantization challenges and potentially leading to the
development of more efficient ProteinLMs with significant implications for
various protein-related applications.Comment: 8 pages, 4 figure
Diffusion basis spectrum imaging for identifying pathologies in MS subtypes
Diffusion basis spectrum imaging (DBSI) combines discrete anisotropic diffusion tensors and the spectrum of isotropic diffusion tensors to model the underlying multiple sclerosis (MS) pathologies. We used clinical MS subtypes as a surrogate of underlying pathologies to assess DBSI as a biomarker of pathology in 55 individuals with MS. Restricted isotropic fraction (reflecting cellularity) and fiber fraction (representing apparent axonal density) were the most important DBSI metrics to classify MS using brain white matter lesions. These DBSI metrics outperformed lesion volume. When analyzing the normal-appearing corpus callosum, the most significant DBSI metrics were fiber fraction, radial diffusivity (reflecting myelination), and nonrestricted isotropic fraction (representing edema). This study provides preliminary evidence supporting the ability of DBSI as a potential noninvasive biomarker of MS neuropathology
- …