Personalized Federated Learning via ADMM with Moreau Envelope

Personalized federated learning (PFL) is an approach proposed to address the issue of poor convergence on heterogeneous data. However, most existing PFL frameworks require strong assumptions for convergence. In this paper, we propose an alternating direction method of multipliers (ADMM) for training PFL models with Moreau envelope (FLAME), which achieves a sublinear convergence rate, relying on the relatively weak assumption of gradient Lipschitz continuity. Moreover, due to the gradient-free nature of ADMM, FLAME alleviates the need for hyperparameter tuning, particularly in avoiding the adjustment of the learning rate when training the global model. In addition, we propose a biased client selection strategy to expedite the convergence of training of PFL models. Our theoretical analysis establishes the global convergence under both unbiased and biased client selection strategies. Our experiments validate that FLAME, when trained on heterogeneous data, outperforms state-of-the-art methods in terms of model performance. Regarding communication efficiency, it exhibits an average speedup of 3.75x compared to the baselines. Furthermore, experimental results validate that the biased client selection strategy speeds up the convergence of both personalized and global models.Comment: 15 page

Efficient Spatial Dataset Search over Multiple Data Sources

In this paper, we investigate a novel spatial dataset search paradigm over multiple spatial data sources, which enables users to conduct join and union searches seamlessly. Specifically, we define two search problems called Maximum Intersection Query (MIQ) and Maximum Coverage Query with a Connection constraint (MCQC). To address these problems, we propose a unified Multi-source Spatial Dataset Search (MSDS) framework. In MSDS, we design a multi-layer index to accelerate the MIQ and MCQC. In addition, we prove that the MCQC is NP-hard and design two greedy algorithms to solve the problem. To deal with the constant update of spatial datasets in each data source, we design a dynamic index updating strategy and optimize search algorithms to reduce communication costs and improve search efficiency. We evaluate the efficiency of MSDS on five real-world data sources, and the experimental results show that our framework is able to achieve a significant reduction in running time and communication cost

Visualization of facet-dependent pseudo-photocatalytic behavior of TiO2 nanorods for water splitting using In situ liquid cell TEM

We report an investigation of the pseudo-photocatalytic behavior of rutile TiO2 nanorods for water splitting observed with liquid cell transmission electron microscopy (TEM). The electron beam serves as a “light” source to initiate the catalytic reaction and a “water-in-salt” aqueous solution is employed as the electrolyte. The observation reveals that bubbles are generated preferentially residing near the {110} facet of a rutile TiO2 nanorod under a low electron dose rate (9.3–18.6 e-/Å2·s). These bubbles are ascribed to hydrogen gas generated from the pseudo-photocatalytic water splitting. As the electron beam current density increases to 93 e-/Å2 ·s, bubbles are also found at the {001} and {111} facets as well as in the bulk liquid solution, demonstrating the dominant effects of water electrolysis by electron beam under higher dose rates. The facet-dependent pseudo-photocatalytic behavior of rutile TiO2 nanorods is further validated using density functional theory (DFT)calculation. Our work establishes a facile liquid cell TEM setup for the study of pseudo-photocatalytic water splitting and it may also be applied to investigation of other photo-activated phenomena occurring at the solid-liquid interfaces

Cavity locking with spatial modulation of optical phase front for laser stabilization

We study optical cavity locking for laser stabilization through spatial modulation of the phase front of a light beam. A theoretical description of the underlying principle is developed for this method and special attention is paid to residual amplitude modulation (RAM) caused by experimental imperfections, especially the manufacture errors of the spatial phase modulator. The studied locking method owns the common advantages of the Pound-Drever-Hall method and the tilt-locking one, and it can provide a more artful way to eliminate RAM noise in phase modulation for the ultimate stability of lasers. In situations where cost and portability are a practical issue, the studied method allows one to realize compact laser stabilization systems locked to Fabry-P\acute{\mbox{e}}rot cavities without use of expensive bulky devices, such as signal generators and electro-optic modulators.Comment: 3 figure

Exploring Post-Training Quantization of Protein Language Models

Recent advancements in unsupervised protein language models (ProteinLMs), like ESM-1b and ESM-2, have shown promise in different protein prediction tasks. However, these models face challenges due to their high computational demands, significant memory needs, and latency, restricting their usage on devices with limited resources. To tackle this, we explore post-training quantization (PTQ) for ProteinLMs, focusing on ESMFold, a simplified version of AlphaFold based on ESM-2 ProteinLM. Our study is the first attempt to quantize all weights and activations of ProteinLMs. We observed that the typical uniform quantization method performs poorly on ESMFold, causing a significant drop in TM-Score when using 8-bit quantization. We conducted extensive quantization experiments, uncovering unique challenges associated with ESMFold, particularly highly asymmetric activation ranges before Layer Normalization, making representation difficult using low-bit fixed-point formats. To address these challenges, we propose a new PTQ method for ProteinLMs, utilizing piecewise linear quantization for asymmetric activation values to ensure accurate approximation. We demonstrated the effectiveness of our method in protein structure prediction tasks, demonstrating that ESMFold can be accurately quantized to low-bit widths without compromising accuracy. Additionally, we applied our method to the contact prediction task, showcasing its versatility. In summary, our study introduces an innovative PTQ method for ProteinLMs, addressing specific quantization challenges and potentially leading to the development of more efficient ProteinLMs with significant implications for various protein-related applications.Comment: 8 pages, 4 figure

Diffusion basis spectrum imaging for identifying pathologies in MS subtypes

Diffusion basis spectrum imaging (DBSI) combines discrete anisotropic diffusion tensors and the spectrum of isotropic diffusion tensors to model the underlying multiple sclerosis (MS) pathologies. We used clinical MS subtypes as a surrogate of underlying pathologies to assess DBSI as a biomarker of pathology in 55 individuals with MS. Restricted isotropic fraction (reflecting cellularity) and fiber fraction (representing apparent axonal density) were the most important DBSI metrics to classify MS using brain white matter lesions. These DBSI metrics outperformed lesion volume. When analyzing the normal-appearing corpus callosum, the most significant DBSI metrics were fiber fraction, radial diffusivity (reflecting myelination), and nonrestricted isotropic fraction (representing edema). This study provides preliminary evidence supporting the ability of DBSI as a potential noninvasive biomarker of MS neuropathology