Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles

We study contextual linear bandit problems under uncertainty on features; they are noisy with missing entries. To address the challenges from the noise, we analyze Bayesian oracles given observed noisy features. Our Bayesian analysis finds that the optimal hypothesis can be far from the underlying realizability function, depending on noise characteristics, which is highly non-intuitive and does not occur for classical noiseless setups. This implies that classical approaches cannot guarantee a non-trivial regret bound. We thus propose an algorithm aiming at the Bayesian oracle from observed information under this model, achieving $\tilde{O}(d\sqrt{T})$ regret bound with respect to feature dimension $d$ and time horizon $T$. We demonstrate the proposed algorithm using synthetic and real-world datasets.Comment: 30 page

Efficient Parallel Audio Generation using Group Masked Language Modeling

We present a fast and high-quality codec language model for parallel audio generation. While SoundStorm, a state-of-the-art parallel audio generation model, accelerates inference speed compared to autoregressive models, it still suffers from slow inference due to iterative sampling. To resolve this problem, we propose Group-Masked Language Modeling~(G-MLM) and Group Iterative Parallel Decoding~(G-IPD) for efficient parallel audio generation. Both the training and sampling schemes enable the model to synthesize high-quality audio with a small number of iterations by effectively modeling the group-wise conditional dependencies. In addition, our model employs a cross-attention-based architecture to capture the speaker style of the prompt voice and improves computational efficiency. Experimental results demonstrate that our proposed model outperforms the baselines in prompt-based audio generation.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

FedSoL: Bridging Global Alignment and Local Generality in Federated Learning

Federated Learning (FL) aggregates locally trained models from individual clients to construct a global model. While FL enables learning a model with data privacy, it often suffers from significant performance degradation when client data distributions are heterogeneous. Many previous FL algorithms have addressed this issue by introducing various proximal restrictions. These restrictions aim to encourage global alignment by constraining the deviation of local learning from the global objective. However, they inherently limit local learning by interfering with the original local objectives. Recently, an alternative approach has emerged to improve local learning generality. By obtaining local models within a smooth loss landscape, this approach mitigates conflicts among different local objectives of the clients. Yet, it does not ensure stable global alignment, as local learning does not take the global objective into account. In this study, we propose Federated Stability on Learning (FedSoL), which combines both the concepts of global alignment and local generality. In FedSoL, the local learning seeks a parameter region robust against proximal perturbations. This strategy introduces an implicit proximal restriction effect in local learning while maintaining the original local objective for parameter update. Our experiments show that FedSoL consistently achieves state-of-the-art performance on various setups

FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning

Federated Learning (FL) has emerged as a pivotal framework for the development of effective global models (global FL) or personalized models (personalized FL) across clients with heterogeneous, non-iid data distribution. A key challenge in FL is client drift, where data heterogeneity impedes the aggregation of scattered knowledge. Recent studies have tackled the client drift issue by identifying significant divergence in the last classifier layer. To mitigate this divergence, strategies such as freezing the classifier weights and aligning the feature extractor accordingly have proven effective. Although the local alignment between classifier and feature extractor has been studied as a crucial factor in FL, we observe that it may lead the model to overemphasize the observed classes within each client. Thus, our objectives are twofold: (1) enhancing local alignment while (2) preserving the representation of unseen class samples. This approach aims to effectively integrate knowledge from individual clients, thereby improving performance for both global and personalized FL. To achieve this, we introduce a novel algorithm named FedDr+, which empowers local model alignment using dot-regression loss. FedDr+ freezes the classifier as a simplex ETF to align the features and improves aggregated global models by employing a feature distillation mechanism to retain information about unseen/missing classes. Consequently, we provide empirical evidence demonstrating that our algorithm surpasses existing methods that use a frozen classifier to boost alignment across the diverse distribution

High methane combustion activity of PdO/CeO2–ZrO2–NiO/γ-Al2O3 catalysts

PdO/CeO2–ZrO2–NiO/γ-Al2O3 catalysts were prepared for the combustion of methane at moderate temperatures. The introduction of a small amount of NiO within the cubic fluorite CeO2–ZrO2 structure as a promoter effectively enhanced the oxygen release and storage abilities of the catalysts, thereby achieving the complete oxidation of methane. The catalyst with the highest activity for methane combustion was 11.3 mass% PdO/20 mass% Ce0.64Zr0.16Ni0.2O1.9/γ-Al2O3, and efficient combustion was realized at a temperature as low as 300 °C

Preservation of Global Knowledge by Not-True Distillation in Federated Learning

In federated learning, a strong global model is collaboratively learned by aggregating clients' locally trained models. Although this precludes the need to access clients' data directly, the global model's convergence often suffers from data heterogeneity. This study starts from an analogy to continual learning and suggests that forgetting could be the bottleneck of federated learning. We observe that the global model forgets the knowledge from previous rounds, and the local training induces forgetting the knowledge outside of the local distribution. Based on our findings, we hypothesize that tackling down forgetting will relieve the data heterogeneity problem. To this end, we propose a novel and effective algorithm, Federated Not-True Distillation (FedNTD), which preserves the global perspective on locally available data only for the not-true classes. In the experiments, FedNTD shows state-of-the-art performance on various setups without compromising data privacy or incurring additional communication costs.Comment: Under revie

Three-dimensional surface lattice plasmon resonance effect from plasmonic inclined nanostructures via one-step stencil lithography

Plasmonic nanostructures allow the manipulation and confinement of optical fields on the sub-wavelength scale. The local field enhancement and environmentally sensitive resonance characteristics provided by these nanostructures are of high importance for biological and chemical sensing. Recently, surface lattice plasmon resonance (SLR) research has attracted much interest because of its superior quality factor (Q-factor) compared to that of localized surface plasmon resonances (LSPR), which is facilitated by resonant plasmonic mode coupling between individual nanostructures over a large area. This advantage can be further enhanced by utilizing asymmetric 3D structures rather than low-height (typically height < ∼60 nm) structure arrays, which results in stronger coupling due to an increased mode volume. However, fabricating 3D, high-aspect ratio, symmetry-breaking structures is a complex and challenging process even with state-of-the-art fabrication technology. Here, we report a plasmonic metasurface of 3D inclined structures produced via commercial TEM grid–based stencil lithography with a Q-factor of 101.6, a refractive index sensitivity of 291 nm/RIU, and a figure of merit (FOM) of 44.7 in the visible wavelength range at a refractive index of 1.5 by utilizing the 3D SLR enhancement effect, which exceeds the performance of most LSPR systems (Q < ∼10). The symmetry-breaking 3D inclined structures that are fabricated by electron beam evaporation at an angle increase the polarizability of the metasurface and the directionality of the diffractively scattered radiative field responsible for SLR mode coupling. Additionally, we explore the role of spatial coherence in facilitating the SLR effect and thus a high-Q plasmonic response from the nanostructures. Our work demonstrates the feasibility of producing 3D inclined structure arrays with pronounced SLR enhancement for high biological sensitivity by utilizing the previously unexplored inclined stencil lithography, which opens the way to fabricate highly sensitive plasmonic metasurfaces with this novel simple technique