1,354 research outputs found

    Regret-Optimal Federated Transfer Learning for Kernel Regression with Applications in American Option Pricing

    Full text link
    We propose an optimal iterative scheme for federated transfer learning, where a central planner has access to datasets D1,…,DN{\cal D}_1,\dots,{\cal D}_N for the same learning model fθf_{\theta}. Our objective is to minimize the cumulative deviation of the generated parameters {θi(t)}t=0T\{\theta_i(t)\}_{t=0}^T across all TT iterations from the specialized parameters θ1⋆,…,θN⋆\theta^\star_{1},\ldots,\theta^\star_N obtained for each dataset, while respecting the loss function for the model fθ(T)f_{\theta(T)} produced by the algorithm upon halting. We only allow for continual communication between each of the specialized models (nodes/agents) and the central planner (server), at each iteration (round). For the case where the model fθf_{\theta} is a finite-rank kernel regression, we derive explicit updates for the regret-optimal algorithm. By leveraging symmetries within the regret-optimal algorithm, we further develop a nearly regret-optimal heuristic that runs with O(Np2)\mathcal{O}(Np^2) fewer elementary operations, where pp is the dimension of the parameter space. Additionally, we investigate the adversarial robustness of the regret-optimal algorithm showing that an adversary which perturbs qq training pairs by at-most ε>0\varepsilon>0, across all training sets, cannot reduce the regret-optimal algorithm's regret by more than O(εqNˉ1/2)\mathcal{O}(\varepsilon q \bar{N}^{1/2}), where Nˉ\bar{N} is the aggregate number of training pairs. To validate our theoretical findings, we conduct numerical experiments in the context of American option pricing, utilizing a randomly generated finite-rank kernel.Comment: 54 pages, 3 figure

    Cross-Silo Federated Learning Across Divergent Domains with Iterative Parameter Alignment

    Full text link
    Learning from the collective knowledge of data dispersed across private sources can provide neural networks with enhanced generalization capabilities. Federated learning, a method for collaboratively training a machine learning model across remote clients, achieves this by combining client models via the orchestration of a central server. However, current approaches face two critical limitations: i) they struggle to converge when client domains are sufficiently different, and ii) current aggregation techniques produce an identical global model for each client. In this work, we address these issues by reformulating the typical federated learning setup: rather than learning a single global model, we learn N models each optimized for a common objective. To achieve this, we apply a weighted distance minimization to model parameters shared in a peer-to-peer topology. The resulting framework, Iterative Parameter Alignment, applies naturally to the cross-silo setting, and has the following properties: (i) a unique solution for each participant, with the option to globally converge each model in the federation, and (ii) an optional early-stopping mechanism to elicit fairness among peers in collaborative learning settings. These characteristics jointly provide a flexible new framework for iteratively learning from peer models trained on disparate datasets. We find that the technique achieves competitive results on a variety of data partitions compared to state-of-the-art approaches. Further, we show that the method is robust to divergent domains (i.e. disjoint classes across peers) where existing approaches struggle.Comment: Published at IEEE Big Data 202

    Federated Learning for Sparse Principal Component Analysis

    Full text link
    In the rapidly evolving realm of machine learning, algorithm effectiveness often faces limitations due to data quality and availability. Traditional approaches grapple with data sharing due to legal and privacy concerns. The federated learning framework addresses this challenge. Federated learning is a decentralized approach where model training occurs on client sides, preserving privacy by keeping data localized. Instead of sending raw data to a central server, only model updates are exchanged, enhancing data security. We apply this framework to Sparse Principal Component Analysis (SPCA) in this work. SPCA aims to attain sparse component loadings while maximizing data variance for improved interpretability. Beside the L1 norm regularization term in conventional SPCA, we add a smoothing function to facilitate gradient-based optimization methods. Moreover, in order to improve computational efficiency, we introduce a least squares approximation to original SPCA. This enables analytic solutions on the optimization processes, leading to substantial computational improvements. Within the federated framework, we formulate SPCA as a consensus optimization problem, which can be solved using the Alternating Direction Method of Multipliers (ADMM). Our extensive experiments involve both IID and non-IID random features across various data owners. Results on synthetic and public datasets affirm the efficacy of our federated SPCA approach.Comment: 11 pages, 7 figures, 1 table. Accepted by IEEE BigData 2023, Sorrento, Ital
    • …
    corecore