1,354 research outputs found
Regret-Optimal Federated Transfer Learning for Kernel Regression with Applications in American Option Pricing
We propose an optimal iterative scheme for federated transfer learning, where
a central planner has access to datasets for the
same learning model . Our objective is to minimize the cumulative
deviation of the generated parameters across all
iterations from the specialized parameters
obtained for each dataset, while
respecting the loss function for the model produced by the
algorithm upon halting. We only allow for continual communication between each
of the specialized models (nodes/agents) and the central planner (server), at
each iteration (round). For the case where the model is a
finite-rank kernel regression, we derive explicit updates for the
regret-optimal algorithm. By leveraging symmetries within the regret-optimal
algorithm, we further develop a nearly regret-optimal heuristic that runs with
fewer elementary operations, where is the dimension of
the parameter space. Additionally, we investigate the adversarial robustness of
the regret-optimal algorithm showing that an adversary which perturbs
training pairs by at-most , across all training sets, cannot
reduce the regret-optimal algorithm's regret by more than
, where is the aggregate
number of training pairs. To validate our theoretical findings, we conduct
numerical experiments in the context of American option pricing, utilizing a
randomly generated finite-rank kernel.Comment: 54 pages, 3 figure
Cross-Silo Federated Learning Across Divergent Domains with Iterative Parameter Alignment
Learning from the collective knowledge of data dispersed across private
sources can provide neural networks with enhanced generalization capabilities.
Federated learning, a method for collaboratively training a machine learning
model across remote clients, achieves this by combining client models via the
orchestration of a central server. However, current approaches face two
critical limitations: i) they struggle to converge when client domains are
sufficiently different, and ii) current aggregation techniques produce an
identical global model for each client. In this work, we address these issues
by reformulating the typical federated learning setup: rather than learning a
single global model, we learn N models each optimized for a common objective.
To achieve this, we apply a weighted distance minimization to model parameters
shared in a peer-to-peer topology. The resulting framework, Iterative Parameter
Alignment, applies naturally to the cross-silo setting, and has the following
properties: (i) a unique solution for each participant, with the option to
globally converge each model in the federation, and (ii) an optional
early-stopping mechanism to elicit fairness among peers in collaborative
learning settings. These characteristics jointly provide a flexible new
framework for iteratively learning from peer models trained on disparate
datasets. We find that the technique achieves competitive results on a variety
of data partitions compared to state-of-the-art approaches. Further, we show
that the method is robust to divergent domains (i.e. disjoint classes across
peers) where existing approaches struggle.Comment: Published at IEEE Big Data 202
Model Pruning Enables Localized and Efficient Federated Learning for Yield Forecasting and Data Sharing
31 pages, 4 figures, 4 tablesPreprin
Federated Learning for Sparse Principal Component Analysis
In the rapidly evolving realm of machine learning, algorithm effectiveness
often faces limitations due to data quality and availability. Traditional
approaches grapple with data sharing due to legal and privacy concerns. The
federated learning framework addresses this challenge. Federated learning is a
decentralized approach where model training occurs on client sides, preserving
privacy by keeping data localized. Instead of sending raw data to a central
server, only model updates are exchanged, enhancing data security. We apply
this framework to Sparse Principal Component Analysis (SPCA) in this work. SPCA
aims to attain sparse component loadings while maximizing data variance for
improved interpretability. Beside the L1 norm regularization term in
conventional SPCA, we add a smoothing function to facilitate gradient-based
optimization methods. Moreover, in order to improve computational efficiency,
we introduce a least squares approximation to original SPCA. This enables
analytic solutions on the optimization processes, leading to substantial
computational improvements. Within the federated framework, we formulate SPCA
as a consensus optimization problem, which can be solved using the Alternating
Direction Method of Multipliers (ADMM). Our extensive experiments involve both
IID and non-IID random features across various data owners. Results on
synthetic and public datasets affirm the efficacy of our federated SPCA
approach.Comment: 11 pages, 7 figures, 1 table. Accepted by IEEE BigData 2023,
Sorrento, Ital
- …