79,290 research outputs found
Fair yet Asymptotically Equal Collaborative Learning
In collaborative learning with streaming data, nodes (e.g., organizations)
jointly and continuously learn a machine learning (ML) model by sharing the
latest model updates computed from their latest streaming data. For the more
resourceful nodes to be willing to share their model updates, they need to be
fairly incentivized. This paper explores an incentive design that guarantees
fairness so that nodes receive rewards commensurate to their contributions. Our
approach leverages an explore-then-exploit formulation to estimate the nodes'
contributions (i.e., exploration) for realizing our theoretically guaranteed
fair incentives (i.e., exploitation). However, we observe a "rich get richer"
phenomenon arising from the existing approaches to guarantee fairness and it
discourages the participation of the less resourceful nodes. To remedy this, we
additionally preserve asymptotic equality, i.e., less resourceful nodes achieve
equal performance eventually to the more resourceful/"rich" nodes. We
empirically demonstrate in two settings with real-world streaming data:
federated online incremental learning and federated reinforcement learning,
that our proposed approach outperforms existing baselines in fairness and
learning performance while remaining competitive in preserving equality.Comment: Accepted to 40th International Conference on Machine Learning (ICML
2023), 37 page
An empirical learning-based validation procedure for simulation workflow
Simulation workflow is a top-level model for the design and control of
simulation process. It connects multiple simulation components with time and
interaction restrictions to form a complete simulation system. Before the
construction and evaluation of the component models, the validation of
upper-layer simulation workflow is of the most importance in a simulation
system. However, the methods especially for validating simulation workflow is
very limit. Many of the existing validation techniques are domain-dependent
with cumbersome questionnaire design and expert scoring. Therefore, this paper
present an empirical learning-based validation procedure to implement a
semi-automated evaluation for simulation workflow. First, representative
features of general simulation workflow and their relations with validation
indices are proposed. The calculation process of workflow credibility based on
Analytic Hierarchy Process (AHP) is then introduced. In order to make full use
of the historical data and implement more efficient validation, four learning
algorithms, including back propagation neural network (BPNN), extreme learning
machine (ELM), evolving new-neuron (eNFN) and fast incremental gaussian mixture
model (FIGMN), are introduced for constructing the empirical relation between
the workflow credibility and its features. A case study on a landing-process
simulation workflow is established to test the feasibility of the proposed
procedure. The experimental results also provide some useful overview of the
state-of-the-art learning algorithms on the credibility evaluation of
simulation models
Private Incremental Regression
Data is continuously generated by modern data sources, and a recent challenge
in machine learning has been to develop techniques that perform well in an
incremental (streaming) setting. In this paper, we investigate the problem of
private machine learning, where as common in practice, the data is not given at
once, but rather arrives incrementally over time.
We introduce the problems of private incremental ERM and private incremental
regression where the general goal is to always maintain a good empirical risk
minimizer for the history observed under differential privacy. Our first
contribution is a generic transformation of private batch ERM mechanisms into
private incremental ERM mechanisms, based on a simple idea of invoking the
private batch ERM procedure at some regular time intervals. We take this
construction as a baseline for comparison. We then provide two mechanisms for
the private incremental regression problem. Our first mechanism is based on
privately constructing a noisy incremental gradient function, which is then
used in a modified projected gradient procedure at every timestep. This
mechanism has an excess empirical risk of , where is the
dimensionality of the data. While from the results of [Bassily et al. 2014]
this bound is tight in the worst-case, we show that certain geometric
properties of the input and constraint set can be used to derive significantly
better results for certain interesting regression problems.Comment: To appear in PODS 201
- …