100 research outputs found
Provably Training Overparameterized Neural Network Classifiers with Non-convex Constraints
Training a classifier under non-convex constraints has gotten increasing
attention in the machine learning community thanks to its wide range of
applications such as algorithmic fairness and class-imbalanced classification.
However, several recent works addressing non-convex constraints have only
focused on simple models such as logistic regression or support vector
machines. Neural networks, one of the most popular models for classification
nowadays, are precluded and lack theoretical guarantees. In this work, we show
that overparameterized neural networks could achieve a near-optimal and
near-feasible solution of non-convex constrained optimization problems via the
project stochastic gradient descent. Our key ingredient is the no-regret
analysis of online learning for neural networks in the overparameterization
regime, which may be of independent interest in online learning applications
FairGBM: Gradient Boosting with Fairness Constraints
Machine Learning (ML) algorithms based on gradient boosted decision trees
(GBDT) are still favored on many tabular data tasks across various mission
critical applications, from healthcare to finance. However, GBDT algorithms are
not free of the risk of bias and discriminatory decision-making. Despite GBDT's
popularity and the rapid pace of research in fair ML, existing in-processing
fair ML methods are either inapplicable to GBDT, incur in significant train
time overhead, or are inadequate for problems with high class imbalance. We
present FairGBM, a learning framework for training GBDT under fairness
constraints with little to no impact on predictive performance when compared to
unconstrained LightGBM. Since common fairness metrics are non-differentiable,
we employ a "proxy-Lagrangian" formulation using smooth convex error rate
proxies to enable gradient-based optimization. Additionally, our open-source
implementation shows an order of magnitude speedup in training time when
compared with related work, a pivotal aspect to foster the widespread adoption
of FairGBM by real-world practitioners
An Analysis of Regularized Approaches for Constrained Machine Learning
open4noopenLombardi, Michele; Baldo, Federico; Borghesi, Andrea; Milano, MichelaLombardi, Michele; Baldo, Federico; Borghesi, Andrea; Milano, Michel
- …