48 research outputs found
Class-Incremental Learning Using Generative Experience Replay Based on Time-aware Regularization
Learning new tasks accumulatively without forgetting remains a critical
challenge in continual learning. Generative experience replay addresses this
challenge by synthesizing pseudo-data points for past learned tasks and later
replaying them for concurrent training along with the new tasks' data.
Generative replay is the best strategy for continual learning under a strict
class-incremental setting when certain constraints need to be met: (i) constant
model size, (ii) no pre-training dataset, and (iii) no memory buffer for
storing past tasks' data. Inspired by the biological nervous system mechanisms,
we introduce a time-aware regularization method to dynamically fine-tune the
three training objective terms used for generative replay: supervised learning,
latent regularization, and data reconstruction. Experimental results on major
benchmarks indicate that our method pushes the limit of brain-inspired
continual learners under such strict settings, improves memory retention, and
increases the average performance over continually arriving tasks
SHAPNN: Shapley Value Regularized Tabular Neural Network
We present SHAPNN, a novel deep tabular data modeling architecture designed
for supervised learning. Our approach leverages Shapley values, a
well-established technique for explaining black-box models. Our neural network
is trained using standard backward propagation optimization methods, and is
regularized with realtime estimated Shapley values. Our method offers several
advantages, including the ability to provide valid explanations with no
computational overhead for data instances and datasets. Additionally,
prediction with explanation serves as a regularizer, which improves the model's
performance. Moreover, the regularized prediction enhances the model's
capability for continual learning. We evaluate our method on various publicly
available datasets and compare it with state-of-the-art deep neural network
models, demonstrating the superior performance of SHAPNN in terms of AUROC,
transparency, as well as robustness to streaming data.Comment: 9 pages, 8 figure
Improving the Efficiency of Human-in-the-Loop Systems: Adding Artificial to Human Experts
Information systems increasingly leverage artificial intelligence (AI) and
machine learning (ML) to generate value from vast amounts of data. However, ML
models are imperfect and can generate incorrect classifications. Hence,
human-in-the-loop (HITL) extensions to ML models add a human review for
instances that are difficult to classify. This study argues that continuously
relying on human experts to handle difficult model classifications leads to a
strong increase in human effort, which strains limited resources. To address
this issue, we propose a hybrid system that creates artificial experts that
learn to classify data instances from unknown classes previously reviewed by
human experts. Our hybrid system assesses which artificial expert is suitable
for classifying an instance from an unknown class and automatically assigns it.
Over time, this reduces human effort and increases the efficiency of the
system. Our experiments demonstrate that our approach outperforms traditional
HITL systems for several benchmarks on image classification.Comment: Accepted at International Conference on Wirtschaftsinformatik, 202
ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection
Class-incremental learning (CIL) learns a classification model with training
data of different classes arising progressively. Existing CIL either suffers
from serious accuracy loss due to catastrophic forgetting, or invades data
privacy by revisiting used exemplars. Inspired by linear learning formulations,
we propose an analytic class-incremental learning (ACIL) with absolute
memorization of past knowledge while avoiding breaching of data privacy (i.e.,
without storing historical data). The absolute memorization is demonstrated in
the sense that class-incremental learning using ACIL given present data would
give identical results to that from its joint-learning counterpart which
consumes both present and historical samples. This equality is theoretically
validated. Data privacy is ensured since no historical data are involved during
the learning process. Empirical validations demonstrate ACIL's competitive
accuracy performance with near-identical results for various incremental task
settings (e.g., 5-50 phases). This also allows ACIL to outperform the
state-of-the-art methods for large-phase scenarios (e.g., 25 and 50 phases).Comment: published in NeurIPS 202