5 research outputs found
Vector Quantized Bayesian Neural Network Inference for Data Streams
Bayesian neural networks (BNN) can estimate the uncertainty in predictions,
as opposed to non-Bayesian neural networks (NNs). However, BNNs have been far
less widely used than non-Bayesian NNs in practice since they need iterative NN
executions to predict a result for one data, and it gives rise to prohibitive
computational cost. This computational burden is a critical problem when
processing data streams with low-latency. To address this problem, we propose a
novel model VQ-BNN, which approximates BNN inference for data streams. In order
to reduce the computational burden, VQ-BNN inference predicts NN only once and
compensates the result with previously memorized predictions. To be specific,
VQ-BNN inference for data streams is given by temporal exponential smoothing of
recent predictions. The computational cost of this model is almost the same as
that of non-Bayesian NNs. Experiments including semantic segmentation on
real-world data show that this model performs significantly faster than BNNs
while estimating predictive results comparable to or superior to the results of
BNNs.Comment: AAAI 202
FlexSketch: Estimation of Probability Density for Stationary and Non-Stationary Data Streams
Efficient and accurate estimation of the probability distribution of a data stream is an important problem in many sensor systems. It is especially challenging when the data stream is non-stationary, i.e., its probability distribution changes over time. Statistical models for non-stationary data streams demand agile adaptation for concept drift while tolerating temporal fluctuations. To this end, a statistical model needs to forget old data samples and to detect concept drift swiftly. In this paper, we propose FlexSketch, an online probability density estimation algorithm for data streams. Our algorithm uses an ensemble of histograms, each of which represents a different length of data history. FlexSketch updates each histogram for a new data sample and generates probability distribution by combining the ensemble of histograms while monitoring discrepancy between recent data and existing models periodically. When it detects concept drift, a new histogram is added to the ensemble and the oldest histogram is removed. This allows us to estimate the probability density function with high update speed and high accuracy using only limited memory. Experimental results demonstrate that our algorithm shows improved speed and accuracy compared to existing methods for both stationary and non-stationary data streams