878 research outputs found
Sequential Gaussian Processes for Online Learning of Nonstationary Functions
Many machine learning problems can be framed in the context of estimating
functions, and often these are time-dependent functions that are estimated in
real-time as observations arrive. Gaussian processes (GPs) are an attractive
choice for modeling real-valued nonlinear functions due to their flexibility
and uncertainty quantification. However, the typical GP regression model
suffers from several drawbacks: i) Conventional GP inference scales
with respect to the number of observations; ii) updating a GP model
sequentially is not trivial; and iii) covariance kernels often enforce
stationarity constraints on the function, while GPs with non-stationary
covariance kernels are often intractable to use in practice. To overcome these
issues, we propose an online sequential Monte Carlo algorithm to fit mixtures
of GPs that capture non-stationary behavior while allowing for fast,
distributed inference. By formulating hyperparameter optimization as a
multi-armed bandit problem, we accelerate mixing for real time inference. Our
approach empirically improves performance over state-of-the-art methods for
online GP estimation in the context of prediction for simulated non-stationary
data and hospital time series data
Surrogate Models and Mixtures of Experts in Aerodynamic Performance Prediction for Mission Analysis
Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/140436/1/6.2014-2301.pd
Bitcoin Volatility Forecasting with a Glimpse into Buy and Sell Orders
In this paper, we study the ability to make the short-term prediction of the
exchange price fluctuations towards the United States dollar for the Bitcoin
market. We use the data of realized volatility collected from one of the
largest Bitcoin digital trading offices in 2016 and 2017 as well as order
information. Experiments are performed to evaluate a variety of statistical and
machine learning approaches.Comment: Full version of the paper published at IEEE International Conference
on Data Mining (ICDM), 201
A Modular Task-oriented Dialogue System Using a Neural Mixture-of-Experts
End-to-end Task-oriented Dialogue Systems (TDSs) have attracted a lot of
attention for their superiority (e.g., in terms of global optimization) over
pipeline modularized TDSs. Previous studies on end-to-end TDSs use a
single-module model to generate responses for complex dialogue contexts.
However, no model consistently outperforms the others in all cases. We propose
a neural Modular Task-oriented Dialogue System(MTDS) framework, in which a few
expert bots are combined to generate the response for a given dialogue context.
MTDS consists of a chair bot and several expert bots. Each expert bot is
specialized for a particular situation, e.g., one domain, one type of action of
a system, etc. The chair bot coordinates multiple expert bots and adaptively
selects an expert bot to generate the appropriate response. We further propose
a Token-level Mixture-of-Expert (TokenMoE) model to implement MTDS, where the
expert bots predict multiple tokens at each timestamp and the chair bot
determines the final generated token by fully taking into consideration the
outputs of all expert bots. Both the chair bot and the expert bots are jointly
trained in an end-to-end fashion. To verify the effectiveness of TokenMoE, we
carry out extensive experiments on a benchmark dataset. Compared with the
baseline using a single-module model, our TokenMoE improves the performance by
8.1% of inform rate and 0.8% of success rate.Comment: Proceedings of the 2019 SIGIR Workshop WCIS: Workshop on
Conversational Interaction System
- …