4 research outputs found
A Unifying Framework of Bilinear LSTMs
This paper presents a novel unifying framework of bilinear LSTMs that can
represent and utilize the nonlinear interaction of the input features present
in sequence datasets for achieving superior performance over a linear LSTM and
yet not incur more parameters to be learned. To realize this, our unifying
framework allows the expressivity of the linear vs. bilinear terms to be
balanced by correspondingly trading off between the hidden state vector size
vs. approximation quality of the weight matrix in the bilinear term so as to
optimize the performance of our bilinear LSTM, while not incurring more
parameters to be learned. We empirically evaluate the performance of our
bilinear LSTM in several language-based sequence learning tasks to demonstrate
its general applicability
Hessian-Aware Bayesian Optimization for Decision Making Systems
Many approaches for optimizing decision making systems rely on gradient based
methods requiring informative feedback from the environment. However, in the
case where such feedback is sparse or uninformative, such approaches may result
in poor performance. Derivative-free approaches such as Bayesian Optimization
mitigate the dependency on the quality of gradient feedback, but are known to
scale poorly in the high-dimension setting of complex decision making systems.
This problem is exacerbated if the system requires interactions between several
actors cooperating to accomplish a shared goal. To address the dimensionality
challenge, we propose a compact multi-layered architecture modeling the
dynamics of actor interactions through the concept of role. Additionally, we
introduce Hessian-aware Bayesian Optimization to efficiently optimize the
multi-layered architecture parameterized by a large number of parameters.
Experimental results demonstrate that our method (HA-GP-UCB) works effectively
on several benchmarks under resource constraints and malformed feedback
settings.Comment: Included important citatio