4 research outputs found
Learning Zero-Sum Linear Quadratic Games with Improved Sample Complexity
Zero-sum Linear Quadratic (LQ) games are fundamental in optimal control and
can be used (i) as a dynamic game formulation for risk-sensitive or robust
control, or (ii) as a benchmark setting for multi-agent reinforcement learning
with two competing agents in continuous state-control spaces. In contrast to
the well-studied single-agent linear quadratic regulator problem, zero-sum LQ
games entail solving a challenging nonconvex-nonconcave min-max problem with an
objective function that lacks coercivity. Recently, Zhang et al. discovered an
implicit regularization property of natural policy gradient methods which is
crucial for safety-critical control systems since it preserves the robustness
of the controller during learning. Moreover, in the model-free setting where
the knowledge of model parameters is not available, Zhang et al. proposed the
first polynomial sample complexity algorithm to reach an
-neighborhood of the Nash equilibrium while maintaining the desirable
implicit regularization property. In this work, we propose a simpler nested
Zeroth-Order (ZO) algorithm improving sample complexity by several orders of
magnitude. Our main result guarantees a
sample complexity under the same
assumptions using a single-point ZO estimator. Furthermore, when the estimator
is replaced by a two-point estimator, our method enjoys a better
sample complexity. Our key
improvements rely on a more sample-efficient nested algorithm design and finer
control of the ZO natural gradient estimation error
GNN-encoder: Learning a Dual-encoder Architecture via Graph Neural Networks for Passage Retrieval
Recently, retrieval models based on dense representations are dominant in
passage retrieval tasks, due to their outstanding ability in terms of capturing
semantics of input text compared to the traditional sparse vector space models.
A common practice of dense retrieval models is to exploit a dual-encoder
architecture to represent a query and a passage independently. Though
efficient, such a structure loses interaction between the query-passage pair,
resulting in inferior accuracy. To enhance the performance of dense retrieval
models without loss of efficiency, we propose a GNN-encoder model in which
query (passage) information is fused into passage (query) representations via
graph neural networks that are constructed by queries and their top retrieved
passages. By this means, we maintain a dual-encoder structure, and retain some
interaction information between query-passage pairs in their representations,
which enables us to achieve both efficiency and efficacy in passage retrieval.
Evaluation results indicate that our method significantly outperforms the
existing models on MSMARCO, Natural Questions and TriviaQA datasets, and
achieves the new state-of-the-art on these datasets.Comment: 11 pages, 6 figure