464 research outputs found
Learning to select data for transfer learning with Bayesian Optimization
Domain similarity measures can be used to gauge adaptability and select
suitable data for transfer learning, but existing approaches define ad hoc
measures that are deemed suitable for respective tasks. Inspired by work on
curriculum learning, we propose to \emph{learn} data selection measures using
Bayesian Optimization and evaluate them across models, domains and tasks. Our
learned measures outperform existing domain similarity measures significantly
on three tasks: sentiment analysis, part-of-speech tagging, and parsing. We
show the importance of complementing similarity with diversity, and that
learned measures are -- to some degree -- transferable across models, domains,
and even tasks.Comment: EMNLP 2017. Code available at:
https://github.com/sebastianruder/learn-to-select-dat
A Simple and Tighter Derivation of Achievability for Classical Communication over Quantum Channels
Achievability in information theory refers to demonstrating a coding strategy
that accomplishes a prescribed performance benchmark for the underlying task.
In quantum information theory, the crafted Hayashi-Nagaoka operator inequality
is an essential technique in proving a wealth of one-shot achievability bounds
since it effectively resembles a union bound in various problems. In this work,
we show that the pretty-good measurement naturally plays a role as the union
bound as well. A judicious application of it considerably simplifies the
derivation of one-shot achievability for classical-quantum (c-q) channel coding
via an elegant three-line proof.
The proposed analysis enjoys the following favorable features: (i) The
established one-shot bound admits a closed-form expression as in the celebrated
Holevo-Helstrom Theorem. Namely, the average error probability of sending
messages through a c-q channel is upper bounded by the error of distinguishing
the joint state between channel input and output against -many products
of its marginals. (ii) Our bound directly yields asymptotic results in the
large deviation, small deviation, and moderate deviation regimes in a unified
manner. (iii) The coefficients incurred in applying the Hayashi-Nagaoka
operator inequality are no longer needed. Hence, the derived one-shot bound
sharpens existing results that rely on the Hayashi-Nagaoka operator inequality.
In particular, we obtain the tightest achievable -one-shot capacity
for c-q channel heretofore, and it improves the third-order coding rate in the
asymptotic scenario. (iv) Our result holds for infinite-dimensional Hilbert
space. (v) The proposed method applies to deriving one-shot bounds for data
compression with quantum side information, entanglement-assisted classical
communication over quantum channels, and various quantum network
information-processing protocols
- …