173,331 research outputs found
Distribution of Mutual Information from Complete and Incomplete Data
Mutual information is widely used, in a descriptive way, to measure the
stochastic dependence of categorical random variables. In order to address
questions such as the reliability of the descriptive value, one must consider
sample-to-population inferential approaches. This paper deals with the
posterior distribution of mutual information, as obtained in a Bayesian
framework by a second-order Dirichlet prior distribution. The exact analytical
expression for the mean, and analytical approximations for the variance,
skewness and kurtosis are derived. These approximations have a guaranteed
accuracy level of the order O(1/n^3), where n is the sample size. Leading order
approximations for the mean and the variance are derived in the case of
incomplete samples. The derived analytical expressions allow the distribution
of mutual information to be approximated reliably and quickly. In fact, the
derived expressions can be computed with the same order of complexity needed
for descriptive mutual information. This makes the distribution of mutual
information become a concrete alternative to descriptive mutual information in
many applications which would benefit from moving to the inductive side. Some
of these prospective applications are discussed, and one of them, namely
feature selection, is shown to perform significantly better when inductive
mutual information is used.Comment: 26 pages, LaTeX, 5 figures, 4 table
Adversarial Imitation Learning from Incomplete Demonstrations
Imitation learning targets deriving a mapping from states to actions, a.k.a.
policy, from expert demonstrations. Existing methods for imitation learning
typically require any actions in the demonstrations to be fully available,
which is hard to ensure in real applications. Though algorithms for learning
with unobservable actions have been proposed, they focus solely on state
information and overlook the fact that the action sequence could still be
partially available and provide useful information for policy deriving. In this
paper, we propose a novel algorithm called Action-Guided Adversarial Imitation
Learning (AGAIL) that learns a policy from demonstrations with incomplete
action sequences, i.e., incomplete demonstrations. The core idea of AGAIL is to
separate demonstrations into state and action trajectories, and train a policy
with state trajectories while using actions as auxiliary information to guide
the training whenever applicable. Built upon the Generative Adversarial
Imitation Learning, AGAIL has three components: a generator, a discriminator,
and a guide. The generator learns a policy with rewards provided by the
discriminator, which tries to distinguish state distributions between
demonstrations and samples generated by the policy. The guide provides
additional rewards to the generator when demonstrated actions for specific
states are available. We compare AGAIL to other methods on benchmark tasks and
show that AGAIL consistently delivers comparable performance to the
state-of-the-art methods even when the action sequence in demonstrations is
only partially available.Comment: Accepted to International Joint Conference on Artificial Intelligence
(IJCAI-19
Raw-data attacks in quantum cryptography with partial tomography
We consider a variant of the BB84 protocol for quantum cryptography, the
prototype of tomographically incomplete protocols, where the key is generated
by one-way communication rather than the usual two-way communication. Our
analysis, backed by numerical evidence, establishes thresholds for
eavesdropping attacks on the raw data and on the generated key at quantum bit
error rates of 10% and 6.15%, respectively. Both thresholds are lower than the
threshold for unconditional security in the standard BB84 protocol.Comment: 11 pages, 2 figure
Robust Feature Selection by Mutual Information Distributions
Mutual information is widely used in artificial intelligence, in a
descriptive way, to measure the stochastic dependence of discrete random
variables. In order to address questions such as the reliability of the
empirical value, one must consider sample-to-population inferential approaches.
This paper deals with the distribution of mutual information, as obtained in a
Bayesian framework by a second-order Dirichlet prior distribution. The exact
analytical expression for the mean and an analytical approximation of the
variance are reported. Asymptotic approximations of the distribution are
proposed. The results are applied to the problem of selecting features for
incremental learning and classification of the naive Bayes classifier. A fast,
newly defined method is shown to outperform the traditional approach based on
empirical mutual information on a number of real data sets. Finally, a
theoretical development is reported that allows one to efficiently extend the
above methods to incomplete samples in an easy and effective way.Comment: 8 two-column page
Measuring the Knowledge-Based Economy of China in terms of Synergy among Technological, Organizational, and Geographic Attributes of Firms
Using the possible synergy among geographic, size, and technological
distributions of firms in the Orbis database, we find the greatest reduction of
uncertainty at the level of the 31 provinces of China, and an additional 18.0%
at the national level. Some of the coastal provinces stand out as expected, but
the metropolitan areas of Beijing and Shanghai are (with Tianjan and Chonqing)
most pronounced at the next-lower administrative level of (339) prefectures,
since these four metropoles are administratively defined at both levels.
Focusing on high- and medium-tech manufacturing, a shift toward Beijing and
Shanghai is indicated, and the synergy is on average enhanced (as expected; but
not for all provinces). Unfortunately, the Orbis data is incomplete since it
was collected for commercial and not for administrative or governmental
purposes. However, we show a methodology that can be used by others who may
have access to higher-quality statistical data for the measurement.Comment: accepted for publication in Scientometrics (October, 2013
Robust Inference of Trees
This paper is concerned with the reliable inference of optimal
tree-approximations to the dependency structure of an unknown distribution
generating data. The traditional approach to the problem measures the
dependency strength between random variables by the index called mutual
information. In this paper reliability is achieved by Walley's imprecise
Dirichlet model, which generalizes Bayesian learning with Dirichlet priors.
Adopting the imprecise Dirichlet model results in posterior interval
expectation for mutual information, and in a set of plausible trees consistent
with the data. Reliable inference about the actual tree is achieved by focusing
on the substructure common to all the plausible trees. We develop an exact
algorithm that infers the substructure in time O(m^4), m being the number of
random variables. The new algorithm is applied to a set of data sampled from a
known distribution. The method is shown to reliably infer edges of the actual
tree even when the data are very scarce, unlike the traditional approach.
Finally, we provide lower and upper credibility limits for mutual information
under the imprecise Dirichlet model. These enable the previous developments to
be extended to a full inferential method for trees.Comment: 26 pages, 7 figure
Electrostatic Field Classifier for Deficient Data
This paper investigates the suitability of recently developed models based on the physical
field phenomena for classification problems with incomplete datasets. An original approach
to exploiting incomplete training data with missing features and labels, involving extensive use
of electrostatic charge analogy, has been proposed. Classification of incomplete patterns has been
investigated using a local dimensionality reduction technique, which aims at exploiting all available
information rather than trying to estimate the missing values. The performance of all proposed
methods has been tested on a number of benchmark datasets for a wide range of missing data scenarios
and compared to the performance of some standard techniques. Several modifications of the
original electrostatic field classifier aiming at improving speed and robustness in higher dimensional
spaces are also discussed
- …