14,053 research outputs found
Justification of Logarithmic Loss via the Benefit of Side Information
We consider a natural measure of relevance: the reduction in optimal
prediction risk in the presence of side information. For any given loss
function, this relevance measure captures the benefit of side information for
performing inference on a random variable under this loss function. When such a
measure satisfies a natural data processing property, and the random variable
of interest has alphabet size greater than two, we show that it is uniquely
characterized by the mutual information, and the corresponding loss function
coincides with logarithmic loss. In doing so, our work provides a new
characterization of mutual information, and justifies its use as a measure of
relevance. When the alphabet is binary, we characterize the only admissible
forms the measure of relevance can assume while obeying the specified data
processing property. Our results naturally extend to measuring causal influence
between stochastic processes, where we unify different causal-inference
measures in the literature as instantiations of directed information
Online Learning of k-CNF Boolean Functions
This paper revisits the problem of learning a k-CNF Boolean function from
examples in the context of online learning under the logarithmic loss. In doing
so, we give a Bayesian interpretation to one of Valiant's celebrated PAC
learning algorithms, which we then build upon to derive two efficient, online,
probabilistic, supervised learning algorithms for predicting the output of an
unknown k-CNF Boolean function. We analyze the loss of our methods, and show
that the cumulative log-loss can be upper bounded, ignoring logarithmic
factors, by a polynomial function of the size of each example.Comment: 20 LaTeX pages. 2 Algorithms. Some Theorem
On the Information Bottleneck Problems: An Information Theoretic Perspective
International Zurich Seminar on Information and Communication (IZS), February 26 – 28, 202
Data Offloading in Load Coupled Networks: A Utility Maximization Framework
We provide a general framework for the problem of data offloading in a
heterogeneous wireless network, where some demand of cellular users is served
by a complementary network. The complementary network is either a small-cell
network that shares the same resources as the cellular network, or a WiFi
network that uses orthogonal resources. For a given demand served in a cellular
network, the load, or the level of resource usage, of each cell depends in a
non-linear manner on the load of other cells due to the mutual coupling of
interference seen by one another. With load coupling, we optimize the demand to
be served in the cellular or the complementary networks, so as to maximize a
utility function. We consider three representative utility functions that
balance, to varying degrees, the revenue from serving the users vs the user
fairness. We establish conditions for which the optimization problem has a
feasible solution and is convex, and hence tractable to numerical computations.
Finally, we propose a strategy with theoretical justification to constrain the
load to some maximum value, as required for practical implementation. Numerical
studies are conducted for both under-loaded and over-loaded networks.Comment: 12 pages, accepted for publication in IEEE Transactions on Wireless
Communication
- …