24 research outputs found
Sequentiality and Adaptivity Gains in Active Hypothesis Testing
Consider a decision maker who is responsible to collect observations so as to
enhance his information in a speedy manner about an underlying phenomena of
interest. The policies under which the decision maker selects sensing actions
can be categorized based on the following two factors: i) sequential vs.
non-sequential; ii) adaptive vs. non-adaptive. Non-sequential policies collect
a fixed number of observation samples and make the final decision afterwards;
while under sequential policies, the sample size is not known initially and is
determined by the observation outcomes. Under adaptive policies, the decision
maker relies on the previous collected samples to select the next sensing
action; while under non-adaptive policies, the actions are selected independent
of the past observation outcomes.
In this paper, performance bounds are provided for the policies in each
category. Using these bounds, sequentiality gain and adaptivity gain, i.e., the
gains of sequential and adaptive selection of actions are characterized.Comment: 12 double-column pages, 1 figur
On the Capacity of the Noncausal Relay Channel
This paper studies the noncausal relay channel, also known as the relay
channel with unlimited lookahead, introduced by El Gamal, Hassanpour, and
Mammen. Unlike the standard relay channel model, where the relay encodes its
signal based on the previous received output symbols, the relay in the
noncausal relay channel encodes its signal as a function of the entire received
sequence. In the existing coding schemes, the relay uses this noncausal
information solely to recover the transmitted message and then cooperates with
the sender to communicate this message to the receiver. However, it is shown in
this paper that by applying the Gelfand--Pinsker coding scheme, the relay can
take further advantage of the noncausally available information, which can
achieve strictly higher rates than existing coding schemes. This paper also
provides a new upper bound on the capacity of the noncausal relay that strictly
improves upon the cutset bound. These new lower and upper bounds on the
capacity coincide for the class of degraded noncausal relay channels and
establish the capacity for this class.Comment: To appear in the IEEE Transactions on Information Theor
Active sequential hypothesis testing
Consider a decision maker who is responsible to dynamically collect
observations so as to enhance his information about an underlying phenomena of
interest in a speedy manner while accounting for the penalty of wrong
declaration. Due to the sequential nature of the problem, the decision maker
relies on his current information state to adaptively select the most
``informative'' sensing action among the available ones. In this paper, using
results in dynamic programming, lower bounds for the optimal total cost are
established. The lower bounds characterize the fundamental limits on the
maximum achievable information acquisition rate and the optimal reliability.
Moreover, upper bounds are obtained via an analysis of two heuristic policies
for dynamic selection of actions. It is shown that the first proposed heuristic
achieves asymptotic optimality, where the notion of asymptotic optimality, due
to Chernoff, implies that the relative difference between the total cost
achieved by the proposed policy and the optimal total cost approaches zero as
the penalty of wrong declaration (hence the number of collected samples)
increases. The second heuristic is shown to achieve asymptotic optimality only
in a limited setting such as the problem of a noisy dynamic search. However, by
considering the dependency on the number of hypotheses, under a technical
condition, this second heuristic is shown to achieve a nonzero information
acquisition rate, establishing a lower bound for the maximum achievable rate
and error exponent. In the case of a noisy dynamic search with size-independent
noise, the obtained nonzero rate and error exponent are shown to be maximum.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1144 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Extrinsic Jensen-Shannon Divergence: Applications to Variable-Length Coding
This paper considers the problem of variable-length coding over a discrete
memoryless channel (DMC) with noiseless feedback. The paper provides a
stochastic control view of the problem whose solution is analyzed via a newly
proposed symmetrized divergence, termed extrinsic Jensen-Shannon (EJS)
divergence. It is shown that strictly positive lower bounds on EJS divergence
provide non-asymptotic upper bounds on the expected code length. The paper
presents strictly positive lower bounds on EJS divergence, and hence
non-asymptotic upper bounds on the expected code length, for the following two
coding schemes: variable-length posterior matching and MaxEJS coding scheme
which is based on a greedy maximization of the EJS divergence.
As an asymptotic corollary of the main results, this paper also provides a
rate-reliability test. Variable-length coding schemes that satisfy the
condition(s) of the test for parameters and , are guaranteed to achieve
rate and error exponent . The results are specialized for posterior
matching and MaxEJS to obtain deterministic one-phase coding schemes achieving
capacity and optimal error exponent. For the special case of symmetric
binary-input channels, simpler deterministic schemes of optimal performance are
proposed and analyzed.Comment: 17 pages (two-column), 4 figures, to appear in IEEE Transactions on
Information Theor
A General Class of Throughput Optimal Routing Policies in Multi-hop Wireless Networks
This paper considers the problem of throughput optimal routing/scheduling in
a multi-hop constrained queueing network with random connectivity whose special
case includes opportunistic multi-hop wireless networks and input-queued switch
fabrics. The main challenge in the design of throughput optimal routing
policies is closely related to identifying appropriate and universal Lyapunov
functions with negative expected drift. The few well-known throughput optimal
policies in the literature are constructed using simple quadratic or
exponential Lyapunov functions of the queue backlogs and as such they seek to
balance the queue backlogs across network independent of the topology. By
considering a class of continuous, differentiable, and piece-wise quadratic
Lyapunov functions, this paper provides a large class of throughput optimal
routing policies. The proposed class of Lyapunov functions allow for the
routing policy to control the traffic along short paths for a large portion of
state-space while ensuring a negative expected drift. This structure enables
the design of a large class of routing policies. In particular, and in addition
to recovering the throughput optimality of the well known backpressure routing
policy, an opportunistic routing policy with congestion diversity is proved to
be throughput optimal.Comment: 31 pages (one column), 8 figures, (revision submitted to IEEE
Transactions on Information Theory
Active Learning and Hypothesis Testing
This dissertation considers a generalization of the classical hypothesis testing problem. Suppose there are M hypotheses of interest among which only one is true. A Bayesian decision maker is responsible to collect observation samples so as to enhance his information about the true hypothesis in a speedy manner while accounting for the penalty of wrong declaration. In contrast to the classical hypothesis testing problem, at any given time, the decision maker can choose one of the available sensing actions and hence, exert some control over the collected samples' "information content." This generalization, referred to as the active hypothesis testing, naturally arises in a broad spectrum of applications such as medical diagnosis, cognition, communication, sensor management, image inspection, generalized search, and group testing. The first part of the dissertation provides a theoretical analysis of the problem of active hypothesis testing. Using results in sequential analysis and dynamic programming, lower bounds for the optimal performance are established. The lower bounds are complementary for various values of the parameters of the problem, and characterize the fundamental limits on the maximum achievable information acquisition rate and the optimal reliability. Moreover, upper bounds are obtained via an analysis of the proposed heuristic policies for dynamic selection of actions. From the obtained bounds, sufficient conditions are provided under which the maximum information acquisition rate and reliability are achieved, establishing the asymptotic optimality of the proposed heuristics. The second part of the dissertation investigates the applications of the first part for three important special cases of the active hypothesis testing. Chapter 5 considers the problem of conveying a message over discrete memoryless channels with noiseless feedback. Chapter 6 studies the problem of two-dimensional search to locate a target in an image against a background of distractors. Finally, in Chapter 7, the problem of active learning for multiclass classification is investigated where the outcomes of label queries are corrupted by noise. In each of these chapters, the results in the first part of the dissertation are specialized, new results are obtained, and many of the known results are recovered with concise proof
Optimal Reliability over a Class of Binary-Input Channels with Feedback (Invited)
International audienc