1,335 research outputs found
Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey
Wireless sensor networks (WSNs) consist of autonomous and resource-limited
devices. The devices cooperate to monitor one or more physical phenomena within
an area of interest. WSNs operate as stochastic systems because of randomness
in the monitored environments. For long service time and low maintenance cost,
WSNs require adaptive and robust methods to address data exchange, topology
formulation, resource and power optimization, sensing coverage and object
detection, and security challenges. In these problems, sensor nodes are to make
optimized decisions from a set of accessible strategies to achieve design
goals. This survey reviews numerous applications of the Markov decision process
(MDP) framework, a powerful decision-making tool to develop adaptive algorithms
and protocols for WSNs. Furthermore, various solution methods are discussed and
compared to serve as a guide for using MDPs in WSNs
The Cognitive Compressive Sensing Problem
In the Cognitive Compressive Sensing (CCS) problem, a Cognitive Receiver (CR)
seeks to optimize the reward obtained by sensing an underlying dimensional
random vector, by collecting at most arbitrary projections of it. The
components of the latent vector represent sub-channels states, that change
dynamically from "busy" to "idle" and vice versa, as a Markov chain that is
biased towards producing sparse vectors. To identify the optimal strategy we
formulate the Multi-Armed Bandit Compressive Sensing (MAB-CS) problem,
generalizing the popular Cognitive Spectrum Sensing model, in which the CR can
sense out of the sub-channels, as well as the typical static setting of
Compressive Sensing, in which the CR observes linear combinations of the
dimensional sparse vector. The CR opportunistic choice of the sensing
matrix should balance the desire of revealing the state of as many dimensions
of the latent vector as possible, while not exceeding the limits beyond which
the vector support is no longer uniquely identifiable.Comment: 8 pages, 2 figure
Active MR k-space Sampling with Reinforcement Learning
Deep learning approaches have recently shown great promise in accelerating
magnetic resonance image (MRI) acquisition. The majority of existing work have
focused on designing better reconstruction models given a pre-determined
acquisition trajectory, ignoring the question of trajectory optimization. In
this paper, we focus on learning acquisition trajectories given a fixed image
reconstruction model. We formulate the problem as a sequential decision process
and propose the use of reinforcement learning to solve it. Experiments on a
large scale public MRI dataset of knees show that our proposed models
significantly outperform the state-of-the-art in active MRI acquisition, over a
large range of acceleration factors.Comment: Presented at the 23rd International Conference on Medical Image
Computing and Computer Assisted Intervention, MICCAI 202
Resource-Constrained Adaptive Search and Tracking for Sparse Dynamic Targets
This paper considers the problem of resource-constrained and noise-limited
localization and estimation of dynamic targets that are sparsely distributed
over a large area. We generalize an existing framework [Bashan et al, 2008] for
adaptive allocation of sensing resources to the dynamic case, accounting for
time-varying target behavior such as transitions to neighboring cells and
varying amplitudes over a potentially long time horizon. The proposed adaptive
sensing policy is driven by minimization of a modified version of the
previously introduced ARAP objective function, which is a surrogate function
for mean squared error within locations containing targets. We provide
theoretical upper bounds on the performance of adaptive sensing policies by
analyzing solutions with oracle knowledge of target locations, gaining insight
into the effect of target motion and amplitude variation as well as sparsity.
Exact minimization of the multi-stage objective function is infeasible, but
myopic optimization yields a closed-form solution. We propose a simple
non-myopic extension, the Dynamic Adaptive Resource Allocation Policy (D-ARAP),
that allocates a fraction of resources for exploring all locations rather than
solely exploiting the current belief state. Our numerical studies indicate that
D-ARAP has the following advantages: (a) it is more robust than the myopic
policy to noise, missing data, and model mismatch; (b) it performs comparably
to well-known approximate dynamic programming solutions but at significantly
lower computational complexity; and (c) it improves greatly upon non-adaptive
uniform resource allocation in terms of estimation error and probability of
detection.Comment: 49 pages, 1 table, 11 figure
A Unified Multi-Functional Dynamic Spectrum Access Framework: Tutorial, Theory and Multi-GHz Wideband Testbed
Dynamic spectrum access is a must-have ingredient for future sensors that are ideally cognitive. The goal of this paper is a tutorial treatment of wideband cognitive radio and radar—a convergence of (1) algorithms survey, (2) hardware platforms survey, (3) challenges for multi-function (radar/communications) multi-GHz front end, (4) compressed sensing for multi-GHz waveforms—revolutionary A/D, (5) machine learning for cognitive radio/radar, (6) quickest detection, and (7) overlay/underlay cognitive radio waveforms. One focus of this paper is to address the multi-GHz front end, which is the challenge for the next-generation cognitive sensors. The unifying theme of this paper is to spell out the convergence for cognitive radio, radar, and anti-jamming. Moore’s law drives the system functions into digital parts. From a system viewpoint, this paper gives the first comprehensive treatment for the functions and the challenges of this multi-function (wideband) system. This paper brings together the inter-disciplinary knowledge
- …