We are motivated by the need, in some applications, for impromptu or
as-you-go deployment of wireless sensor networks. A person walks along a line,
starting from a sink node (e.g., a base-station), and proceeds towards a source
node (e.g., a sensor) which is at an a priori unknown location. At equally
spaced locations, he makes link quality measurements to the previous relay, and
deploys relays at some of these locations, with the aim to connect the source
to the sink by a multihop wireless path. In this paper, we consider two
approaches for impromptu deployment: (i) the deployment agent can only move
forward (which we call a pure as-you-go approach), and (ii) the deployment
agent can make measurements over several consecutive steps before selecting a
placement location among them (which we call an explore-forward approach). We
consider a light traffic regime, and formulate the problem as a Markov decision
process, where the trade-off is among the power used by the nodes, the outage
probabilities in the links, and the number of relays placed per unit distance.
We obtain the structures of the optimal policies for the pure as-you-go
approach as well as for the explore-forward approach. We also consider natural
heuristic algorithms, for comparison. Numerical examples show that the
explore-forward approach significantly outperforms the pure as-you-go approach.
Next, we propose two learning algorithms for the explore-forward approach,
based on Stochastic Approximation, which asymptotically converge to the set of
optimal policies, without using any knowledge of the radio propagation model.
We demonstrate numerically that the learning algorithms can converge (as
deployment progresses) to the set of optimal policies reasonably fast and,
hence, can be practical, model-free algorithms for deployment over large
regions.Comment: 29 pages. arXiv admin note: text overlap with arXiv:1308.068
