Algebraic synchronization criterion and computing reset words

We refine a uniform algebraic approach for deriving upper bounds on reset thresholds of synchronizing automata. We express the condition that an automaton is synchronizing in terms of linear algebra, and obtain upper bounds for the reset thresholds of automata with a short word of a small rank. The results are applied to make several improvements in the area. We improve the best general upper bound for reset thresholds of finite prefix codes (Huffman codes): we show that an $n$-state synchronizing decoder has a reset word of length at most $O(n \log^3 n)$. In addition to that, we prove that the expected reset threshold of a uniformly random synchronizing binary $n$-state decoder is at most $O(n \log n)$. We also show that for any non-unary alphabet there exist decoders whose reset threshold is in $\varTheta(n)$. We prove the \v{C}ern\'{y} conjecture for $n$-state automata with a letter of rank at most $\sqrt[3]{6n-6}$. In another corollary, based on the recent results of Nicaud, we show that the probability that the \v{C}ern\'y conjecture does not hold for a random synchronizing binary automaton is exponentially small in terms of the number of states, and also that the expected value of the reset threshold of an $n$-state random synchronizing binary automaton is at most $n^{3/2+o(1)}$. Moreover, reset words of lengths within all of our bounds are computable in polynomial time. We present suitable algorithms for this task for various classes of automata, such as (quasi-)one-cluster and (quasi-)Eulerian automata, for which our results can be applied.Comment: 18 pages, 2 figure

Truncating Temporal Differences: On the Efficient Implementation of TD(lambda) for Reinforcement Learning

Temporal difference (TD) methods constitute a class of methods for learning predictions in multi-step prediction problems, parameterized by a recency factor lambda. Currently the most important application of these methods is to temporal credit assignment in reinforcement learning. Well known reinforcement learning algorithms, such as AHC or Q-learning, may be viewed as instances of TD learning. This paper examines the issues of the efficient and general implementation of TD(lambda) for arbitrary lambda, for use with reinforcement learning algorithms optimizing the discounted sum of rewards. The traditional approach, based on eligibility traces, is argued to suffer from both inefficiency and lack of generality. The TTD (Truncated Temporal Differences) procedure is proposed as an alternative, that indeed only approximates TD(lambda), but requires very little computation per action and can be used with arbitrary function representation methods. The idea from which it is derived is fairly simple and not new, but probably unexplored so far. Encouraging experimental results are presented, suggesting that using lambda &gt 0 with the TTD procedure allows one to obtain a significant learning speedup at essentially the same cost as usual TD(0) learning.Comment: See http://www.jair.org/ for any accompanying file

Seeding with Costly Network Information

We study the task of selecting $k$ nodes in a social network of size $n$, to seed a diffusion with maximum expected spread size, under the independent cascade model with cascade probability $p$. Most of the previous work on this problem (known as influence maximization) focuses on efficient algorithms to approximate the optimal seed set with provable guarantees, given the knowledge of the entire network. However, in practice, obtaining full knowledge of the network is very costly. To address this gap, we first study the achievable guarantees using $o(n)$ influence samples. We provide an approximation algorithm with a tight (1-1/e){\mbox{OPT}}-\epsilon n guarantee, using $O_{\epsilon}(k^2\log n)$ influence samples and show that this dependence on $k$ is asymptotically optimal. We then propose a probing algorithm that queries ${O}_{\epsilon}(p n^2\log^4 n + \sqrt{k p} n^{1.5}\log^{5.5} n + k n\log^{3.5}{n})$ edges from the graph and use them to find a seed set with the same almost tight approximation guarantee. We also provide a matching (up to logarithmic factors) lower-bound on the required number of edges. To address the dependence of our probing algorithm on the independent cascade probability $p$, we show that it is impossible to maintain the same approximation guarantees by controlling the discrepancy between the probing and seeding cascade probabilities. Instead, we propose to down-sample the probed edges to match the seeding cascade probability, provided that it does not exceed that of probing. Finally, we test our algorithms on real world data to quantify the trade-off between the cost of obtaining more refined network information and the benefit of the added information for guiding improved seeding strategies