Convergence and Convergence Rate of Stochastic Gradient Search in the Case of Multiple and Non-Isolated Extrema

The asymptotic behavior of stochastic gradient algorithms is studied. Relying on results from differential geometry (Lojasiewicz gradient inequality), the single limit-point convergence of the algorithm iterates is demonstrated and relatively tight bounds on the convergence rate are derived. In sharp contrast to the existing asymptotic results, the new results presented here allow the objective function to have multiple and non-isolated minima. The new results also offer new insights into the asymptotic properties of several classes of recursive algorithms which are routinely used in engineering, statistics, machine learning and operations research

Packet transport on scale free networks

We introduce a model of information packet transport on networks in which the packets are posted by a given rate and move in parallel according to a local search algorithm. By performing a number of simulations we investigate the major kinetic properties of the transport as a function of the network geometry, the packet input rate and the buffer size. We find long-range correlations in the power spectra of arriving packet density and the network's activity bursts. The packet transit time distribution shows a power-law dependence with average transit time increasing with network size. This implies dynamic queueing on the network, in which many interacting queues are mutually driven by temporally correlated packet stream

Asymptotic Bias of Stochastic Gradient Search

The asymptotic behavior of the stochastic gradient algorithm with a biased gradient estimator is analyzed. Relying on arguments based on the dynamic system theory (chain-recurrence) and the differential geometry (Yomdin theorem and Lojasiewicz inequality), tight bounds on the asymptotic bias of the iterates generated by such an algorithm are derived. The obtained results hold under mild conditions and cover a broad class of high-dimensional nonlinear algorithms. Using these results, the asymptotic properties of the policy-gradient (reinforcement) learning and adaptive population Monte Carlo sampling are studied. Relying on the same results, the asymptotic behavior of the recursive maximum split-likelihood estimation in hidden Markov models is analyzed, too.Comment: arXiv admin note: text overlap with arXiv:0907.102

Preferential Behaviour and Scaling in Diffusive Dynamics on Networks

We study the fluctuation properties and return-time statistics on inhomogeneous scale-free networks using packets moving with two different dynamical rules; random diffusion and locally navigated diffusive motion with preferred edges. Scaling in the fluctuations occurs when the dispersion of a quantity at each node or edge increases like the its mean to the power $\mu$. We show that the occurrence of scaling in the fluctuations of both the number of packets passing nodes and the number flowing along edges is related to preferential behaviour in either the topology (in the case of nodes) or in the dynamics (in case the of edges). Within our model the absence of any preference leads to the absence of scaling, and when scaling occurs it is non-universal; for random diffusion the number of packets passing a node scales with an exponent $\mu$ which increases continuously with increased acquisition time window from $\mu =1/2$ at small windows, to $\mu =1$ at long time windows; In the preferentially navigated diffusive motion, busy nodes and edges have exponent $\mu =1$, in contrast to less busy parts of the network, where an exponent $\mu =1/2$ is found. Broad distributions of the return times at nodes and edges illustrate that the basis of the observed scaling is the cooperative behaviour between groups of nodes or edges. These conclusions are relevant for a large class of diffusive dynamics on networks, including packet transport with local navigation rules

Bias of Particle Approximations to Optimal Filter Derivative

In many applications, a state-space model depends on a parameter which needs to be inferred from a data set. Quite often, it is necessary to perform the parameter inference online. In the maximum likelihood approach, this can be done using stochastic gradient search and the optimal filter derivative. However, the optimal filter and its derivative are not analytically tractable for a non-linear state-space model and need to be approximated numerically. In [Poyiadjis, Doucet and Singh, Biometrika 2011], a particle approximation to the optimal filter derivative has been proposed, while the corresponding $L_{p}$ error bonds and the central limit theorem have been provided in [Del Moral, Doucet and Singh, SIAM Journal on Control and Optimization 2015]. Here, the bias of this particle approximation is analyzed. We derive (relatively) tight bonds on the bias in terms of the number of particles. Under (strong) mixing conditions, the bounds are uniform in time and inversely proportional to the number of particles. The obtained results apply to a (relatively) broad class of state-space models met in practice

Analyticity of Entropy Rates of Continuous-State Hidden Markov Models

The analyticity of the entropy and relative entropy rates of continuous-state hidden Markov models is studied here. Using the analytic continuation principle and the stability properties of the optimal filter, the analyticity of these rates is shown for analytically parameterized models. The obtained results hold under relatively mild conditions and cover several classes of hidden Markov models met in practice. These results are relevant for several (theoretically and practically) important problems arising in statistical inference, system identification and information theory

Fractal properties of relaxation clusters and phase transition in a stochastic sandpile automaton

We study numerically the spatial properties of relaxation clusters in a two dimensional sandpile automaton with dynamic rules depending stochastically on a parameter p, which models the effects of static friction. In the limiting cases p=1 and p=0 the model reduces to the critical height model and critical slope model, respectively. At p=p_c, a continuous phase transition occurs to the state characterized by a nonzero average slope. Our analysis reveals that the loss of finite average slope at the transition is accompanied by the loss of fractal properties of the relaxation clusters.Comment: 11 page

Modeling latent infection transmissions through biosocial stochastic dynamics

The events of the recent SARS-CoV-2 epidemics have shown the importance of social factors, especially given the large number of asymptomatic cases that effectively spread the virus, which can cause a medical emergency to very susceptible individuals. Besides, the SARS-CoV-2 virus survives for several hours on different surfaces, where a new host can contract it with a delay. These passive modes of infection transmission remain an unexplored area for traditional mean-field epidemic models. Here, we design an agent-based model for simulations of infection transmission in an open system driven by the dynamics of social activity; the model takes into account the personal characteristics of individuals, as well as the survival time of the virus and its potential mutations. A growing bipartite graph embodies this biosocial process, consisting of active carriers (host) nodes that produce viral nodes during their infectious period. With its directed edges passing through viral nodes between two successive hosts, this graph contains complete information about the routes leading to each infected individual. We determine temporal fluctuations of the number of exposed and the number of infected individuals, the number of active carriers and active viruses at hourly resolution. The simulated processes underpin the latent infection transmissions, contributing significantly to the spread of the virus within a large time window. More precisely, being brought by social dynamics and exposed to the currently existing infection, an individual passes through the infectious state until eventually spontaneously recovers or otherwise is moves to a controlled hospital environment. Our results reveal complex feedback mechanisms that shape the dependence of the infection curve on the intensity of social dynamics and other sociobiological factors. In particular, the results show how the lockdown effectively reduces the spread of infection and how it increases again after the lockdown is removed. Furthermore, a reduced level of social activity but prolonged exposure of susceptible individuals have adverse effects. On the other hand, virus mutations that can gradually reduce the transmission rate by hopping to each new host along the infection path can significantly reduce the extent of the infection, but can not stop the spreading without additional social strategies. Our stochastic processes, based on graphs at the interface of biology and social dynamics, provide a new mathematical framework for simulations of various epidemic control strategies with high temporal resolution and virus traceability