A Weighted Correlation Index for Rankings with Ties

Understanding the correlation between two different scores for the same set of items is a common problem in information retrieval, and the most commonly used statistics that quantifies this correlation is Kendall's $\tau$. However, the standard definition fails to capture that discordances between items with high rank are more important than those between items with low rank. Recently, a new measure of correlation based on average precision has been proposed to solve this problem, but like many alternative proposals in the literature it assumes that there are no ties in the scores. This is a major deficiency in a number of contexts, and in particular while comparing centrality scores on large graphs, as the obvious baseline, indegree, has a very large number of ties in web and social graphs. We propose to extend Kendall's definition in a natural way to take into account weights in the presence of ties. We prove a number of interesting mathematical properties of our generalization and describe an $O(n\log n)$ algorithm for its computation. We also validate the usefulness of our weighted measure of correlation using experimental data

Spectral coarse graining for random walk in bipartite networks

Many real-world networks display a natural bipartite structure, while analyzing or visualizing large bipartite networks is one of the most challenges. As a result, it is necessary to reduce the complexity of large bipartite systems and preserve the functionality at the same time. We observe, however, the existing coarse graining methods for binary networks fail to work in the bipartite networks. In this paper, we use the spectral analysis to design a coarse graining scheme specifically for bipartite networks and keep their random walk properties unchanged. Numerical analysis on artificial and real-world bipartite networks indicates that our coarse graining scheme could obtain much smaller networks from large ones, keeping most of the relevant spectral properties. Finally, we further validate the coarse graining method by directly comparing the mean first passage time between the original network and the reduced one.Comment: 7 pages, 3 figure

Quantum Operation Time Reversal

The dynamics of an open quantum system can be described by a quantum operation, a linear, complete positive map of operators. Here, I exhibit a compact expression for the time reversal of a quantum operation, which is closely analogous to the time reversal of a classical Markov transition matrix. Since open quantum dynamics are stochastic, and not, in general, deterministic, the time reversal is not, in general, an inversion of the dynamics. Rather, the system relaxes towards equilibrium in both the forward and reverse time directions. The probability of a quantum trajectory and the conjugate, time reversed trajectory are related by the heat exchanged with the environment.Comment: 4 page

Probability distribution of residence times of grains in models of ricepiles

We study the probability distribution of residence time of a grain at a site, and its total residence time inside a pile, in different ricepile models. The tails of these distributions are dominated by the grains that get deeply buried in the pile. We show that, for a pile of size $L$, the probabilities that the residence time at a site or the total residence time is greater than $t$, both decay as $1/t(\ln t)^x$ for $L^{\omega} \ll t \ll \exp(L^{\gamma})$ where $\gamma$ is an exponent $\ge 1$, and values of $x$ and $\omega$ in the two cases are different. In the Oslo ricepile model we find that the probability that the residence time $T_i$ at a site $i$ being greater than or equal to $t$, is a non-monotonic function of $L$ for a fixed $t$ and does not obey simple scaling. For model in $d$ dimensions, we show that the probability of minimum slope configuration in the steady state, for large $L$, varies as $\exp(-\kappa L^{d+2})$ where $\kappa$ is a constant, and hence $\gamma = d+2$.Comment: 13 pages, 23 figures, Submitted to Phys. Rev.

Micromagnetic understanding of stochastic resonance driven by spin-transfertorque

In this paper, we employ micromagnetic simulations to study non-adiabatic stochastic resonance (NASR) excited by spin-transfer torque in a super-paramagnetic free layer nanomagnet of a nanoscale spin valve. We find that NASR dynamics involves thermally activated transitions among two static states and a single dynamic state of the nanomagnet and can be well understood in the framework of Markov chain rate theory. Our simulations show that a direct voltage generated by the spin valve at the NASR frequency is at least one order of magnitude greater than the dc voltage generated off the NASR frequency. Our computations also reproduce the main experimentally observed features of NASR such as the resonance frequency, the temperature dependence and the current bias dependence of the resonance amplitude. We propose a simple design of a microwave signal detector based on NASR driven by spin transfer torque.Comment: 25 pages 8 figures, accepted for pubblication on Phys. Rev.

Excessive functions of continuous time Markov chains

AbstractWe consider transient continuous time Markov chains P(t) with P′ij(0)=qiΠij for i≠j and −qi for i=j. We assume 0<qi<∞ for all i. Then 1/qi is the mean time the process remains in state i, and Π is the transition matrix of the imbedded jump process. We let q be a diagonal matrix with diagonal entries qi.A non-negative function h is P(t)-excessive (invariant) if h≥P(t)h, (h=P(t) h) for all t. It is Π-superregular (regular) if h≥Πh (h=Πh). Our main results characterize the excessive functions of the minimal process in terms of q and Π. These results can also be used to characterize excessive functions of certain non-minimal processes

Computing the entropy of user navigation in the web

Navigation through the web, colloquially known as "surfing", is one of the main activities of users during web interaction. When users follow a navigation trail they often tend to get disoriented in terms of the goals of their original query and thus the discovery of typical user trails could be useful in providing navigation assistance. Herein, we give a theoretical underpinning of user navigation in terms of the entropy of an underlying Markov chain modelling the web topology. We present a novel method for online incremental computation of the entropy and a large deviation result regarding the length of a trail to realize the said entropy. We provide an error analysis for our estimation of the entropy in terms of the divergence between the empirical and actual probabilities. We then indicate applications of our algorithm in the area of web data mining. Finally, we present an extension of our technique to higher-order Markov chains by a suitable reduction of a higher-order Markov chain model to a first-order one

Improved coarse-graining of Markov state models via explicit consideration of statistical uncertainty

Markov state models (MSMs)---or discrete-time master equation models---are a powerful way of modeling the structure and function of molecular systems like proteins. Unfortunately, MSMs with sufficiently many states to make a quantitative connection with experiments (often tens of thousands of states even for small systems) are generally too complicated to understand. Here, I present a Bayesian agglomerative clustering engine (BACE) for coarse-graining such Markov models, thereby reducing their complexity and making them more comprehensible. An important feature of this algorithm is its ability to explicitly account for statistical uncertainty in model parameters that arises from finite sampling. This advance builds on a number of recent works highlighting the importance of accounting for uncertainty in the analysis of MSMs and provides significant advantages over existing methods for coarse-graining Markov state models. The closed-form expression I derive here for determining which states to merge is equivalent to the generalized Jensen-Shannon divergence, an important measure from information theory that is related to the relative entropy. Therefore, the method has an appealing information theoretic interpretation in terms of minimizing information loss. The bottom-up nature of the algorithm likely makes it particularly well suited for constructing mesoscale models. I also present an extremely efficient expression for Bayesian model comparison that can be used to identify the most meaningful levels of the hierarchy of models from BACE