Quantum estimation via minimum Kullback entropy principle

We address quantum estimation in situations where one has at disposal data from the measurement of an incomplete set of observables and some a priori information on the state itself. By expressing the a priori information in terms of a bias toward a given state the problem may be faced by minimizing the quantum relative entropy (Kullback entropy) with the constraint of reproducing the data. We exploit the resulting minimum Kullback entropy principle for the estimation of a quantum state from the measurement of a single observable, either from the sole mean value or from the complete probability distribution, and apply it as a tool for the estimation of weak Hamiltonian processes. Qubit and harmonic oscillator systems are analyzed in some details.Comment: 7 pages, slightly revised version, no figure

A Quantitative Occam's Razor

This paper derives an objective Bayesian "prior" based on considerations of entropy/information. By this means, it produces a quantitative measure of goodness of fit (the "H-statistic") that balances higher likelihood against the number of fitting parameters employed. The method is intended for phenomenological applications where the underlying theory is uncertain or unknown. For example, it can help decide whether the large angle anomalies in the CMB data should be taken seriously. I am therefore posting it now, even though it was published before the arxiv existed.Comment: plainTeX, 16 pages, no figures. Most current version is available at http://www.physics.syr.edu/~sorkin/some.papers/ (or wherever my home-page may be

Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search

We present a framework for quantifying and mitigating algorithmic bias in mechanisms designed for ranking individuals, typically used as part of web-scale search and recommendation systems. We first propose complementary measures to quantify bias with respect to protected attributes such as gender and age. We then present algorithms for computing fairness-aware re-ranking of results. For a given search or recommendation task, our algorithms seek to achieve a desired distribution of top ranked results with respect to one or more protected attributes. We show that such a framework can be tailored to achieve fairness criteria such as equality of opportunity and demographic parity depending on the choice of the desired distribution. We evaluate the proposed algorithms via extensive simulations over different parameter choices, and study the effect of fairness-aware ranking on both bias and utility measures. We finally present the online A/B testing results from applying our framework towards representative ranking in LinkedIn Talent Search, and discuss the lessons learned in practice. Our approach resulted in tremendous improvement in the fairness metrics (nearly three fold increase in the number of search queries with representative results) without affecting the business metrics, which paved the way for deployment to 100% of LinkedIn Recruiter users worldwide. Ours is the first large-scale deployed framework for ensuring fairness in the hiring domain, with the potential positive impact for more than 630M LinkedIn members.Comment: This paper has been accepted for publication at ACM KDD 201

Effective rate equations for the over-damped motion in fluctuating potentials

We discuss physical and mathematical aspects of the over-damped motion of a Brownian particle in fluctuating potentials. It is shown that such a system can be described quantitatively by fluctuating rates if the potential fluctuations are slow compared to relaxation within the minima of the potential, and if the position of the minima does not fluctuate. Effective rates can be calculated; they describe the long-time dynamics of the system. Furthermore, we show the existence of a stationary solution of the Fokker-Planck equation that describes the motion within the fluctuating potential under some general conditions. We also show that a stationary solution of the rate equations with fluctuating rates exists.Comment: 18 pages, 2 figures, standard LaTeX2

Probability density of determinants of random matrices

In this brief paper the probability density of a random real, complex and quaternion determinant is rederived using singular values. The behaviour of suitably rescaled random determinants is studied in the limit of infinite order of the matrices

Information Theory based on Non-additive Information Content

We generalize the Shannon's information theory in a nonadditive way by focusing on the source coding theorem. The nonadditive information content we adopted is consistent with the concept of the form invariance structure of the nonextensive entropy. Some general properties of the nonadditive information entropy are studied, in addition, the relation between the nonadditivity $q$ and the codeword length is pointed out.Comment: 9 pages, no figures, RevTex, accepted for publication in Phys. Rev. E(an error in proof of theorem 1 was corrected, typos corrected

Some Objects Are More Equal Than Others: Measuring and Predicting Importance

We observe that everyday images contain dozens of objects, and that humans, in describing these images, give different priority to these objects. We argue that a goal of visual recognition is, therefore, not only to detect and classify objects but also to associate with each a level of priority which we call 'importance'. We propose a definition of importance and show how this may be estimated reliably from data harvested from human observers. We conclude by showing that a first-order estimate of importance may be computed from a number of simple image region measurements and does not require access to image meaning

Statistical distinguishability between unitary operations

The problem of distinguishing two unitary transformations, or quantum gates, is analyzed and a function reflecting their statistical distinguishability is found. Given two unitary operations, $U_1$ and $U_2$, it is proved that there always exists a finite number $N$ such that $U_1^{\otimes N}$ and $U_2^{\otimes N}$ are perfectly distinguishable, although they were not in the single-copy case. This result can be extended to any finite set of unitary transformations. Finally, a fidelity for one-qubit gates, which satisfies many useful properties from the point of view of quantum information theory, is presented.Comment: 6 pages, REVTEX. The perfect distinguishability result is extended to any finite set of gate

Quantifying Self-Organization with Optimal Predictors

Despite broad interest in self-organizing systems, there are few quantitative, experimentally-applicable criteria for self-organization. The existing criteria all give counter-intuitive results for important cases. In this Letter, we propose a new criterion, namely an internally-generated increase in the statistical complexity, the amount of information required for optimal prediction of the system's dynamics. We precisely define this complexity for spatially-extended dynamical systems, using the probabilistic ideas of mutual information and minimal sufficient statistics. This leads to a general method for predicting such systems, and a simple algorithm for estimating statistical complexity. The results of applying this algorithm to a class of models of excitable media (cyclic cellular automata) strongly support our proposal.Comment: Four pages, two color figure