Validation of Matching

We introduce a technique to compute probably approximately correct (PAC) bounds on precision and recall for matching algorithms. The bounds require some verified matches, but those matches may be used to develop the algorithms. The bounds can be applied to network reconciliation or entity resolution algorithms, which identify nodes in different networks or values in a data set that correspond to the same entity. For network reconciliation, the bounds do not require knowledge of the network generation process

On Certified Generalization in Structured Prediction

In structured prediction, target objects have rich internal structure which does not factorize into independent components and violates common i.i.d. assumptions. This challenge becomes apparent through the exponentially large output space in applications such as image segmentation or scene graph generation. We present a novel PAC-Bayesian risk bound for structured prediction wherein the rate of generalization scales not only with the number of structured examples but also with their size. The underlying assumption, conforming to ongoing research on generative models, is that data are generated by the Knothe-Rosenblatt rearrangement of a factorizing reference measure. This allows to explicitly distill the structure between random output variables into a Wasserstein dependency matrix. Our work makes a preliminary step towards leveraging powerful generative models to establish generalization bounds for discriminative downstream tasks in the challenging setting of structured prediction

PAC-Bayes bounds for stable algorithms with instance-dependent priors

PAC-Bayes bounds have been proposed to get risk estimates based on a training sample. In this paper the PAC-Bayes approach is combined with stability of the hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting is used with a Gaussian prior centered at the expected output. Thus a novelty of our paper is using priors defined in terms of the data-generating distribution. Our main result estimates the risk of the randomized algorithm in terms of the hypothesis stability coefficients. We also provide a new bound for the SVM classifier, which is compared to other known bounds experimentally. Ours appears to be the first stability-based bound that evaluates to non-trivial values.Comment: 16 pages, discussion of theory and experiments in the main body, detailed proofs and experimental details in the appendice

How large is congressional dependence in agriculture. Bayesian inference about ‘scale’ and ‘scope’ in measuring a spatial externality

The political economy literature on agriculture emphasizes influence over political outcomes via lobbying conduits in general, political action committee contributions in particular and the pervasive view that political preferences with respect to agricultural issues are inherently geographic. In this context, ‘interdependence’ in Congressional vote behaviour manifests itself in two dimensions. One dimension is the intensity by which neighboring vote propensities influence one another and the second is the geographic extent of voter influence. We estimate these facets of dependence using data on a Congressional vote on the 2001 Farm Bill using routine Markov chain Monte Carlo procedures and Bayesian model averaging, in particular. In so doing, we develop a novel procedure to examine both the reliability and the consequences of different model representations for measuring both the ‘scale’ and the ‘scope’ of spatial (geographic) co-relations in voting behaviour

PAC-Bayesian Computation

Risk bounds, which are also called generalisation bounds in the statistical learning literature, are important objects of study because they give some information on the expected error that a predictor may incur on randomly chosen data points. In classical statistical learning, the analyses focus on individual hypotheses, and the aim is deriving risk bounds that are valid for the data-dependent hypothesis output by some learning method. Often, however, such risk bounds are valid uniformly over a hypothesis class, which is a consequence of the methods used to derive them, namely the theory of uniform convergence of empirical processes. This is a source of looseness of these classical kinds of bounds which has lead to debates and criticisms, and motivated the search of alternative methods to derive tighter bounds. The PAC-Bayes analysis focuses on distributions over hypotheses and randomised predictors defined by such distributions. Other prediction schemes can be devised based on a distribution over hypotheses, however, the randomised predictor is a typical starting point. Lifting the analysis to distributions over hypotheses, rather than individual hypotheses, makes available sharp analysis tools, which arguably account for the tightness of PAC-Bayes bounds. Two main uses of PAC-Bayes bounds are (1) risk certification, and (2) cost function derivation. The first consists of evaluating numerical risk certificates for the distributions over hypotheses learned by some method, while the second consists of turning a PAC-Bayes bound into a training objective, to learn a distribution by minimising the bound. This thesis revisits both kinds of uses of PAC-Bayes bounds. We contribute results on certifying the risk of randomised kernel and neural network classifiers, adding evidence to the success of PAC-Bayes bounds at delivering tight certificates. This thesis proposes the name “PAC-Bayesian Computation” as a generic name to encompass the class of methods that learn a distribution over hypotheses by minimising a PAC-Bayes bound (i.e. the second use case described above: cost function derivation), and reports an interesting case of PAC-Bayesian Computation leading to self-certified learning: we develop a learning and certification strategy that uses all the available data to produce a predictor together with a tight risk certificate, as demonstrated with randomised neural network classifiers on two benchmark data sets (MNIST, CIFAR-10)

A Bayesian Framework to Constrain the Photon Mass with a Catalog of Fast Radio Bursts

A hypothetical photon mass, $m_\gamma$, gives an energy-dependent light speed in a Lorentz-invariant theory. Such a modification causes an additional time delay between photons of different energies when they travel through a fixed distance. Fast radio bursts (FRBs), with their short time duration and cosmological propagation distance, are excellent astrophysical objects to constrain $m_\gamma$. Here for the first time we develop a Bayesian framework to study this problem with a catalog of FRBs. Those FRBs with and without redshift measurement are both useful in this framework, and can be combined in a Bayesian way. A catalog of 21 FRBs (including 20 FRBs without redshift measurement, and one, FRB 121102, with a measured redshift $z=0.19273 \pm 0.00008$) give a combined limit $m_\gamma \leq 8.7 \times 10^{-51}\, {\rm kg}$, or equivalently $m_\gamma \leq 4.9 \times 10^{-15}\, {\rm eV}/c^2$ ($m_\gamma \leq 1.5\times10^{-50} \, {\rm kg}$, or equivalently $m_\gamma \leq 8.4 \times 10^{-15} \,{\rm eV}/c^2$) at 68% (95%) confidence level, which represents the best limit that comes purely from kinematics. The framework proposed here will be valuable when FRBs are observed daily in the future. Increment in the number of FRBs, and refinement in the knowledge about the electron distributions in the Milky Way, the host galaxies of FRBs, and the intergalactic median, will further tighten the constraint.Comment: 10 pages, 6 figures; Physical Review D, in pres

A Model for Prejudiced Learning in Noisy Environments

Based on the heuristics that maintaining presumptions can be beneficial in uncertain environments, we propose a set of basic axioms for learning systems to incorporate the concept of prejudice. The simplest, memoryless model of a deterministic learning rule obeying the axioms is constructed, and shown to be equivalent to the logistic map. The system's performance is analysed in an environment in which it is subject to external randomness, weighing learning defectiveness against stability gained. The corresponding random dynamical system with inhomogeneous, additive noise is studied, and shown to exhibit the phenomena of noise induced stability and stochastic bifurcations. The overall results allow for the interpretation that prejudice in uncertain environments entails a considerable portion of stubbornness as a secondary phenomenon.Comment: 21 pages, 11 figures; reduced graphics to slash size, full version on Author's homepage. Minor revisions in text and references, identical to version to be published in Applied Mathematics and Computatio

Subjectivity: A Case of Biological Individuation and an Adaptive Response to Informational Overflow

The article presents a perspective on the scientific explanation of the subjectivity of conscious experience. It proposes plausible answers for two empirically valid questions: the ‘how’ question concerning the developmental mechanisms of subjectivity, and the ‘why’ question concerning its function. Biological individuation, which is acquired in several different stages, serves as a provisional description of how subjective perspectives may have evolved. To the extent that an individuated informational space seems the most efficient way for a given organism to select biologically valuable information, subjectivity is deemed to constitute an adaptive response to informational overflow. One of the possible consequences of this view is that subjectivity might be (at least functionally) dissociated from consciousness, insofar as the former primarily facilitates selection, the latter action