PAC-Bayesian Theory Meets Bayesian Inference

We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam's razor criteria, under the assumption that the data is generated by an i.i.d distribution. Moreover, as the negative log-likelihood is an unbounded loss function, we motivate and propose a PAC-Bayesian theorem tailored for the sub-gamma loss family, and we show that our approach is sound on classical Bayesian linear regression tasks.Comment: Published at NIPS 2015 (http://papers.nips.cc/paper/6569-pac-bayesian-theory-meets-bayesian-inference

SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives

In this work we introduce a new optimisation method called SAGA in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and has support for composite objectives where a proximal operator is used on the regulariser. Unlike SDCA, SAGA supports non-strongly convex problems directly, and is adaptive to any inherent strong convexity of the problem. We give experimental results showing the effectiveness of our method.Comment: Advances In Neural Information Processing Systems, Nov 2014, Montreal, Canad

Rethinking LDA: moment matching for discrete ICA

We consider moment matching techniques for estimation in Latent Dirichlet Allocation (LDA). By drawing explicit links between LDA and discrete versions of independent component analysis (ICA), we first derive a new set of cumulant-based tensors, with an improved sample complexity. Moreover, we reuse standard ICA techniques such as joint diagonalization of tensors to improve over existing methods based on the tensor power method. In an extensive set of experiments on both synthetic and real datasets, we show that our new combination of tensors and orthogonal joint diagonalization techniques outperforms existing moment matching methods.Comment: 30 pages; added plate diagrams and clarifications, changed style, corrected typos, updated figures. in Proceedings of the 29-th Conference on Neural Information Processing Systems (NIPS), 201

On the Equivalence between Herding and Conditional Gradient Algorithms

We show that the herding procedure of Welling (2009) takes exactly the form of a standard convex optimization algorithm--namely a conditional gradient algorithm minimizing a quadratic moment discrepancy. This link enables us to invoke convergence results from convex optimization and to consider faster alternatives for the task of approximating integrals in a reproducing kernel Hilbert space. We study the behavior of the different variants through numerical simulations. The experiments indicate that while we can improve over herding on the task of approximating integrals, the original herding algorithm tends to approach more often the maximum entropy distribution, shedding more light on the learning bias behind herding

Sequential Kernel Herding: Frank-Wolfe Optimization for Particle Filtering

Recently, the Frank-Wolfe optimization algorithm was suggested as a procedure to obtain adaptive quadrature rules for integrals of functions in a reproducing kernel Hilbert space (RKHS) with a potentially faster rate of convergence than Monte Carlo integration (and "kernel herding" was shown to be a special case of this procedure). In this paper, we propose to replace the random sampling step in a particle filter by Frank-Wolfe optimization. By optimizing the position of the particles, we can obtain better accuracy than random or quasi-Monte Carlo sampling. In applications where the evaluation of the emission probabilities is expensive (such as in robot localization), the additional computational cost to generate the particles through optimization can be justified. Experiments on standard synthetic examples as well as on a robot localization task indicate indeed an improvement of accuracy over random and quasi-Monte Carlo sampling.Comment: in 18th International Conference on Artificial Intelligence and Statistics (AISTATS), May 2015, San Diego, United States. 38, JMLR Workshop and Conference Proceeding

A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method

In this note, we present a new averaging technique for the projected stochastic subgradient method. By using a weighted average with a weight of t+1 for each iterate w_t at iteration t, we obtain the convergence rate of O(1/t) with both an easy proof and an easy implementation. The new scheme is compared empirically to existing techniques, with similar performance behavior.Comment: 8 pages, 6 figures. Changes with previous version: Added reference to concurrently submitted work arXiv:1212.1824v1; clarifications added; typos corrected; title changed to 'subgradient method' as 'subgradient descent' is misnome

Transposon-mediated BAC transgenesis in human ES cells

Transgenesis is a cornerstone of molecular biology. The ability to integrate a specifically engineered piece of DNA into the genome of a living system is fundamental to our efforts to understand life and exploit its implications for medicine, nanotechnology and bioprospecting. However, transgenesis has been hampered by position effects and multi-copy integration problems, which are mainly due to the use of small, plasmid-based transgenes. Large transgenes based on native genomic regions cloned into bacterial artificial chromosomes (BACs) circumvent these problems but are prone to fragmentation. Herein, we report that contrary to widely held notions, large BAC-sized constructs do not prohibit transposition. We also report the first reliable method for BAC transgenesis in human embryonic stem cells (hESCs). The PiggyBac or Sleeping Beauty transposon inverted repeats were integrated into BAC vectors by recombineering, followed by co-lipofection with the corresponding transposase in hESCs to generate robust fluorescent protein reporter lines for OCT4, NANOG, GATA4 and PAX6. BAC transposition delivers several advantages, including increased frequencies of single-copy, full-length integration, which will be useful in all transgenic systems but especially in difficult venues like hESCs

Heythrop, Copleston and the Jesuit contribution to philosophy

There has been public outcry from philosophers and others at the prospect of the closure of Heythrop College, University of London; yet the nature and history of Heythrop remain little known. It is apt and timely, therefore, as its likely dissolution approaches, to provide a brief account of its origins and development up to and including the period of its entry into London University under the leadership of the most famous modern historian of philosophy Frederick Copleston. Following on from this the idea of a distinctive Jesuit intellectual tradition, and more specifically of the Jesuit contribution to philosophy is explored.PostprintPeer reviewe

Gradual emergence followed by exponential spread of the SARS-CoV-2 Omicron variant in Africa.

The geographic and evolutionary origins of the SARS-CoV-2 Omicron variant (BA.1), which was first detected mid-November 2021 in Southern Africa, remain unknown. We tested 13,097 COVID-19 patients sampled between mid-2021 to early 2022 from 22 African countries for BA.1 by real-time RT-PCR. By November-December 2021, BA.1 had replaced the Delta variant in all African sub-regions following a South-North gradient, with a peak Rt of 4.1. Polymerase chain reaction and near-full genome sequencing data revealed genetically diverse Omicron ancestors already existed across Africa by August 2021. Mutations, altering viral tropism, replication and immune escape, gradually accumulated in the spike gene. Omicron ancestors were therefore present in several African countries months before Omicron dominated transmission. These data also indicate that travel bans are ineffective in the face of undetected and widespread infection