Gaussian Process Optimization with Adaptive Sketching: Scalable and No Regret

Gaussian processes (GP) are a well studied Bayesian approach for the optimization of black-box functions. Despite their effectiveness in simple problems, GP-based algorithms hardly scale to high-dimensional functions, as their per-iteration time and space cost is at least quadratic in the number of dimensions $d$ and iterations $t$. Given a set of $A$ alternatives to choose from, the overall runtime $O(t^3A)$ is prohibitive. In this paper we introduce BKB (budgeted kernelized bandit), a new approximate GP algorithm for optimization under bandit feedback that achieves near-optimal regret (and hence near-optimal convergence rate) with near-constant per-iteration complexity and remarkably no assumption on the input space or covariance of the GP. We combine a kernelized linear bandit algorithm (GP-UCB) with randomized matrix sketching based on leverage score sampling, and we prove that randomly sampling inducing points based on their posterior variance gives an accurate low-rank approximation of the GP, preserving variance estimates and confidence intervals. As a consequence, BKB does not suffer from variance starvation, an important problem faced by many previous sparse GP approximations. Moreover, we show that our procedure selects at most $\tilde{O}(d_{eff})$ points, where $d_{eff}$ is the effective dimension of the explored space, which is typically much smaller than both $d$ and $t$. This greatly reduces the dimensionality of the problem, thus leading to a $O(TAd_{eff}^2)$ runtime and $O(A d_{eff})$ space complexity.Comment: Accepted at COLT 2019. Corrected typos and improved comparison with existing method

On Fast Leverage Score Sampling and Optimal Learning

International audienceLeverage score sampling provides an appealing way to perform approximate computations for large matrices. Indeed, it allows to derive faithful approximations with a complexity adapted to the problem at hand. Yet, performing leverage scores sampling is a challenge in its own right requiring further approximations. In this paper, we study the problem of leverage score sampling for positive definite matrices defined by a kernel. Our contribution is twofold. First we provide a novel algorithm for leverage score sampling and second, we exploit the proposed method in statistical learning by deriving a novel solver for kernel ridge regression. Our main technical contribution is showing that the proposed algorithms are currently the most efficient and accurate for these problems

Gaussian process optimization with adaptive sketching: Scalable and no regret

International audienceGaussian processes (GP) are a stochastic processes, used as Bayesian approach for the optimization of black-box functions. Despite their effectiveness in simple problems, GP-based algorithms hardly scale to high-dimensional functions, as their per-iteration time and space cost is at least quadratic in the number of dimensions d and iterations t. Given a set of A alternatives to choose from, the overall runtime O(t 3 A) is prohibitive. In this paper, we introduce BKB (budgeted kernelized bandit), a new approximate GP algorithm for optimization under bandit feedback that achieves near-optimal regret (and hence near-optimal convergence rate) with near-constant per-iteration complexity and remarkably no assumption on the input space or covariance of the GP. We combine a kernelized linear bandit algorithm (GP-UCB) leverage score sampling as a randomized matrix sketching and prove that selecting inducing points based on their posterior variance gives an accurate low-rank approximation of the GP, preserving variance estimates and confidence intervals. As a consequence, BKB does not suffer from variance starvation, an important problem faced by many previous sparse GP approximations. Moreover, we show that our procedure selects at most O(d eff) points, where d eff is the effective dimension of the explored space, which is typically much smaller than both d and t. This greatly reduces the dimensionality of the problem, thus leading to a O(T Ad 2 eff) runtime and O(Ad eff) space complexity

Gaussian process optimization with adaptive sketching: Scalable and no regret

International audienceGaussian processes (GP) are a stochastic processes, used as Bayesian approach for the optimization of black-box functions. Despite their effectiveness in simple problems, GP-based algorithms hardly scale to high-dimensional functions, as their per-iteration time and space cost is at least quadratic in the number of dimensions d and iterations t. Given a set of A alternatives to choose from, the overall runtime O(t 3 A) is prohibitive. In this paper, we introduce BKB (budgeted kernelized bandit), a new approximate GP algorithm for optimization under bandit feedback that achieves near-optimal regret (and hence near-optimal convergence rate) with near-constant per-iteration complexity and remarkably no assumption on the input space or covariance of the GP. We combine a kernelized linear bandit algorithm (GP-UCB) leverage score sampling as a randomized matrix sketching and prove that selecting inducing points based on their posterior variance gives an accurate low-rank approximation of the GP, preserving variance estimates and confidence intervals. As a consequence, BKB does not suffer from variance starvation, an important problem faced by many previous sparse GP approximations. Moreover, we show that our procedure selects at most O(d eff) points, where d eff is the effective dimension of the explored space, which is typically much smaller than both d and t. This greatly reduces the dimensionality of the problem, thus leading to a O(T Ad 2 eff) runtime and O(Ad eff) space complexity

Hares (Lepus europaeus) population control in free game breeding of South Italy

The area availability, habitat characteristics and management, influence animals survival in great part of game breeding. Aim of the work was the evaluation of management influence on population trend (16 years) in a private game breeding of south Italy

Serum mitogenic activity on in vitro glial cells in Neurofibromatosis type 1

Glial mitogenic effect was investigated in sera from the following groups of subjects: group (1) 31 patients clinically and genetically affected by Neurofibromatosis type 1 (NF1) belonging to different families; group (2) 42 patients without family history of NF1 affected by sporadic neoplasms of the same histogenetic origin as the proliferative lesions that are present in NF1; group (3) 51 healthy volunteers without family history of NF1 nor of neoplastic disease; group (4) 54 clinically healthy relatives of the NF1 patients included in the first group. All NF1 patients and 3/54 healthy relatives had alterations of exons 31 or 32 of NF1 gene. Glial proliferation, measured by [H-3]thymidine incorporation, was significantly increased by sera from all NF1 patients and from 23/54 of clinically healthy relatives, as compared to sera from healthy volunteers. This serum mitogenic activity strongly suggests the existence of soluble glial proliferating molecules in NF1 families. The molecular weight (3-30 kDa), the heat- and freeze-stability and the specificity for glial cells, suggest that the molecules responsible for this mitogenic effect are different from the growth factors previously described in NF1-associated tumor extracts and from lymphokines. Within each NF1 family, the maximal serum dilution stimulating glial proliferation was similar both in affected members and in their clinically healthy relatives. Since none of the clinically healthy relatives showing serum mitogenic activity was positive for the NF1 mutation analysis and, conversely, those having altered exons 31 or 32 of NF1 gene did not show any mitogenic activity, these results suggest that the phenotype expression of NF1 might depend not only on the NF1 mutations per se, but also on other genetic or epigenetic factors, such as serum glial proliferating molecules. (C) 1998 Elsevier Science B.V. All rights reserved

A Modified One-Stage Early Correction of Blepharophimosis Syndrome Using Tutopatch Slings

PURPOSE:To investigate the efficacy of a one-stage early correction of blepharophimosis-ptosis-epicanthus inversus syndrome (BPES), using bovine pericardium derived membrane (TUTOPATCH(®)) for the frontalis suspension. METHODS:We prospectively studied 12 eyes from 6 patients (median age 14 months) affected by BPES with severe ptosis. All patients were submitted to a one-stage early correction of ptosis (frontalis suspension with TUTOPACH(®)) and telecanthus and epicanthus inversus. Upper margin reflex distance (MRD), nasal inner intercanthal distance (IICD), horizontal fissure length (HFL), and IICD/HFL ratio were evaluated using photographs. RESULTS:The Wilcoxon signed-rank test showed a statistically significant difference between pre- and post-operative MRD, IICD, HFL, and the IICD/HFL ratio. CONCLUSION:An early TUTOPATCH-assisted frontalis suspension, together with the correction of telecanthus and epicanthus inversus, is an effective procedure for BPES cases with severe ptosis

Cavernous venous malformation (cavernous hemangioma) of the orbit: Current concepts and a review of the literature

The cavernous venous malformation of the orbit, previously called cavernous hemangioma, is the most common primary orbital lesion of adults. Cavernous venous malformation occurs more often in women and typically presents in the fourth and fifth decades of life. It is a benign vascular malformation characterized by a well-defined capsule and numerous large vascular channels. The most common sign of cavernous venous malformation is progressive axial proptosis from the preferential involvement of the intraconal orbital space. Optic nerve damage and other signs of orbital pathology may be present, with a variable degree of visual impairment. The combination of ultrasound, computed tomography, and magnetic resonance imaging leads to an accurate diagnosis in the vast majority of cases. Surgical and nonsurgical treatments are required in case of symptomatic lesions, with a characteristic multidisciplinary management influencing optimal outcome. Orbitotomy represents the traditional surgical approach. Recently, the endoscopic transnasal approach to the orbital cavity has gained interest, representing a feasible and safe, less-invasive surgical technique for the management of cavernous venous malformation