Differentiating the multipoint Expected Improvement for optimal batch design

This work deals with parallel optimization of expensive objective functions which are modeled as sample realizations of Gaussian processes. The study is formalized as a Bayesian optimization problem, or continuous multi-armed bandit problem, where a batch of q > 0 arms is pulled in parallel at each iteration. Several algorithms have been developed for choosing batches by trading off exploitation and exploration. As of today, the maximum Expected Improvement (EI) and Upper Confidence Bound (UCB) selection rules appear as the most prominent approaches for batch selection. Here, we build upon recent work on the multipoint Expected Improvement criterion, for which an analytic expansion relying on Tallis' formula was recently established. The computational burden of this selection rule being still an issue in application, we derive a closed-form expression for the gradient of the multipoint Expected Improvement, which aims at facilitating its maximization using gradient-based ascent algorithms. Substantial computational savings are shown in application. In addition, our algorithms are tested numerically and compared to state-of-the-art UCB-based batch-sequential algorithms. Combining starting designs relying on UCB with gradient-based EI local optimization finally appears as a sound option for batch design in distributed Gaussian Process optimization

Sharp template estimation in a shifted curves model

This paper considers the problem of adaptive estimation of a template in a randomly shifted curve model. Using the Fourier transform of the data, we show that this problem can be transformed into a stochastic linear inverse problem. Our aim is to approach the estimator that has the smallest risk on the true template over a finite set of linear estimators defined in the Fourier domain. Based on the principle of unbiased empirical risk minimization, we derive a nonasymptotic oracle inequality in the case where the law of the random shifts is known. This inequality can then be used to obtain adaptive results on Sobolev spaces as the number of observed curves tend to infinity. Some numerical experiments are given to illustrate the performances of our approach

Classification with the nearest neighbor rule in general finite dimensional spaces: necessary and sufficient conditions

Given an $n$-sample of random vectors $(X_i,Y_i)_{1 \leq i \leq n}$ whose joint law is unknown, the long-standing problem of supervised classification aims to \textit{optimally} predict the label $Y$ of a given a new observation $X$. In this context, the nearest neighbor rule is a popular flexible and intuitive method in non-parametric situations. Even if this algorithm is commonly used in the machine learning and statistics communities, less is known about its prediction ability in general finite dimensional spaces, especially when the support of the density of the observations is $\mathbb{R}^d$. This paper is devoted to the study of the statistical properties of the nearest neighbor rule in various situations. In particular, attention is paid to the marginal law of $X$, as well as the smoothness and margin properties of the \textit{regression function} $\eta(X) = \mathbb{E}[Y | X]$. We identify two necessary and sufficient conditions to obtain uniform consistency rates of classification and to derive sharp estimates in the case of the nearest neighbor rule. Some numerical experiments are proposed at the end of the paper to help illustrate the discussion.Comment: 53 Pages, 3 figure

Intensity estimation of non-homogeneous Poisson processes from shifted trajectories

This paper considers the problem of adaptive estimation of a non-homogeneous intensity function from the observation of n independent Poisson processes having a common intensity that is randomly shifted for each observed trajectory. We show that estimating this intensity is a deconvolution problem for which the density of the random shifts plays the role of the convolution operator. In an asymptotic setting where the number n of observed trajectories tends to infinity, we derive upper and lower bounds for the minimax quadratic risk over Besov balls. Non-linear thresholding in a Meyer wavelet basis is used to derive an adaptive estimator of the intensity. The proposed estimator is shown to achieve a near-minimax rate of convergence. This rate depends both on the smoothness of the intensity function and the density of the random shifts, which makes a connection between the classical deconvolution problem in nonparametric statistics and the estimation of a mean intensity from the observations of independent Poisson processes

Intensity estimation of non-homogeneous Poisson processes from shifted trajectories

In this paper, we consider the problem of estimating nonparametrically a mean pattern intensity λ from the observation of n independent and non-homogeneous Poisson processes N1,…,Nn on the interval [0,1]. This problem arises when data (counts) are collected independently from n individuals according to similar Poisson processes. We show that estimating this intensity is a deconvolution problem for which the density of the random shifts plays the role of the convolution operator. In an asymptotic setting where the number n of observed trajectories tends to infinity, we derive upper and lower bounds for the minimax quadratic risk over Besov balls. Non-linear thresholding in a Meyer wavelet basis is used to derive an adaptive estimator of the intensity. The proposed estimator is shown to achieve a near-minimax rate of convergence. This rate depends both on the smoothness of the intensity function and the density of the random shifts, which makes a connection between the classical deconvolution problem in nonparametric statistics and the estimation of a mean intensity from the observations of independent Poisson processes

Noisy classification with boundary assumptions

We address the problem of classification when data are collected from two samples with measurement errors. This problem turns to be an inverse problem and requires a specific treatment. In this context, we investigate the minimax rates of convergence using both a margin assumption, and a smoothness condition on the boundary of the set associated to the Bayes classifier. We establish lower and upper bounds (based on a deconvolution classifier) on these rates

Noisy classification with boundary assumptions

We address the problem of classification when data are collected from two samples with measurement errors. This problem turns to be an inverse problem and requires a specific treatment. In this context, we investigate the minimax rates of convergence using both a margin assumption, and a smoothness condition on the boundary of the set associated to the Bayes classifier. We establish lower and upper bounds (based on a deconvolution classifier) on these rates

Transient Response of the Head Kinematics - Influence of a Disturbed Visual Flow

Vision influences the controlled kinematics of human body. Previous studies have shown the influence of vision on head stabilization or whole posture. However, latencies between the stimuli and the head motion have never been quantified. The aim of this study is to quantify the influence of a perturbed vision on the head kinematics. Seven healthy volunteers without uncorrected vision (26.7±6.9 years old, 1 female, 2 right-handed/right-dominant eye, 5 right-handed/left-dominant eye) were studied. Visual stimuli were performed through an immersive personal 3D viewer (HMZ-T1, Sony), securely tied on the head. Motion analysis of the head and the torso were performed using the optoelectronic Vicon system (100Hz). Three markers were glued on the personal viewer, close to the nasion, left and right tragus, in order to create the head frame. Three markers were glued to create the torso frame (both acromia and C7). Two different 3D animated scenes were created on Blender and displayed at 24Hz. The first animation was a landscape with a ball rolling on the ground, and then the ball stopped before being virtually launched via a catapult toward the screen. Two velocities were programmed: 4.67 and 10.58 m.s-1. The second animation was a beach with sea and sky, where horizon tilted anticlockwise at 2 different constant rates: 0.24 deg.s-1 and 0.48 deg.s-1 with maximal amplitude of 8° and 16° respectively. The motion of the head relative to the torso was calculated for both scenes on seated and upright position, at the 2 different velocities, 2 times each, for a total of 16 random tests on each volunteer. For the launched ball animated scene, the reaction time seated was, as expected, shorter for the fast launches. For the beach animated scene, the head profiles followed most of the time the kinematic profile of the tilted animation, linearly or by steps, and not necessary until the end. Volunteers who were right-handed and right dominant eye tilted their head clockwise, at the inverse of the stimuli. Both experiments confirmed that visual stimulus could influence the kinematics of the head-neck system. In the ball animation, velocity of the stimulus does not seem to affect the amplitude of movement. In the beach animation, the head motions were variable, but performed at the same mean speed than the stimuli. Furthermore, the limited number of volunteer cannot conclude on the direction of rotation of the head, depending of the dominant hand and eye