L’absolu et la dévastation des pommes de terres, ou le romantisme en question en 1837 (Lettres d’un voyageur, X)

En cette année 1837 où sont réunies les Lettres d’un voyageur paraissent également deux autres œuvres témoignant d’une pareille attitude critique à l’égard du romantisme, à savoir la première partie d’Illusions perdues de Balzac et les Lettres de Dupuis et Cotonet de Musset. Ensemble, ces trois ouvrages participent à une même remise en question du romantisme historique de 1830, celui d’Hernani et de la Symphonie fantastique, pour prendre l’exemple de deux productions emblématiques de cette an..

On Medians of (Randomized) Pairwise Means

Tournament procedures, recently introduced in Lugosi & Mendelson (2016), offer an appealing alternative, from a theoretical perspective at least, to the principle of Empirical Risk Minimization in machine learning. Statistical learning by Median-of-Means (MoM) basically consists in segmenting the training data into blocks of equal size and comparing the statistical performance of every pair of candidate decision rules on each data block: that with highest performance on the majority of the blocks is declared as the winner. In the context of nonparametric regression, functions having won all their duels have been shown to outperform empirical risk minimizers w.r.t. the mean squared error under minimal assumptions, while exhibiting robustness properties. It is the purpose of this paper to extend this approach in order to address other learning problems, in particular for which the performance criterion takes the form of an expectation over pairs of observations rather than over one single observation, as may be the case in pairwise ranking, clustering or metric learning. Precisely, it is proved here that the bounds achieved by MoM are essentially conserved when the blocks are built by means of independent sampling without replacement schemes instead of a simple segmentation. These results are next extended to situations where the risk is related to a pairwise loss function and its empirical counterpart is of the form of a $U$-statistic. Beyond theoretical results guaranteeing the performance of the learning/estimation methods proposed, some numerical experiments provide empirical evidence of their relevance in practice

L’absolu et la dévastation des pommes de terres, ou le romantisme en question en 1837 (Lettres d’un voyageur, X)

En cette année 1837 où sont réunies les Lettres d’un voyageur paraissent également deux autres œuvres témoignant d’une pareille attitude critique à l’égard du romantisme, à savoir la première partie d’Illusions perdues de Balzac et les Lettres de Dupuis et Cotonet de Musset. Ensemble, ces trois ouvrages participent à une même remise en question du romantisme historique de 1830, celui d’Hernani et de la Symphonie fantastique, pour prendre l’exemple de deux productions emblématiques de cette a..

Le mauvais ton de Stendhal

L’affaire est entendue : Stendhal écrit mal, et il écrit mal parce qu’il n’a pas de style ou parce qu’il ne soigne pas son style. Il n’a pas de style : lui-même a avoué lire une page du Code civil chaque jour, avant de commencer à écrire, pour obtenir le degré de sécheresse convenable et se prémunir contre les fausses élégances du beau style romantique ; il ne soigne pas son style : Balzac dans son grand article sur La Chartreuse le lui reproche vertement et donne des exemples de ses néglige..

Multitask Online Learning: Listen to the Neighborhood Buzz

We study multitask online learning in a setting where agents can only exchange information with their neighbors on an arbitrary communication network. We introduce $\texttt{MT-CO}_2\texttt{OL}$, a decentralized algorithm for this setting whose regret depends on the interplay between the task similarities and the network structure. Our analysis shows that the regret of $\texttt{MT-CO}_2\texttt{OL}$ is never worse (up to constants) than the bound obtained when agents do not share information. On the other hand, our bounds significantly improve when neighboring agents operate on similar tasks. In addition, we prove that our algorithm can be made differentially private with a negligible impact on the regret when the losses are linear. Finally, we provide experimental support for our theory

Le mauvais ton de Stendhal

L’affaire est entendue : Stendhal écrit mal, et il écrit mal parce qu’il n’a pas de style ou parce qu’il ne soigne pas son style. Il n’a pas de style : lui-même a avoué lire une page du Code civil chaque jour, avant de commencer à écrire, pour obtenir le degré de sécheresse convenable et se prémunir contre les fausses élégances du beau style romantique ; il ne soigne pas son style : Balzac dans son grand article sur La Chartreuse le lui reproche vertement et donne des exemples de ses négligen..

Linear Bandits with Memory: from Rotting to Rising

Nonstationary phenomena, such as satiation effects in recommendation, are a common feature of sequential decision-making problems. While these phenomena have been mostly studied in the framework of bandits with finitely many arms, in many practically relevant cases linear bandits provide a more effective modeling choice. In this work, we introduce a general framework for the study of nonstationary linear bandits, where current rewards are influenced by the learner's past actions in a fixed-size window. In particular, our model includes stationary linear bandits as a special case. After showing that the best sequence of actions is NP-hard to compute in our model, we focus on cyclic policies and prove a regret bound for a variant of the OFUL algorithm that balances approximation and estimation errors. Our theoretical findings are supported by experiments (which also include misspecified settings) where our algorithm is seen to perform well against natural baselines

Multitask Online Mirror Descent

We introduce and analyze MT-OMD, a multitask generalization of Online Mirror Descent (OMD) which operates by sharing updates between tasks. We prove that the regret of MT-OMD is of order $\sqrt{1 + \sigma^2(N-1)}\sqrt{T}$, where $\sigma^2$ is the task variance according to the geometry induced by the regularizer, $N$ is the number of tasks, and $T$ is the time horizon. Whenever tasks are similar, that is $\sigma^2 \le 1$, our method improves upon the $\sqrt{NT}$ bound obtained by running independent OMDs on each task. We further provide a matching lower bound, and show that our multitask extensions of Online Gradient Descent and Exponentiated Gradient, two major instances of OMD, enjoy closed-form updates, making them easy to use in practice. Finally, we present experiments on both synthetic and real-world datasets supporting our findings