Selective Sampling with Drift

Recently there has been much work on selective sampling, an online active learning setting, in which algorithms work in rounds. On each round an algorithm receives an input and makes a prediction. Then, it can decide whether to query a label, and if so to update its model, otherwise the input is discarded. Most of this work is focused on the stationary case, where it is assumed that there is a fixed target model, and the performance of the algorithm is compared to a fixed model. However, in many real-world applications, such as spam prediction, the best target function may drift over time, or have shifts from time to time. We develop a novel selective sampling algorithm for the drifting setting, analyze it under no assumptions on the mechanism generating the sequence of instances, and derive new mistake bounds that depend on the amount of drift in the problem. Simulations on synthetic and real-world datasets demonstrate the superiority of our algorithms as a selective sampling algorithm in the drifting setting

Passive-Aggressive online learning with nonlinear embeddings

[EN] Nowadays, there is an increasing demand for machine learning techniques which can deal with problems where the instances are produced as a stream or in real time. In these scenarios, online learning is able to learn a model from data that comes continuously. The adaptability, efficiency and scalability of online learning techniques have been gaining interest last years with the increasing amount of data generated every day. In this paper, we propose a novel binary classification approach based on nonlinear mapping functions under an online learning framework. The non-convex optimization problem that arises is split into three different convex problems that are solved by means of Passive-Aggressive Online Learning. We evaluate both the adaptability and generalization of our model through several experiments comparing with the state of the art techniques. We improve significantly the results in several datasets widely used previously by the online learning community. (C) 2018 Elsevier Ltd. All rights reserved.This work was developed in the framework of the PROM-ETEOII/2014/030 research project "Adaptive learning and multi modality in machine translation and text transcription", funded by the Generalitat Valenciana. The work of the first author is financed by Grant FPU14/03981, from the Spanish Ministry of Education, Culture and Sport.Jorge-Cano, J.; Paredes Palacios, R. (2018). Passive-Aggressive online learning with nonlinear embeddings. Pattern Recognition. 79:162-171. https://doi.org/10.1016/j.patcog.2018.01.019S1621717

Aspects of a phase transition in high-dimensional random geometry

A phase transition in high-dimensional random geometry is analyzed as it arises in a variety of problems. A prominent example is the feasibility of a minimax problem that represents the extremal case of a class of financial risk measures, among them the current regulatory market risk measure Expected Shortfall. Others include portfolio optimization with a ban on short selling, the storage capacity of the perceptron, the solvability of a set of linear equations with random coefficients, and competition for resources in an ecological system. These examples shed light on various aspects of the underlying geometric phase transition, create links between problems belonging to seemingly distant fields and offer the possibility for further ramifications