Regularized Bundle Methods for Convex and Non-Convex Risks

Machine learning is most often cast as an optimization problem. Ideally, one expects a convex objective function to rely on efficient convex optimizers with nice guarantees such as no local optima. Yet, non-convexity is very frequent in practice and it may sometimes be inappropriate to look for convexity at any price. Alternatively one can decide not to limit a priori the modeling expressivity to models whose learning may be solved by convex optimization and rely on non-convex optimization algorithms. The main motivation of this work is to provide efficient and scalable algorithms for non-convex optimization. We focus on regularized unconstrained optimization problems which cover a large number of modern machine learning problems such as logistic regression, conditional random fields, large margin estimation, etc. We propose a novel algorithm for minimizing a regularized objective that is able to handle convex and non-convex, smooth and non-smooth risks. The algorithm is based on the cutting plane technique and on the idea of exploiting the regularization term in the objective function. It may be thought as a limited memory extension of convex regularized bundle methods for dealing with convex and non convex risks. In case the risk is convex the algorithm is proved to converge to a stationary solution with accuracy ε with a rate O(1/λε) where λ is the regularization parameter of the objective function under the assumption of a Lipschitz empirical risk. In case the risk is not convex getting such a proof is more difficult and requires a stronger and more disputable assumption. Yet we provide experimental results on artificial test problems, and on five standard and difficult machine learning problems that are cast as convex and non-convex optimization problems that show how our algorithm compares well in practice with state of the art optimization algorithms

The Places of Our Lives: Visiting Patterns and Automatic Labeling from Longitudinal Smartphone Data

The location tracking functionality of modern mobile devices provides unprecedented opportunity to the understanding of individual mobility in daily life. Instead of studying raw geographic coordinates, we are interested in understanding human mobility patterns based on sequences of place visits which encode, at a coarse resolution, most daily activities. This paper presents a study on place characterization in people's everyday life based on data recorded continuously by smartphones. First, we study human mobility from sequences of place visits, including visiting patterns on different place categories. Second, we address the problem of automatic place labeling from smartphone data without using any geo-location information. Our study on a large-scale data collected from 114 smartphone users over 18 months confirms many intuitions, and also reveals findings regarding both regularly and novelty trends in visiting patterns. Considering the problem of place labeling with 10 place categories, we show that frequently visited places can be recognized reliably (over 80%) while it is much more challenging to recognize infrequent places

Human Interaction Discovery in Smartphone Proximity Networks

Since humans are fundamentally social beings and interact frequently with others in their daily life, understanding social context is of primary importance in building context-aware applications. In this paper, using smartphone Bluetooth as a proximity sensor to create social networks, we present a probabilistic approach to mine human interaction types in real life. Our analysis is conducted on Bluetooth data continuously sensed with smartphones for over one year from 40 individuals who are professionally or personally related. The results show that the model can automatically discover a variety of social contexts. We objectively validated our model by studying its predictive and retrieval performance

Where and What: Using Smartphones to Predict Next Locations and Applications in Daily Life

This paper investigates the prediction of two aspects of human behavior us- ing smartphones as sensing devices. We present a framework for predicting where users will go and which app they will use in the next ten minutes by ex- ploiting the rich contextual information from smartphone sensors. Our first goal is to understand which smartphone sensor data types are important for the two prediction tasks. Secondly, we aim at extracting generic (i.e., user- independent) behavioral patterns and study how generic behavior models can improve the predictive performance of personalized models. Experimen- tal validation was conducted on the Lausanne Data Collection Campaign (LDCC) dataset, with longitudinal smartphone data collected over a period of 17 months from 71 users

By their apps you shall understand them: mining large-scale patterns of mobile phone usage

Mobile phones are becoming more and more widely used nowadays, and people do not use the phone only for communication: there is a wide variety of phone applications allowing users to select those that fit their needs. Aggregated over time, application usage patterns exhibit not only what people are consistently interested in but also the way in which they use their phones, and can help improving phone design and personalized services. This work aims at mining automatically usage patterns from apps data recorded continuously with smartphones. A new probabilistic framework for mining usage patterns is proposed. Our methodology involves the design of a bag-of-apps model that robustly represents level of phone usage over specific times of the day, and the use of a probabilistic topic model that jointly discovers patterns of usage over multiple applications and describes users as mixtures of such patterns. Our framework is evaluated using 230 000+ hours of real-life app phone log data, demonstrates that relevant patterns of usage can be extracted, and is objectively validated on a user retrieval task with competitive performance

Contextual Conditional Models for Smartphone-based Human Mobility Prediction

Human behavior is often complex and context-dependent. This paper presents a general technique to exploit this ``multidimensional'' contextual variable for human mobility prediction. We use an ensemble method, in which we extract different mobility patterns with multiple models and then combine these models under a probabilistic framework. The key idea lies in the assumption that human mobility can be explained by several mobility patterns that depend on a subset of the contextual variables and these can be learned by a simple model. We showed how this idea can be applied to two specific online prediction tasks: \textit{what is the next place a user will visit?} and \textit{how long will he stay in the current place?}. Using smartphone data collected from 153 users during 17 months, we show the potential of our method in predicting human mobility in real life

Contextual grouping: discovering real-life interaction types from longitudinal Bluetooth data

By exploiting built-in sensors, mobile smartphone have become attractive options for large-scale sensing of human behavior as well as social interaction. In this paper, we present a new probabilistic model to analyze longitudinal dynamic social networks created by the physical proximity of people sensed continuously by the phone Bluetooth sensors. A new probabilistic model is proposed in order to jointly infer emergent grouping modes of the community together with their temporal context. We present experimental results on a Bluetooth proximity network sensed with mobile smart-phones over 9 months of continuous real-life, and show the effectiveness of our method

GroupUs: Smartphone Proximity Data and Human Interaction Type Mining

There is an increasing interest in analyzing social interaction from mobile sensor data, and smartphones are rapidly becoming the most attractive sensing option. We propose a new probabilistic relational model to analyze long-term dynamic social networks created by physical proximity of people. Our model can infer different interaction types from the network, revealing the participants of a given group interaction, and discovering a variety of social contexts. Our analysis is conducted on Bluetooth data sensed with smartphones for over one year on the life of 40 individuals related by professional or personal links. We objectively validate our model by studying its predictive performance, showing a significant advantage over a recently proposed model

From Foursquare to my Square: Learning Check-in Behavior from Multiple Sources

Location-based services often use only a single mobility data source, which typically will be scarce for any new user when the system starts out. We propose a transfer learning method to characterize the temporal distribution of places of individuals by using an external, additional, large-scale check-in data set such as Foursquare data. The method is applied to the next place prediction problem, and we show that the incorporation of additional data through the proposed method improves the prediction accuracy when there is a limited amount of prior data

A Probabilistic Kernel Method for Human Mobility Prediction with Smartphones

Human mobility prediction is an important problem which has a large num- ber of applications, especially in context-aware services. This paper presents a study on location prediction using smartphone data, in which we address mod- eling and application aspects. Building personalized location prediction models from smartphone data remains a technical challenge due to data sparsity, which comes from the complexity of human behavior and the typically limited amount of data available for individual users. To address this problem, we propose an approach based on kernel density estimation, a popular smoothing technique for sparse data. Our approach contributes to existing work in two ways. First, our proposed model can estimate the probability that a user will be at a given location at a specific time in the future, by using both spatial and temporal information via multiple kernel functions. Second, we also show how our prob- abilistic framework extends to a more practical task of location prediction for a time window in the future. Our approach is validated on an everyday life location datasets consisting of 133 smartphone users. Our method reaches an accuracy of 84% for the next hour, and an accuracy of 77% for the next three hours