A Cluster Elastic Net for Multivariate Regression

We propose a method for estimating coefficients in multivariate regression when there is a clustering structure to the response variables. The proposed method includes a fusion penalty, to shrink the difference in fitted values from responses in the same cluster, and an L1 penalty for simultaneous variable selection and estimation. The method can be used when the grouping structure of the response variables is known or unknown. When the clustering structure is unknown the method will simultaneously estimate the clusters of the response and the regression coefficients. Theoretical results are presented for the penalized least squares case, including asymptotic results allowing for p >> n. We extend our method to the setting where the responses are binomial variables. We propose a coordinate descent algorithm for both the normal and binomial likelihood, which can easily be extended to other generalized linear model (GLM) settings. Simulations and data examples from business operations and genomics are presented to show the merits of both the least squares and binomial methods.Comment: 37 Pages, 11 Figure

Binary Linear Classification and Feature Selection via Generalized Approximate Message Passing

For the problem of binary linear classification and feature selection, we propose algorithmic approaches to classifier design based on the generalized approximate message passing (GAMP) algorithm, recently proposed in the context of compressive sensing. We are particularly motivated by problems where the number of features greatly exceeds the number of training examples, but where only a few features suffice for accurate classification. We show that sum-product GAMP can be used to (approximately) minimize the classification error rate and max-sum GAMP can be used to minimize a wide variety of regularized loss functions. Furthermore, we describe an expectation-maximization (EM)-based scheme to learn the associated model parameters online, as an alternative to cross-validation, and we show that GAMP's state-evolution framework can be used to accurately predict the misclassification rate. Finally, we present a detailed numerical study to confirm the accuracy, speed, and flexibility afforded by our GAMP-based approaches to binary linear classification and feature selection

Treatment Response in Couple Therapy: Relationship Adjustment and Individual Functioning Change Processes

This study, a naturalistic investigation of the process of change in relationship adjustment and individual functioning during conjoint therapy, examined the first 8 sessions of a multisystemic model of couple therapy, integrative problem-centered metaframeworks (Breunlin, Pinsof, Russell, & Lebow, 2011; Pinsof, Breunlin, Russell, & Lebow, 2011). The sample consisted of 125 heterosexual couples who reported on their relationship adjustment and individual functioning before every session using the Systemic Therapy Inventory of Change (Pinsof et al., 2009; Pinsof, Zinbarg, et al., in press). Data were analyzed using dyadic latent growth curve and cross-lagged models. For both men and women, relationship adjustment and individual functioning showed nonlinear change, increasing during Sessions 1–4 and stabilizing during Sessions 5–8. At pretreatment, women reported lower levels of relationship adjustment than men; no gender differences existed in initial levels of individual functioning or in the change trajectories of relationship adjustment or individual functioning. Higher relationship adjustment predicted positive change in individual functioning for men (but not for women). In contrast, there were no cross-lagged effects of individual functioning on relationship adjustment for men or women. The results demonstrate the importance of examining the processes by which relational and individual pathology respond to couple-based interventions

The Role of the Mangement Sciences in Research on Personalization

We present a review of research studies that deal with personalization. We synthesize current knowledge about these areas, and identify issues that we envision will be of interest to researchers working in the management sciences. We take an interdisciplinary approach that spans the areas of economics, marketing, information technology, and operations. We present an overarching framework for personalization that allows us to identify key players in the personalization process, as well as, the key stages of personalization. The framework enables us to examine the strategic role of personalization in the interactions between a firm and other key players in the firm's value system. We review extant literature in the strategic behavior of firms, and discuss opportunities for analytical and empirical research in this regard. Next, we examine how a firm can learn a customer's preferences, which is one of the key components of the personalization process. We use a utility-based approach to formalize such preference functions, and to understand how these preference functions could be learnt based on a customer's interactions with a firm. We identify well-established techniques in management sciences that can be gainfully employed in future research on personalization.CRM, Persoanlization, Marketing, e-commerce,

Standard error estimation for EM applications related to Latent class models

The EM algorithm is a popular method for computing maximum likelihood estimates. It tends to be numerically stable, reduces execution time compared to other estimation procedures and is easy to implement in latent class models. However, the EM algorithm fails to provide a consistent estimator of the standard errors of maximum likelihood estimates in incomplete data applications. Correct standard errors can be obtained by numerical differentiation. The technique requires computation of a complete-data gradient vector and Hessian matrix, but not those associated with the incomplete data likelihood. Obtaining first and second derivatives numerically is computationally very intensive and execution time may become very expensive when fitting Latent class models using a Newton-type algorithm. When the execution time is too high one is motivated to use the EM algorithm solution to initialize the Newton Raphson algorithm. We also investigate the effect on the execution time when a final Newton-Raphson step follows the EM algorithm after convergence. In this paper we compare the standard errors provided by the EM and Newton-Raphson algorithms for two models and analyze how this bias is affected by the number of parameters in the model fit.peer-reviewe

A Contextual-Bandit Approach to Personalized News Article Recommendation

Personalized web services strive to adapt their services (advertisements, news articles, etc) to individual users by making use of both content and user information. Despite a few recent advances, this problem remains challenging for at least two reasons. First, web service is featured with dynamically changing pools of content, rendering traditional collaborative filtering methods inapplicable. Second, the scale of most web services of practical interest calls for solutions that are both fast in learning and computation. In this work, we model personalized recommendation of news articles as a contextual bandit problem, a principled approach in which a learning algorithm sequentially selects articles to serve users based on contextual information about the users and articles, while simultaneously adapting its article-selection strategy based on user-click feedback to maximize total user clicks. The contributions of this work are three-fold. First, we propose a new, general contextual bandit algorithm that is computationally efficient and well motivated from learning theory. Second, we argue that any bandit algorithm can be reliably evaluated offline using previously recorded random traffic. Finally, using this offline evaluation method, we successfully applied our new algorithm to a Yahoo! Front Page Today Module dataset containing over 33 million events. Results showed a 12.5% click lift compared to a standard context-free bandit algorithm, and the advantage becomes even greater when data gets more scarce.Comment: 10 pages, 5 figure