Improved customer choice predictions using ensemble methods

In this paper various ensemble learning methods from machinelearning and statistics are considered and applied to the customerchoice modeling problem. The application of ensemble learningusually improves the prediction quality of flexible models likedecision trees and thus leads to improved predictions. We giveexperimental results for two real-life marketing datasets usingdecision trees, ensemble versions of decision trees and thelogistic regression model, which is a standard approach for thisproblem. The ensemble models are found to improve upon individualdecision trees and outperform logistic regression.Next, an additive decomposition of the prediction error of amodel, the bias/variance decomposition, is considered. A modelwith a high bias lacks the flexibility to fit the data well. Ahigh variance indicates that a model is instable with respect todifferent datasets. Decision trees have a high variance componentand a low bias component in the prediction error, whereas logisticregression has a high bias component and a low variance component.It is shown that ensemble methods aim at minimizing the variancecomponent in the prediction error while leaving the bias componentunaltered. Bias/variance decompositions for all models for bothcustomer choice datasets are given to illustrate these concepts.brand choice;data mining;boosting;choice models;Bias/Variance decomposition;Bagging;CART;ensembles

Hedonic price models and indices based on boosting applied to the Dutch housing market

We create a hedonic price model for house prices for six geographical submarkets in the Netherlands. Our model is based on a recent data mining technique called boosting. Boosting is an ensemble technique that combines multiple models, in our case decision trees, into a combined prediction. Boosting enables capturing of complex nonlinear relationships and interaction effects between input variables.We report mean relative errors and mean absolute error for all regions and compare our models with a standard linear regression approach. Our model improves prediction performance with up to 40% compared with Linear Regression. Next, we interpret the boosted models: we determine the most influential characteristics and graphically depict the relationship between the most important input variables and the house price. We find the size of the house to be the most important input for all but one region, and find some interesting nonlinear relationships between inputs and price.Finally, we construct hedonic price indices and compare these to the mean and median index and find that these indices differ notably in the urban regions of Amsterdam and Rotterdam.data mining;machine learning;gradient boosting;housing;hedonic price models;hedonic price index

Modeling brand choice using boosted and stacked neural networks

The brand choice problem in marketing has recently been addressed with methods from computational intelligence such as neural networks. Another class of methods from computational intelligence, the so-called ensemble methods such as boosting and stacking have never been applied to the brand choice problem, as far as we know. Ensemble methods generate a number of models for the same problem using any base method and combine the outcomes of these different models. It is well known that in many cases the predictive performance of ensemble methods significantly exceeds the predictive performance of the their base methods. In this report we use boosting and stacking of neural networks and apply this to a scanner dataset that is a benchmark dataset in the marketing literature. Using these methods, we find a significant improvement in predictive performance on this dataset.

Visualizing clickstream data with multidimensional scaling

We visualize a a web server log by means of multidimensionalscaling. To that end, a so-called dissimilarity metric is introduced inthe sets of sessions and pages respectively. We interpret the resultingvisualizations and find some interesting patterns.

Choosing Attribute Weights for Item Dissimilarity using Clikstream Data with an Application to a Product Catalog Map

In content- and knowledge-based recommender systems often a measure of (dis)similarity between items is used. Frequently, this measure is based on the attributes of the items. However, which attributes are important for the users of the system remains an important question to answer. In this paper, we present an approach to determine attribute weights in a dissimilarity measure using clickstream data of an e-commerce website. Counted is how many times products are sold and based on this a Poisson regression model is estimated. Estimates of this model are then used to determine the attribute weights in the dissimilarity measure. We show an application of this approach on a product catalog of MP3 players provided by Compare Group, owner of the Dutch price comparison site http://www.vergelijk.nl, and show how the dissimilarity measure can be used to improve 2D product catalog visualizations.dissimilarity measure;attribute weights;clickstream data;comparison

Boosting the accuracy of hedonic pricing models

Hedonic pricing models attempt to model a relationship between object attributes andthe object's price. Traditional hedonic pricing models are often parametric models that sufferfrom misspecification. In this paper we create these models by means of boosted CARTmodels. The method is explained in detail and applied to various datasets. Empirically,we find substantial reduction of errors on out-of-sample data for two out of three datasetscompared with a stepwise linear regression model. We interpret the boosted models by partialdependence plots and relative importance plots. This reveals some interesting nonlinearitiesand differences in attribute importance across the model types.pricing;marketing;data mining;conjoint analysis;ensemble learning;gradient boosting;hedonic pricing

Including Item Characteristics in the Probabilistic Latent Semantic Analysis Model for Collaborative Filtering

We propose a new hybrid recommender system that combines some advantages of collaborative and content-based recommender systems. While it uses ratings data of all users, as do collaborative recommender systems, it is also able to recommend new items and provide an explanation of its recommendations, as do content-based systems. Our approach is based on the idea that there are communities of users that find the same characteristics important to like or dislike a product. This model is an extension of the probabilistic latent semantic model for collaborative filtering with ideas based on clusterwise linear regression. On a movie data set, we show that the model is competitive to other recommenders and can be used to explain the recommendations to the users.algorithms;probabilistic latent semantic analysis;hybrid recommender systems;recommender systems

CBPRS: A City Based Parking and Routing System

Navigational systems assist drivers in finding a route between two locations that is time optimal in theory but seldom in practice due to delaying circumstances the system is unaware of, such as traffic jams. Upon arrival at the destination the service of the system ends and the driver is forced to locate a parking place without further assistance. We propose a City Based Parking Routing System (CBPRS) that monitors and reserves parking places for CBPRS participants within a city. The CBPRS guides vehicles using an ant based distributed hierarchical routing algorithm to their reserved parking place. Through means of experiments in a simulation environment we found that reductions of travel times for participants were significant in comparison to a situation where vehicles relied on static routing information generated by the well known Dijkstra’s algorithm. Furthermore, we found that the CBPRS was able to increase city wide traffic flows and decrease the number and duration of traffic jams throughout the city once the number of participants increased.information systems;computer simulation;dynamic routing

Improved customer choice predictions using ensemble methods

In this paper various ensemble learning methods from machine learning and statistics are considered and applied to the customer choice modeling problem. The application of ensemble learning usually improves the prediction quality of flexible models like decision trees and thus leads to improved predictions. We give experimental results for two real-life marketing datasets using decision trees, ensemble versions of decision trees and the logistic regression model, which is a standard approach for this problem. The ensemble models are found to improve upon individual decision trees and outperform logistic regression. Next, an additive decomposition of the prediction error of a model, the bias/variance decomposition, is considered. A model with a high bias lacks the flexibility to fit the data well. A high variance indicates that a model is instable with respect to different datasets. Decision trees have a high variance component and a low bias component in the prediction error, whereas logistic regression has a high bias component and a low variance component. It is shown that ensemble methods aim at minimizing the variance component in the prediction error while leaving the bias component unaltered. Bias/variance decompositions for all models for both customer choice datasets are given to illustrate these concepts

Generating artificial data with monotonicity constraints

The monotonicity constraint is a common side condition imposed on modeling problems as diverse as hedonic pricing, personnel selection and credit rating. Experience tells us that it is not trivial to generate artificial data for supervised learning problems when the monotonicity constraint holds. Two algorithms are presented in this paper for such learning problems. The first one can be used to generate random monotone data sets without an underlying model, and the second can be used to generate monotone decision tree models. If needed, noise can be added to the generated data. The second algorithm makes use of the first one. Both algorithms are illustrated with an example