Transforms of pseudo-Boolean random variables

As in earlier works, we consider {0, 1}n as a sample space with a probability measure on it, thus making pseudo-Boolean functions into random variables. Under the assumption that the coordinate random variables are independent, we show it is very easy to give an orthonormal basis for the space of pseudo-Boolean random variables of degree at most k. We use this orthonormal basis to find the transform of a given pseudo-Boolean random variable and to answer various least squares minimization questions. © 2009 Elsevier B.V. All rights reserved

Weighted Banzhaf power and interaction indexes through weighted approximations of games

The Banzhaf power index was introduced in cooperative game theory to measure the real power of players in a game. The Banzhaf interaction index was then proposed to measure the interaction degree inside coalitions of players. It was shown that the power and interaction indexes can be obtained as solutions of a standard least squares approximation problem for pseudo-Boolean functions. Considering certain weighted versions of this approximation problem, we define a class of weighted interaction indexes that generalize the Banzhaf interaction index. We show that these indexes define a subclass of the family of probabilistic interaction indexes and study their most important properties. Finally, we give an interpretation of the Banzhaf and Shapley interaction indexes as centers of mass of this subclass of interaction indexes

Least Square Approximations and Conic Values of Cooperative Games

URL des Documents de travail : http://centredeconomiesorbonne.univ-paris1.fr/documents-de-travail/Documents de travail du Centre d'Economie de la Sorbonne 2015.47 - ISSN : 1955-611XThe problem of least square approximation for set functions by set functions satisfying specified linear equality or inequality constraints is considered. The problem has important applications in the field of pseudo-Boolean functions, decision making and in cooperative game theory, where approximation by additive set functions yields so-called least square values. In fact, it is seem that every linear value for cooperative games arises from least square approximation. We provide a general approach and problem overview. In particular, we derive explicit formulas for solutions under mild constraints, which include and extend previous results in the literature.On considère le problème de l'approximation au sens des moindres carrés des fonctions d'ensemble par des fonctions d'ensemble satisfaisant des contraintes linéaires d'égalité ou d'inégalité. Le problème a des applications importantes dans le domaine des fonctions pseudo-Booléennes, la décision et la théorie des jeux coopératifs, où l'approximation par des jeux additifs mène à la notion de valeur aux moindres carrés. En fait, on voit que toute valeur linéaire pour les jeux coopératifs vient d'un problème d'approximation par les moindres carrés. Nous proposons une approche générale du problème. En particulier, nous obtenons des formules explicites pour les solutions sous des hypothèses faibles, qui incluent et étendent des résultats précédents de la littérature

A POWER INDEX BASED FRAMEWORKFOR FEATURE SELECTION PROBLEMS

One of the most challenging tasks in the Machine Learning context is the feature selection. It consists in selecting the best set of features to use in the training and prediction processes. There are several benefits from pruning the set of actually operational features: the consequent reduction of the computation time, often a better quality of the prediction, the possibility to use less data to create a good predictor. In its most common form, the problem is called single-view feature selection problem, to distinguish it from the feature selection task in Multi-view learning. In the latter, each view corresponds to a set of features and one would like to enact feature selection on each view, subject to some global constraints. A related problem in the context of Multi-View Learning, is Feature Partitioning: it consists in splitting the set of features of a single large view into two or more views so that it becomes possible to create a good predictor based on each view. In this case, the best features must be distributed between the views, each view should contain synergistic features, while features that interfere disruptively must be placed in different views. In the semi-supervised multi-view task known as Co-training, one requires also that each predictor trained on an individual view is able to teach something to the other views: in classification tasks for instance, one view should learn to classify unlabelled examples based on the guess provided by the other views. There are several ways to address these problems. A set of techniques is inspired by Coalitional Game Theory. Such theory defines several useful concepts, among which two are of high practical importance: the concept of power index and the concept of interaction index. When used in the context of feature selection, they take the following meaning: the power index is a (context-dependent) synthesis measure of the prediction\u2019s capability of a feature, the interaction index is a (context-dependent) synthesis measure of the interaction (constructive/disruptive interference) between two features: it can be used to quantify how the collaboration between two features enhances their prediction capabilities. An important point is that the powerindex of a feature is different from the predicting power of the feature in isolation: it takes into account, by a suitable averaging, the context, i.e. the fact that the feature is acting, together with other features, to train a model. Similarly, the interaction index between two features takes into account the context, by suitably averaging the interaction with all the other features. In this work we address both the single-view and the multi-view problems as follows. The single-view feature selection problem, is formalized as the problem of maximization of a pseudo-boolean function, i.e. a real valued set function (that maps sets of features into a performance metric). Since one has to enact a search over (a considerable portion of) the Boolean lattice (without any special guarantees, except, perhaps, positivity) the problem is in general NP-hard. We address the problem producing candidate maximum coalitions through the selection of the subset of features characterized by the highest power indices and using the coalition to approximate the actual maximum. Although the exact computation of the power indices is an exponential task, the estimates of the power indices for the purposes of the present problem can be achieved in polynomial time. The multi-view feature selection problem is formalized as the generalization of the above set-up to the case of multi-variable pseudo-boolean functions. The multi-view splitting problem is formalized instead as the problem of maximization of a real function defined over the partition lattice. Also this problem is typically NP-hard. However, candidate solutions can be found by suitably partitioning the top power-index features and keeping in different views the pairs of features that are less interactive or negatively interactive. The sum of the power indices of the participating features can be used to approximate the prediction capability of the view (i.e. they can be used as a proxy for the predicting power). The sum of the feature pair interactivity across views can be used as proxy for the orthogonality of the views. Also the capability of a view to pass information (to teach) to other views, within a co-training procedure can benefit from the use of power indices based on a suitable definition of information transfer (a set of features { a coalition { classifies examples that are subsequently used in the training of a second set of features). As to the feature selection task, not only we demonstrate the use of state of the art power index concepts (e.g. Shapley Value and Banzhaf along the 2lines described above Value), but we define new power indices, within the more general class of probabilistic power indices, that contains the Shapley and the Banzhaf Values as special cases. Since the number of features to select is often a predefined parameter of the problem, we also introduce some novel power indices, namely k-Power Index (and its specializations k-Shapley Value, k-Banzhaf Value): they help selecting the features in a more efficient way. For the feature partitioning, we use the more general class of probabilistic interaction indices that contains the Shapley and Banzhaf Interaction Indices as members. We also address the problem of evaluating the teaching ability of a view, introducing a suitable teaching capability index. The last contribution of the present work consists in comparing the Game Theory approach to the classical Greedy Forward Selection approach for feature selection. In the latter the candidate is obtained by aggregating one feature at time to the current maximal coalition, by choosing always the feature with the maximal marginal contribution. In this case we show that in typical cases the two methods are complementary, and that when used in conjunction they reduce one another error in the estimate of the maximum value. Moreover, the approach based on game theory has two advantages: it samples the space of all possible features\u2019 subsets, while the greedy algorithm scans a selected subspace excluding totally the rest of it, and it is able, for each feature, to assign a score that describes a context-aware measure of importance in the prediction process