Measuring the interactions among variables of functions over the unit hypercube

By considering a least squares approximation of a given square integrable function $f\colon[0,1]^n\to\R$ by a multilinear polynomial of a specified degree, we define an index which measures the overall interaction among variables of $f$. This definition extends the concept of Banzhaf interaction index introduced in cooperative game theory. Our approach is partly inspired from multilinear regression analysis, where interactions among the independent variables are taken into consideration. We show that this interaction index has appealing properties which naturally generalize the properties of the Banzhaf interaction index. In particular, we interpret this index as an expected value of the difference quotients of $f$ or, under certain natural conditions on $f$, as an expected value of the derivatives of $f$. These interpretations show a strong analogy between the introduced interaction index and the overall importance index defined by Grabisch and Labreuche [7]. Finally, we discuss a few applications of the interaction index

Weighted Banzhaf power and interaction indexes through weighted approximations of games

The Banzhaf power index was introduced in cooperative game theory to measure the real power of players in a game. The Banzhaf interaction index was then proposed to measure the interaction degree inside coalitions of players. It was shown that the power and interaction indexes can be obtained as solutions of a standard least squares approximation problem for pseudo-Boolean functions. Considering certain weighted versions of this approximation problem, we define a class of weighted interaction indexes that generalize the Banzhaf interaction index. We show that these indexes define a subclass of the family of probabilistic interaction indexes and study their most important properties. Finally, we give an interpretation of the Banzhaf and Shapley interaction indexes as centers of mass of this subclass of interaction indexes

Bisemivalues for bicooperative games

We introduce bisemivalues for bicooperative games and we also provide an interesting characterization of this kind of values by means of weighting coefficients in a similar way as it was given for semivalues in the context of cooperative games. Moreover, the notion of induced bisemivalues on lower cardinalities also makes sense and an adaptation of Dragan’s recurrence formula is obtained. For the particular case of (p, q)-bisemivalues, a computational procedure in terms of the multilinear extension of the game is given.Peer ReviewedPostprint (author's final draft

A note on the Sobol' indices and interactive criteria

The Choquet integral and the Owen extension (or multilinear extension) are the most popular tools in multicriteria decision making to take into account the interaction between criteria. It is known that the interaction transform and the Banzhaf interaction transform arise as the average total variation of the Choquet integral and multilinear extension respectively. We consider in this note another approach to define interaction, by using the Sobol' indices which are related to the analysis of variance of a multivariate model. We prove that the Sobol' indices of the multilinear extension gives the square of the Fourier transform, a well-known concept in computer sciences. We also relate the latter to the Banzhaf interaction transform and compute the Sobol' indices for the 2-additive Choquet integral

Approximations of Lovasz extensions and their induced interaction index

The Lovasz extension of a pseudo-Boolean function $f : \{0,1\}^n \to R$ is defined on each simplex of the standard triangulation of $[0,1]^n$ as the unique affine function $\hat f : [0,1]^n \to R$ that interpolates $f$ at the $n+1$ vertices of the simplex. Its degree is that of the unique multilinear polynomial that expresses $f$. In this paper we investigate the least squares approximation problem of an arbitrary Lovasz extension $\hat f$ by Lovasz extensions of (at most) a specified degree. We derive explicit expressions of these approximations. The corresponding approximation problem for pseudo-Boolean functions was investigated by Hammer and Holzman (1992) and then solved explicitly by Grabisch, Marichal, and Roubens (2000), giving rise to an alternative definition of Banzhaf interaction index. Similarly we introduce a new interaction index from approximations of $\hat f$ and we present some of its properties. It turns out that its corresponding power index identifies with the power index introduced by Grabisch and Labreuche (2001).Comment: 19 page

Discrete and fuzzy dynamical genetic programming in the XCSF learning classifier system

A number of representation schemes have been presented for use within learning classifier systems, ranging from binary encodings to neural networks. This paper presents results from an investigation into using discrete and fuzzy dynamical system representations within the XCSF learning classifier system. In particular, asynchronous random Boolean networks are used to represent the traditional condition-action production system rules in the discrete case and asynchronous fuzzy logic networks in the continuous-valued case. It is shown possible to use self-adaptive, open-ended evolution to design an ensemble of such dynamical systems within XCSF to solve a number of well-known test problems

Symmetric approximations of pseudo-Boolean functions with applications to influence indexes

We introduce an index for measuring the influence of the k-th smallest variable on a pseudo-Boolean function. This index is defined from a weighted least squares approximation of the function by linear combinations of order statistic functions. We give explicit expressions for both the index and the approximation and discuss some properties of the index. Finally, we show that this index subsumes the concept of system signature in engineering reliability and that of cardinality index in decision making

Derivative of functions over lattices as a basis for the notion of interaction between attributes

The paper proposes a general notion of interaction between attributes, which can be applied to many fields in decision making and data analysis. It generalizes the notion of interaction defined for criteria modelled by capacities, by considering functions defined on lattices. For a given problem, the lattice contains for each attribute the partially ordered set of remarkable points or levels. The interaction is based on the notion of derivative of a function defined on a lattice, and appears as a generalization of the Shapley value or other probabilistic values

A POWER INDEX BASED FRAMEWORKFOR FEATURE SELECTION PROBLEMS

One of the most challenging tasks in the Machine Learning context is the feature selection. It consists in selecting the best set of features to use in the training and prediction processes. There are several benefits from pruning the set of actually operational features: the consequent reduction of the computation time, often a better quality of the prediction, the possibility to use less data to create a good predictor. In its most common form, the problem is called single-view feature selection problem, to distinguish it from the feature selection task in Multi-view learning. In the latter, each view corresponds to a set of features and one would like to enact feature selection on each view, subject to some global constraints. A related problem in the context of Multi-View Learning, is Feature Partitioning: it consists in splitting the set of features of a single large view into two or more views so that it becomes possible to create a good predictor based on each view. In this case, the best features must be distributed between the views, each view should contain synergistic features, while features that interfere disruptively must be placed in different views. In the semi-supervised multi-view task known as Co-training, one requires also that each predictor trained on an individual view is able to teach something to the other views: in classification tasks for instance, one view should learn to classify unlabelled examples based on the guess provided by the other views. There are several ways to address these problems. A set of techniques is inspired by Coalitional Game Theory. Such theory defines several useful concepts, among which two are of high practical importance: the concept of power index and the concept of interaction index. When used in the context of feature selection, they take the following meaning: the power index is a (context-dependent) synthesis measure of the prediction\u2019s capability of a feature, the interaction index is a (context-dependent) synthesis measure of the interaction (constructive/disruptive interference) between two features: it can be used to quantify how the collaboration between two features enhances their prediction capabilities. An important point is that the powerindex of a feature is different from the predicting power of the feature in isolation: it takes into account, by a suitable averaging, the context, i.e. the fact that the feature is acting, together with other features, to train a model. Similarly, the interaction index between two features takes into account the context, by suitably averaging the interaction with all the other features. In this work we address both the single-view and the multi-view problems as follows. The single-view feature selection problem, is formalized as the problem of maximization of a pseudo-boolean function, i.e. a real valued set function (that maps sets of features into a performance metric). Since one has to enact a search over (a considerable portion of) the Boolean lattice (without any special guarantees, except, perhaps, positivity) the problem is in general NP-hard. We address the problem producing candidate maximum coalitions through the selection of the subset of features characterized by the highest power indices and using the coalition to approximate the actual maximum. Although the exact computation of the power indices is an exponential task, the estimates of the power indices for the purposes of the present problem can be achieved in polynomial time. The multi-view feature selection problem is formalized as the generalization of the above set-up to the case of multi-variable pseudo-boolean functions. The multi-view splitting problem is formalized instead as the problem of maximization of a real function defined over the partition lattice. Also this problem is typically NP-hard. However, candidate solutions can be found by suitably partitioning the top power-index features and keeping in different views the pairs of features that are less interactive or negatively interactive. The sum of the power indices of the participating features can be used to approximate the prediction capability of the view (i.e. they can be used as a proxy for the predicting power). The sum of the feature pair interactivity across views can be used as proxy for the orthogonality of the views. Also the capability of a view to pass information (to teach) to other views, within a co-training procedure can benefit from the use of power indices based on a suitable definition of information transfer (a set of features { a coalition { classifies examples that are subsequently used in the training of a second set of features). As to the feature selection task, not only we demonstrate the use of state of the art power index concepts (e.g. Shapley Value and Banzhaf along the 2lines described above Value), but we define new power indices, within the more general class of probabilistic power indices, that contains the Shapley and the Banzhaf Values as special cases. Since the number of features to select is often a predefined parameter of the problem, we also introduce some novel power indices, namely k-Power Index (and its specializations k-Shapley Value, k-Banzhaf Value): they help selecting the features in a more efficient way. For the feature partitioning, we use the more general class of probabilistic interaction indices that contains the Shapley and Banzhaf Interaction Indices as members. We also address the problem of evaluating the teaching ability of a view, introducing a suitable teaching capability index. The last contribution of the present work consists in comparing the Game Theory approach to the classical Greedy Forward Selection approach for feature selection. In the latter the candidate is obtained by aggregating one feature at time to the current maximal coalition, by choosing always the feature with the maximal marginal contribution. In this case we show that in typical cases the two methods are complementary, and that when used in conjunction they reduce one another error in the estimate of the maximum value. Moreover, the approach based on game theory has two advantages: it samples the space of all possible features\u2019 subsets, while the greedy algorithm scans a selected subspace excluding totally the rest of it, and it is able, for each feature, to assign a score that describes a context-aware measure of importance in the prediction process