PAC models in stochastic multi-objective multi-armed bandits

\u3cp\u3eMany real-world applications, such as stock markets, energy consumption time series, and scheduling in noisy environments, are characterised by stochastic feedback. In this paper, the evolutionary multi-objective (EMO) techniques, like elitist selection strategies, and the probably approximatively correct (PAC) model are used to analyse the multi-armed bandits (MAB) paradigm that identifies the Pareto front from a finite set of arms with stochastic reward vectors. Each arm is associated with a confidence ball centred in the sampling's mean vector that decreases towards its true vector when the number of samples increases. The Pareto lower upper confidence bound algorithm samples the alternatives for which their confidence ball overlaps with the confidence regions of the Pareto optimal arms. Pareto racing deletes the arms classified with certainty as either suboptimal or Pareto optimal arms. The sample complexity estimates the number of samples required for an accurate approximation of the Pareto front using two different statistics, i.e. empirically determined means or quantiles. The analysed PAC models are empirically compared on realistic datasets with two and three objectives.\u3c/p\u3

Sets of interacting scalarization functions in local search for multi-objective combinatorial optimization problems

Searching in multi-objective search spaces is considered a challenging problem. Pareto local search (PLS) searches directly into the multi-objective search space maintaining an archive of best non-dominated solutions found so far, the non-dominated archive. PLS' advantage is the exploitation of relationships between solutions in the non-dominated archive at the cost of high maintenance costs of the archive. The scalarized local search (SLS) uses scalarization functions to transform the multi-objective search space into a single objective search space. SLS is faster because it is searching in a single objective search space but the independent scalarization functions do not systematic exploit the structure of the multi-objective search space. We improve the performance of SLS algorithms by allowing interactions between scalarization functions. The adaptive scalarization functions select frequently the scalarization function that generates well performing SLS. The genetic scalarization functions assume that the scalarization functions have commonalities that can be exploited using genetic like operators. We experimentally show that the proposed techniques can improve the performance of local search algorithms on correlated bi-objective QAP in-stances

Generating QAP instances with known optimum solution and additively decomposable cost function

Quadratic assignment problems (QAPs) is a NP-hard combinatorial optimization problem. QAPs are often used to compare the performance of meta-heuristics. In this paper, we propose a QAP problem instance generator that can be used for benchmarking for heuristic algorithms. Our QAP generator combines small size QAPs with known optimum solution into a larger size QAP instance. We call these instances composite QAPs (cQAPs), and we show that the cost function of cQAPs is additively decomposable. We give mild conditions for which a cQAP instance has known optimum solution. We generate cQAP instances using uniform distributions with different bounds for the component QAPs and for the rest of the cQAP elements. Numerical and analytical techniques that measure the difficulty of the cQAP instances in comparison with other QAPs from the literature are introduced. These methods point out that some cQAP instances are difficult for local search with many local optimum of various values, low epistasis and non-trivial asymptotic behaviour

A Bayesian model for anomaly detection in SQL databases for security systems

We focus on automatic anomaly detection in SQL databases for security systems.\u3cbr/\u3eMany logs of database systems, here the Townhall database, contain detailed information about users, like the SQL queries and the response of the database.\u3cbr/\u3eA database is a list of log instances, where each log instance is a Cartesian product of feature values with an attached anomaly score. All log instances with the anomaly score in the top percentile are identified as anomalous. Our contribution is multi-folded. We define a model for anomaly detection of SQL databases that learns the structure of Bayesian networks from data. Our method for automatic feature extraction generates the maximal spanning tree to detect the strongest similarities between features. Novel anomaly scores based on the joint probability distribution of the database features and the log-likelihood of the maximal spanning tree detect both point and contextual anomalies. Multiple anomaly scores are combined within a robust anomaly analysis algorithm. We validate our method on the Townhall database showing the performance of our anomaly detection algorithm

Multi-objective quadratic assignment problem instances generator with a known optimum solution

\u3cp\u3eMulti-objective quadratic assignment problems (mQAPs) are NP-hard problems that optimally allocate facilities to locations using a distance matrix and several flow matrices. mQAPs are often used to compare the performance of the multi-objective meta-heuristics. We generate large mQAP instances by combining small size mQAP with known local optimum. We call these instances composite mQAPs, and we show that the cost function of these mQAPs is additively decomposable. We give mild conditions for which a composite mQAP instance has known optimum solution.We generate composite mQAP instances using a set of uniform distributions that obey these conditions. Using numerical experiments we show that composite mQAPs are difficult for multi-objective meta-heuristics.\u3c/p\u3

Infinite horizon multi-armed bandits with reward vectors:exploration/exploitation trade-off

\u3cp\u3eWe focus on the effect of the exploration/exploitation tradeoff strategies on the algorithmic design off multi-armed bandits (MAB) with reward vectors. Pareto dominance relation assesses the quality of reward vectors in infinite horizon MABs, like the UCB1 and UCB2 algorithms. In single objective MABs, there is a trade-off between the exploration of the suboptimal arms, and exploitation of a single optimal arm. Pareto dominance based MABs fairly exploit all Pareto optimal arms, and explore suboptimal arms. We study the exploration vs exploitation trade-off for two UCB like algorithms for reward vectors. We analyse the properties of the proposed MAB algorithms in terms of upper regret bounds and we experimentally compare their exploration vs exploitation trade-off on a bi-objective Bernoulli environment coming from control theory.\u3c/p\u3

Generalized adaptive pursuit algorithm for genetic pareto local search algorithms

The standard adaptive pursuit technique (AP) shows preference for a single operator at a time but is not able to simultaneously pursue multiple operators. We generalize AP by allowing any target distribution to be pursued for operator selection probabilities. We call this the generalized adaptive pursuit algorithm (GAPA). We show that the probability matching and multi-armed bandit strategies, with particular settings, can be integrated in the GAPA framework. We propose and experimentally test two instances of GAPA. Assuming that there are multiple useful operators, the multi-operator AP pursues them all simultaneously. The multi-layer AP is intended to scale up the pursuit algorithm to a large set of operators. To experimentally test the proposed GAPA instances, we introduce the adaptive genetic Pareto local search (aGPLS) that selects on-line genetic operators to restart the Pareto local search. We show on a bi-objective Quadratic assignment problem (bQAP) instance with a large number of facilities and high correlation that aGPLSs are the algorithms with best performance tested

Designing multi-objective multi-armed bandits algorithms : a study

We propose an algorithmic framework for multi-objective multi-armed bandits with multiple rewards. Different partial order relationships from multi-objective optimization can be considered for a set of reward vectors, such as scalarization functions and Pareto search. A scalarization function transforms the multi-objective environment into a single objective environment and are a popular choice in multi-objective reinforcement learning. Scalarization techniques can be straightforwardly implemented into the current multi-armed bandit framework, but the efficiency of these algorithms depends very much on their type, linear or non-linear (e.g. Chebyshev), and their parameters. Using Pareto dominance order relationship allows to explore the multi-objective environment directly, however this can result in large sets of Pareto optimal solutions. In this paper we propose and evaluate the performance of multi-objective MABs using three regret metric criteria. The standard UCB1 is extended to scalarized multi-objective UCB1 and we propose a Pareto UCB1 algorithm. Both algorithms are proven to have a logarithmic upper bound for their expected regret. We also introduce a variant of the scalarized multi-objective UCB1 that removes online inefficient scalarizations in order to improve the algorithm's efficiency. These algorithms are experimentally compared on multi-objective Bernoulli distributions, Pareto UCB1 being the algorithm with the best empirical performance

Feature selection for Bayesian network classifiers using the MDL-FS score

When constructing a Bayesian network classifier from data, the more or less redundant features included in a dataset may bias the classifier and as a consequence may result in a relatively poor classification accuracy. In this paper, we study the problem of selecting appropriate subsets of features for such classifiers. To this end, we propose a new definition of the concept of redundancy in noisy data. For comparing alternative classifiers, we use the Minimum Description Length for Feature Selection (MDL-FS) function that we introduced before. Our function differs from the well-known MDL function in that it captures a classifier’s conditional log-likelihood. We show that the MDL-FS function serves to identify redundancy at different levels and is able to eliminate redundant features from different types of classifier. We support our theoretical findings by comparing the feature-selection behaviours of the various functions in a practical setting. Our results indicate that the MDL-FS function is more suited to the task of feature selection than MDL as it often yields classifiers of equal or better performance with significantly fewer attributes