A PERTURBATION­BASED APPROACH FOR MULTI­CLASSIFIER SYSTEM DESIGN

Microsoft, Motorola, Siemens, Hitachi, IAPR, NICI, IUF This paper presents a perturbation­based approach useful to select the best combination method for a multi­classifier system. The basic idea is to simulate small variations in the performance of the set of classifiers and to evaluate to what extent they influence the performance of the combined classifier. In the experimental phase, the Behavioural Knowledge Space and the Dempster­Shafer combination methods have been considered. The experimental results, carried out in the field of hand­written numeral recognition, demonstrate the effectiveness of the new approach

ZONING DESIGN FOR HAND­WRITTEN NUMERAL RECOGNITION

Microsoft, Motorola, Siemens, Hitachi, IAPR, NICI, IUF In the field of Optical Character Recognition (OCR), zoning is used to extract topological information from patterns. In this paper zoning is considered as the result of an optimisation problem and a new technique is presented for automatic zoning. More precisely, local analysis of feature distribution based on Shannon's entropy estimation is performed to determine "core" zones of patterns. An iterative region­growing procedure is applied on the "core" zones to determine the final zoning

On the Iteration Complexity of Hypergradient Computation

We study a general class of bilevel problems, consisting in the minimization of an upper-level objective which depends on the solution to a parametric fixed-point equation. Important instances arising in machine learning include hyperparameter optimization, meta-learning, and certain graph and recurrent neural networks. Typically the gradient of the upper-level objective (hypergradient) is hard or even impossible to compute exactly, which has raised the interest in approximation methods. We investigate some popular approaches to compute the hypergradient, based on reverse mode iterative differentiation and approximate implicit differentiation. Under the hypothesis that the fixed point equation is defined by a contraction mapping, we present a unified analysis which allows for the first time to quantitatively compare these methods, providing explicit bounds for their iteration complexity. This analysis suggests a hierarchy in terms of computational efficiency among the above methods, with approximate implicit differentiation based on conjugate gradient performing best. We present an extensive experimental comparison among the methods which confirm the theoretical findings

Bilevel Programming for Hyperparameter Optimization and Meta-Learning

We introduce a framework based on bilevel programming that unifies gradient-based hyperparameter optimization and meta-learning. We show that an approximate version of the bilevel problem can be solved by taking into explicit account the optimization dynamics for the inner objective. Depending on the specific setting, the outer variables take either the meaning of hyperparameters in a supervised learning problem or parameters of a meta-learner. We provide sufficient conditions under which solutions of the approximate problem converge to those of the exact problem. We instantiate our approach for meta-learning in the case of deep learning where representation layers are treated as hyperparameters shared across a set of training episodes. In experiments, we confirm our theoretical findings, present encouraging results for few-shot learning and contrast the bilevel approach against classical approaches for learning-to-learn

Increasing the Number of Classifiers in Multi-classifier Systems: A Complementarity-Based Analysis

n/