333,485 research outputs found
Chain Reduction for Binary and Zero-Suppressed Decision Diagrams
Chain reduction enables reduced ordered binary decision diagrams (BDDs) and
zero-suppressed binary decision diagrams (ZDDs) to each take advantage of the
others' ability to symbolically represent Boolean functions in compact form.
For any Boolean function, its chain-reduced ZDD (CZDD) representation will be
no larger than its ZDD representation, and at most twice the size of its BDD
representation. The chain-reduced BDD (CBDD) of a function will be no larger
than its BDD representation, and at most three times the size of its CZDD
representation. Extensions to the standard algorithms for operating on BDDs and
ZDDs enable them to operate on the chain-reduced versions. Experimental
evaluations on representative benchmarks for encoding word lists, solving
combinatorial problems, and operating on digital circuits indicate that chain
reduction can provide significant benefits in terms of both memory and
execution time
Composite multiclass losses
We consider loss functions for multiclass prediction problems. We show when a multiclass loss can be expressed as a “proper composite loss”, which is the composition of a proper loss and a link function. We extend existing results for binary losses to multiclass losses. We subsume results on “classification calibration” by relating it to properness. We determine the stationarity condition, Bregman representation, order-sensitivity, and quasi-convexity of multiclass proper losses. We then characterise the existence and uniqueness of the composite representation formulti class losses. We show how the composite representation is related to other core properties of a loss: mixability, admissibility and (strong) convexity of multiclass losses which we characterise in terms of the Hessian of the Bayes risk. We show that the simple integral representation for binary proper losses can not be extended to multiclass losses but offer concrete guidance regarding how to design different loss functions. The conclusion drawn from these results is that the proper composite representation is a natural and convenient tool for the design of multiclass loss functions
Solving Multiclass Learning Problems via Error-Correcting Output Codes
Multiclass learning problems involve finding a definition for an unknown
function f(x) whose range is a discrete set containing k > 2 values (i.e., k
``classes''). The definition is acquired by studying collections of training
examples of the form [x_i, f (x_i)]. Existing approaches to multiclass learning
problems include direct application of multiclass algorithms such as the
decision-tree algorithms C4.5 and CART, application of binary concept learning
algorithms to learn individual binary functions for each of the k classes, and
application of binary concept learning algorithms with distributed output
representations. This paper compares these three approaches to a new technique
in which error-correcting codes are employed as a distributed output
representation. We show that these output representations improve the
generalization performance of both C4.5 and backpropagation on a wide range of
multiclass learning tasks. We also demonstrate that this approach is robust
with respect to changes in the size of the training sample, the assignment of
distributed representations to particular classes, and the application of
overfitting avoidance techniques such as decision-tree pruning. Finally, we
show that---like the other methods---the error-correcting code technique can
provide reliable class probability estimates. Taken together, these results
demonstrate that error-correcting output codes provide a general-purpose method
for improving the performance of inductive learning programs on multiclass
problems.Comment: See http://www.jair.org/ for any accompanying file
Geometric generalisation of surrogate model-based optimisation to combinatorial and program spaces
Open access journalSurrogate models (SMs) can profitably be employed, often in conjunction with evolutionary algorithms, in optimisation in which it is expensive to test candidate solutions. The spatial intuition behind SMs makes them naturally suited to continuous problems, and the only combinatorial problems that have been previously addressed are those with solutions that can be encoded as integer vectors. We show how radial basis functions can provide a generalised SM for combinatorial problems which have a geometric solution representation, through the conversion of that representation to a different metric space. This approach allows an SM to be cast in a natural way for the problem at hand, without ad hoc adaptation to a specific representation. We test this adaptation process on problems involving binary strings, permutations, and tree-based genetic programs. © 2014 Yong-Hyuk Kim et al
Constrained Optimization of Rank-One Functions with Indicator Variables
Optimization problems involving minimization of a rank-one convex function
over constraints modeling restrictions on the support of the decision variables
emerge in various machine learning applications. These problems are often
modeled with indicator variables for identifying the support of the continuous
variables. In this paper we investigate compact extended formulations for such
problems through perspective reformulation techniques. In contrast to the
majority of previous work that relies on support function arguments and
disjunctive programming techniques to provide convex hull results, we propose a
constructive approach that exploits a hidden conic structure induced by
perspective functions. To this end, we first establish a convex hull result for
a general conic mixed-binary set in which each conic constraint involves a
linear function of independent continuous variables and a set of binary
variables. We then demonstrate that extended representations of sets associated
with epigraphs of rank-one convex functions over constraints modeling indicator
relations naturally admit such a conic representation. This enables us to
systematically give perspective formulations for the convex hull descriptions
of these sets with nonlinear separable or non-separable objective functions,
sign constraints on continuous variables, and combinatorial constraints on
indicator variables. We illustrate the efficacy of our results on sparse
nonnegative logistic regression problems
- …