10 research outputs found
SFO: A Toolbox for Submodular Function Optimization
In recent years, a fundamental problem structure has emerged as very useful in a variety of machine learning applications: Submodularity is an intuitive diminishing returns property, stating that adding an element to a smaller set helps more than adding it to a larger set. Similarly to convexity, submodularity allows one to efficiently find provably (near-) optimal solutions for large problems. We present SFO, a toolbox for use in MATLAB or Octave that implements algorithms for minimization and maximization of submodular functions. A tutorial script illustrates the application of submodularity to machine learning and AI problems such as feature selection, clustering, inference and optimized information gathering
A note on the Minimum Norm Point algorithm
We present a provably more efficient implementation of the Minimum Norm Point
Algorithm conceived by Fujishige than the one presented in \cite{FUJI06}. The
algorithm solves the minimization problem for a class of functions known as
submodular. Many important functions, such as minimum cut in the graph, have
the so called submodular property \cite{FUJI82}. It is known that the problem
can also be efficiently solved in strongly polynomial time \cite{IWAT01},
however known theoretical bounds are far from being practical. We present an
improved implementation of the algorithm, for which unfortunately no worst case
bounds are know, but which performs very well in practice. With the
modifications presented, the algorithm performs an order of magnitude faster
for certain submodular functions
The Role of Synthetic Geometry in Representational Measurement Theory
Geometric representations of data and the formulation of quantitative models of observed phenomena are of main interest in all kinds of empirical sciences. To support the formulation of quantitative models, {\it representational measurement theory} studies the foundations of measurement. By mathematical methods it is analysed under which conditions attributes have numerical measurements and which numerical manipulations of the measurement values are meaningful (see Krantz et al.~(1971)). In this paper, we suggest to discuss within the measurement theory approach both, the idea of geometric representations of data and the request to provide algebraic descriptions of dependencies of attributes. We show that, within such a broader paradigm of representational measurement theory, synthetic geometry can play a twofold role which enriches the theory and the possibilities of data interpretation
Efficient Minimization of Higher Order Submodular Functions using Monotonic Boolean Functions
Submodular function minimization is a key problem in a wide variety of
applications in machine learning, economics, game theory, computer vision, and
many others. The general solver has a complexity of where is the time required to evaluate the function and
is the number of variables \cite{Lee2015}. On the other hand, many computer
vision and machine learning problems are defined over special subclasses of
submodular functions that can be written as the sum of many submodular cost
functions defined over cliques containing few variables. In such functions, the
pseudo-Boolean (or polynomial) representation \cite{BorosH02} of these
subclasses are of degree (or order, or clique size) where . In
this work, we develop efficient algorithms for the minimization of this useful
subclass of submodular functions. To do this, we define novel mapping that
transform submodular functions of order into quadratic ones. The underlying
idea is to use auxiliary variables to model the higher order terms and the
transformation is found using a carefully constructed linear program. In
particular, we model the auxiliary variables as monotonic Boolean functions,
allowing us to obtain a compact transformation using as few auxiliary variables
as possible
Optimal Bounds on Approximation of Submodular and XOS Functions by Juntas
We investigate the approximability of several classes of real-valued
functions by functions of a small number of variables ({\em juntas}). Our main
results are tight bounds on the number of variables required to approximate a
function within -error over
the uniform distribution: 1. If is submodular, then it is -close
to a function of variables.
This is an exponential improvement over previously known results. We note that
variables are necessary even for linear
functions. 2. If is fractionally subadditive (XOS) it is -close
to a function of variables. This result holds for all
functions with low total -influence and is a real-valued analogue of
Friedgut's theorem for boolean functions. We show that
variables are necessary even for XOS functions.
As applications of these results, we provide learning algorithms over the
uniform distribution. For XOS functions, we give a PAC learning algorithm that
runs in time . For submodular functions we give
an algorithm in the more demanding PMAC learning model (Balcan and Harvey,
2011) which requires a multiplicative factor approximation with
probability at least over the target distribution. Our uniform
distribution algorithm runs in time .
This is the first algorithm in the PMAC model that over the uniform
distribution can achieve a constant approximation factor arbitrarily close to 1
for all submodular functions. As follows from the lower bounds in (Feldman et
al., 2013) both of these algorithms are close to optimal. We also give
applications for proper learning, testing and agnostic learning with value
queries of these classes.Comment: Extended abstract appears in proceedings of FOCS 201