214 research outputs found
On the ERM Principle with Networked Data
Networked data, in which every training example involves two objects and may
share some common objects with others, is used in many machine learning tasks
such as learning to rank and link prediction. A challenge of learning from
networked examples is that target values are not known for some pairs of
objects. In this case, neither the classical i.i.d.\ assumption nor techniques
based on complete U-statistics can be used. Most existing theoretical results
of this problem only deal with the classical empirical risk minimization (ERM)
principle that always weights every example equally, but this strategy leads to
unsatisfactory bounds. We consider general weighted ERM and show new universal
risk bounds for this problem. These new bounds naturally define an optimization
problem which leads to appropriate weights for networked examples. Though this
optimization problem is not convex in general, we devise a new fully
polynomial-time approximation scheme (FPTAS) to solve it.Comment: accepted by AAAI. arXiv admin note: substantial text overlap with
arXiv:math/0702683 by other author
Lifted Algorithms for Symmetric Weighted First-Order Model Sampling
Weighted model counting (WMC) is the task of computing the weighted sum of
all satisfying assignments (i.e., models) of a propositional formula.
Similarly, weighted model sampling (WMS) aims to randomly generate models with
probability proportional to their respective weights. Both WMC and WMS are hard
to solve exactly, falling under the -hard complexity class.
However, it is known that the counting problem may sometimes be tractable, if
the propositional formula can be compactly represented and expressed in
first-order logic. In such cases, model counting problems can be solved in time
polynomial in the domain size, and are known as domain-liftable. The following
question then arises: Is it also the case for weighted model sampling? This
paper addresses this question and answers it affirmatively. Specifically, we
prove the domain-liftability under sampling for the two-variables fragment of
first-order logic with counting quantifiers in this paper, by devising an
efficient sampling algorithm for this fragment that runs in time polynomial in
the domain size. We then further show that this result continues to hold even
in the presence of cardinality constraints. To empirically verify our approach,
we conduct experiments over various first-order formulas designed for the
uniform generation of combinatorial structures and sampling in
statistical-relational models. The results demonstrate that our algorithm
outperforms a start-of-the-art WMS sampler by a substantial margin, confirming
the theoretical results.Comment: 47 pages, 6 figures. An expanded version of "On exact sampling in the
two-variable fragment of first-order logic" in LICS23, submitted to AIJ.
arXiv admin note: substantial text overlap with arXiv:2302.0273
Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution
Strong intelligent machines powered by deep neural networks are increasingly
deployed as black boxes to make decisions in risk-sensitive domains, such as
finance and medical. To reduce potential risk and build trust with users, it is
critical to interpret how such machines make their decisions. Existing works
interpret a pre-trained neural network by analyzing hidden neurons, mimicking
pre-trained models or approximating local predictions. However, these methods
do not provide a guarantee on the exactness and consistency of their
interpretation. In this paper, we propose an elegant closed form solution named
to compute exact and consistent interpretations for the family of
Piecewise Linear Neural Networks (PLNN). The major idea is to first transform a
PLNN into a mathematically equivalent set of linear classifiers, then interpret
each linear classifier by the features that dominate its prediction. We further
apply to demonstrate the effectiveness of non-negative and sparse
constraints on improving the interpretability of PLNNs. The extensive
experiments on both synthetic and real world data sets clearly demonstrate the
exactness and consistency of our interpretation.Comment: KDD 201
- …