2,334 research outputs found
Consistent Second-Order Conic Integer Programming for Learning Bayesian Networks
Bayesian Networks (BNs) represent conditional probability relations among a
set of random variables (nodes) in the form of a directed acyclic graph (DAG),
and have found diverse applications in knowledge discovery. We study the
problem of learning the sparse DAG structure of a BN from continuous
observational data. The central problem can be modeled as a mixed-integer
program with an objective function composed of a convex quadratic loss function
and a regularization penalty subject to linear constraints. The optimal
solution to this mathematical program is known to have desirable statistical
properties under certain conditions. However, the state-of-the-art optimization
solvers are not able to obtain provably optimal solutions to the existing
mathematical formulations for medium-size problems within reasonable
computational times. To address this difficulty, we tackle the problem from
both computational and statistical perspectives. On the one hand, we propose a
concrete early stopping criterion to terminate the branch-and-bound process in
order to obtain a near-optimal solution to the mixed-integer program, and
establish the consistency of this approximate solution. On the other hand, we
improve the existing formulations by replacing the linear "big-" constraints
that represent the relationship between the continuous and binary indicator
variables with second-order conic constraints. Our numerical results
demonstrate the effectiveness of the proposed approaches
Flexible Differentiable Optimization via Model Transformations
We introduce DiffOpt.jl, a Julia library to differentiate through the
solution of optimization problems with respect to arbitrary parameters present
in the objective and/or constraints. The library builds upon MathOptInterface,
thus leveraging the rich ecosystem of solvers and composing well with modeling
languages like JuMP. DiffOpt offers both forward and reverse differentiation
modes, enabling multiple use cases from hyperparameter optimization to
backpropagation and sensitivity analysis, bridging constrained optimization
with end-to-end differentiable programming. DiffOpt is built on two known rules
for differentiating quadratic programming and conic programming standard forms.
However, thanks ability to differentiate through model transformation, the user
is not limited to these forms and can differentiate with respect to the
parameters of any model that can be reformulated into these standard forms.
This notably includes programs mixing affine conic constraints and convex
quadratic constraints or objective function
- …