3,346 research outputs found
Learning Large-Scale Bayesian Networks with the sparsebn Package
Learning graphical models from data is an important problem with wide
applications, ranging from genomics to the social sciences. Nowadays datasets
often have upwards of thousands---sometimes tens or hundreds of thousands---of
variables and far fewer samples. To meet this challenge, we have developed a
new R package called sparsebn for learning the structure of large, sparse
graphical models with a focus on Bayesian networks. While there are many
existing software packages for this task, this package focuses on the unique
setting of learning large networks from high-dimensional data, possibly with
interventions. As such, the methods provided place a premium on scalability and
consistency in a high-dimensional setting. Furthermore, in the presence of
interventions, the methods implemented here achieve the goal of learning a
causal network from data. Additionally, the sparsebn package is fully
compatible with existing software packages for network analysis.Comment: To appear in the Journal of Statistical Software, 39 pages, 7 figure
Lying Your Way to Better Traffic Engineering
To optimize the flow of traffic in IP networks, operators do traffic
engineering (TE), i.e., tune routing-protocol parameters in response to traffic
demands. TE in IP networks typically involves configuring static link weights
and splitting traffic between the resulting shortest-paths via the
Equal-Cost-MultiPath (ECMP) mechanism. Unfortunately, ECMP is a notoriously
cumbersome and indirect means for optimizing traffic flow, often leading to
poor network performance. Also, obtaining accurate knowledge of traffic demands
as the input to TE is elusive, and traffic conditions can be highly variable,
further complicating TE. We leverage recently proposed schemes for increasing
ECMP's expressiveness via carefully disseminated bogus information ("lies") to
design COYOTE, a readily deployable TE scheme for robust and efficient network
utilization. COYOTE leverages new algorithmic ideas to configure (static)
traffic splitting ratios that are optimized with respect to all (even
adversarially chosen) traffic scenarios within the operator's "uncertainty
bounds". Our experimental analyses show that COYOTE significantly outperforms
today's prevalent TE schemes in a manner that is robust to traffic uncertainty
and variation. We discuss experiments with a prototype implementation of
COYOTE
A hybrid algorithm for Bayesian network structure learning with application to multi-label learning
We present a novel hybrid algorithm for Bayesian network structure learning,
called H2PC. It first reconstructs the skeleton of a Bayesian network and then
performs a Bayesian-scoring greedy hill-climbing search to orient the edges.
The algorithm is based on divide-and-conquer constraint-based subroutines to
learn the local structure around a target variable. We conduct two series of
experimental comparisons of H2PC against Max-Min Hill-Climbing (MMHC), which is
currently the most powerful state-of-the-art algorithm for Bayesian network
structure learning. First, we use eight well-known Bayesian network benchmarks
with various data sizes to assess the quality of the learned structure returned
by the algorithms. Our extensive experiments show that H2PC outperforms MMHC in
terms of goodness of fit to new data and quality of the network structure with
respect to the true dependence structure of the data. Second, we investigate
H2PC's ability to solve the multi-label learning problem. We provide
theoretical results to characterize and identify graphically the so-called
minimal label powersets that appear as irreducible factors in the joint
distribution under the faithfulness condition. The multi-label learning problem
is then decomposed into a series of multi-class classification problems, where
each multi-class variable encodes a label powerset. H2PC is shown to compare
favorably to MMHC in terms of global classification accuracy over ten
multi-label data sets covering different application domains. Overall, our
experiments support the conclusions that local structural learning with H2PC in
the form of local neighborhood induction is a theoretically well-motivated and
empirically effective learning framework that is well suited to multi-label
learning. The source code (in R) of H2PC as well as all data sets used for the
empirical tests are publicly available.Comment: arXiv admin note: text overlap with arXiv:1101.5184 by other author
- …