123,128 research outputs found
ASlib: A Benchmark Library for Algorithm Selection
The task of algorithm selection involves choosing an algorithm from a set of
algorithms on a per-instance basis in order to exploit the varying performance
of algorithms over a set of instances. The algorithm selection problem is
attracting increasing attention from researchers and practitioners in AI. Years
of fruitful applications in a number of domains have resulted in a large amount
of data, but the community lacks a standard format or repository for this data.
This situation makes it difficult to share and compare different approaches
effectively, as is done in other, more established fields. It also
unnecessarily hinders new researchers who want to work in this area. To address
this problem, we introduce a standardized format for representing algorithm
selection scenarios and a repository that contains a growing number of data
sets from the literature. Our format has been designed to be able to express a
wide variety of different scenarios. Demonstrating the breadth and power of our
platform, we describe a set of example experiments that build and evaluate
algorithm selection models through a common interface. The results display the
potential of algorithm selection to achieve significant performance
improvements across a broad range of problems and algorithms.Comment: Accepted to be published in Artificial Intelligence Journa
A planning approach to the automated synthesis of template-based process models
The design-time specification of flexible processes can be time-consuming and error-prone, due to the high number of tasks involved and their context-dependent nature. Such processes frequently suffer from potential interference among their constituents, since resources are usually shared by the process participants and it is difficult to foresee all the potential tasks interactions in advance. Concurrent tasks may not be independent from each other (e.g., they could operate on the same data at the same time), resulting in incorrect outcomes. To tackle these issues, we propose an approach for the automated synthesis of a library of template-based process models that achieve goals in dynamic and partially specified environments. The approach is based on a declarative problem definition and partial-order planning algorithms for template generation. The resulting templates guarantee sound concurrency in the execution of their activities and are reusable in a variety of partially specified contextual environments. As running example, a disaster response scenario is given. The approach is backed by a formal model and has been tested in experiment
Methods for Population Adjustment with Limited Access to Individual Patient Data: A Review and Simulation Study
Population-adjusted indirect comparisons estimate treatment effects when
access to individual patient data is limited and there are cross-trial
differences in effect modifiers. Popular methods include matching-adjusted
indirect comparison (MAIC) and simulated treatment comparison (STC). There is
limited formal evaluation of these methods and whether they can be used to
accurately compare treatments. Thus, we undertake a comprehensive simulation
study to compare standard unadjusted indirect comparisons, MAIC and STC across
162 scenarios. This simulation study assumes that the trials are investigating
survival outcomes and measure continuous covariates, with the log hazard ratio
as the measure of effect. MAIC yields unbiased treatment effect estimates under
no failures of assumptions. The typical usage of STC produces bias because it
targets a conditional treatment effect where the target estimand should be a
marginal treatment effect. The incompatibility of estimates in the indirect
comparison leads to bias as the measure of effect is non-collapsible. Standard
indirect comparisons are systematically biased, particularly under stronger
covariate imbalance and interaction effects. Standard errors and coverage rates
are often valid in MAIC but the robust sandwich variance estimator
underestimates variability where effective sample sizes are small. Interval
estimates for the standard indirect comparison are too narrow and STC suffers
from bias-induced undercoverage. MAIC provides the most accurate estimates and,
with lower degrees of covariate overlap, its bias reduction outweighs the loss
in effective sample size and precision under no failures of assumptions. An
important future objective is the development of an alternative formulation to
STC that targets a marginal treatment effect.Comment: 73 pages (34 are supplementary appendices and references), 8 figures,
2 tables. Full article (following Round 4 of minor revisions). arXiv admin
note: text overlap with arXiv:2008.0595
- …