39,431 research outputs found
Validating Sample Average Approximation Solutions with Negatively Dependent Batches
Sample-average approximations (SAA) are a practical means of finding
approximate solutions of stochastic programming problems involving an extremely
large (or infinite) number of scenarios. SAA can also be used to find estimates
of a lower bound on the optimal objective value of the true problem which, when
coupled with an upper bound, provides confidence intervals for the true optimal
objective value and valuable information about the quality of the approximate
solutions. Specifically, the lower bound can be estimated by solving multiple
SAA problems (each obtained using a particular sampling method) and averaging
the obtained objective values. State-of-the-art methods for lower-bound
estimation generate batches of scenarios for the SAA problems independently. In
this paper, we describe sampling methods that produce negatively dependent
batches, thus reducing the variance of the sample-averaged lower bound
estimator and increasing its usefulness in defining a confidence interval for
the optimal objective value. We provide conditions under which the new sampling
methods can reduce the variance of the lower bound estimator, and present
computational results to verify that our scheme can reduce the variance
significantly, by comparison with the traditional Latin hypercube approach
Problem-driven scenario generation: an analytical approach for stochastic programs with tail risk measure
Scenario generation is the construction of a discrete random vector to
represent parameters of uncertain values in a stochastic program. Most
approaches to scenario generation are distribution-driven, that is, they
attempt to construct a random vector which captures well in a probabilistic
sense the uncertainty. On the other hand, a problem-driven approach may be able
to exploit the structure of a problem to provide a more concise representation
of the uncertainty.
In this paper we propose an analytic approach to problem-driven scenario
generation. This approach applies to stochastic programs where a tail risk
measure, such as conditional value-at-risk, is applied to a loss function.
Since tail risk measures only depend on the upper tail of a distribution,
standard methods of scenario generation, which typically spread their scenarios
evenly across the support of the random vector, struggle to adequately
represent tail risk. Our scenario generation approach works by targeting the
construction of scenarios in areas of the distribution corresponding to the
tails of the loss distributions. We provide conditions under which our approach
is consistent with sampling, and as proof-of-concept demonstrate how our
approach could be applied to two classes of problem, namely network design and
portfolio selection. Numerical tests on the portfolio selection problem
demonstrate that our approach yields better and more stable solutions compared
to standard Monte Carlo sampling
SCOPE: Scalable Composite Optimization for Learning on Spark
Many machine learning models, such as logistic regression~(LR) and support
vector machine~(SVM), can be formulated as composite optimization problems.
Recently, many distributed stochastic optimization~(DSO) methods have been
proposed to solve the large-scale composite optimization problems, which have
shown better performance than traditional batch methods. However, most of these
DSO methods are not scalable enough. In this paper, we propose a novel DSO
method, called \underline{s}calable \underline{c}omposite
\underline{op}timization for l\underline{e}arning~({SCOPE}), and implement it
on the fault-tolerant distributed platform \mbox{Spark}. SCOPE is both
computation-efficient and communication-efficient. Theoretical analysis shows
that SCOPE is convergent with linear convergence rate when the objective
function is convex. Furthermore, empirical results on real datasets show that
SCOPE can outperform other state-of-the-art distributed learning methods on
Spark, including both batch learning methods and DSO methods
Recommended from our members
A two-stage stochastic programming with recourse model for determining robust planting plans in horticulture
A two-stage stochastic programming with recourse model for the problem of determining optimal planting plans for a vegetable crop is presented in this paper. Uncertainty caused by factors such as weather on yields is a major influence on many systems arising in horticulture. Traditional linear programming models are generally unsatisfactory in dealing with the uncertainty and produce solutions that are considered to involve an unacceptable level of risk. The first stage of the model relates to finding a planting plan which is common to all scenarios and the second stage is concerned with deriving a harvesting schedule for each scenario. Solutions are obtained for a range of risk aversion factors that not only result in greater expected profit compared to the corresponding deterministic model, but also are more robust
Harnessing Flexible and Reliable Demand Response Under Customer Uncertainties
Demand response (DR) is a cost-effective and environmentally friendly
approach for mitigating the uncertainties in renewable energy integration by
taking advantage of the flexibility of customers' demands. However, existing DR
programs suffer from either low participation due to strict commitment
requirements or not being reliable in voluntary programs. In addition, the
capacity planning for energy storage/reserves is traditionally done separately
from the demand response program design, which incurs inefficiencies. Moreover,
customers often face high uncertainties in their costs in providing demand
response, which is not well studied in literature.
This paper first models the problem of joint capacity planning and demand
response program design by a stochastic optimization problem, which
incorporates the uncertainties from renewable energy generation, customer power
demands, as well as the customers' costs in providing DR. We propose online DR
control policies based on the optimal structures of the offline solution. A
distributed algorithm is then developed for implementing the control policies
without efficiency loss. We further offer enhanced policy design by allowing
flexibilities into the commitment level. We perform real world trace based
numerical simulations. Results demonstrate that the proposed algorithms can
achieve near optimal social costs, and significant social cost savings compared
to baseline methods
Feature Selection via Binary Simultaneous Perturbation Stochastic Approximation
Feature selection (FS) has become an indispensable task in dealing with
today's highly complex pattern recognition problems with massive number of
features. In this study, we propose a new wrapper approach for FS based on
binary simultaneous perturbation stochastic approximation (BSPSA). This
pseudo-gradient descent stochastic algorithm starts with an initial feature
vector and moves toward the optimal feature vector via successive iterations.
In each iteration, the current feature vector's individual components are
perturbed simultaneously by random offsets from a qualified probability
distribution. We present computational experiments on datasets with numbers of
features ranging from a few dozens to thousands using three widely-used
classifiers as wrappers: nearest neighbor, decision tree, and linear support
vector machine. We compare our methodology against the full set of features as
well as a binary genetic algorithm and sequential FS methods using
cross-validated classification error rate and AUC as the performance criteria.
Our results indicate that features selected by BSPSA compare favorably to
alternative methods in general and BSPSA can yield superior feature sets for
datasets with tens of thousands of features by examining an extremely small
fraction of the solution space. We are not aware of any other wrapper FS
methods that are computationally feasible with good convergence properties for
such large datasets.Comment: This is the Istanbul Sehir University Technical Report
#SHR-ISE-2016.01. A short version of this report has been accepted for
publication at Pattern Recognition Letter
- …