Box-constraints limit the domain of decision variables and are common in
real-world optimization problems, for example, due to physical, natural or
spatial limitations. Consequently, solutions violating a box-constraint may not
be evaluable. This assumption is often ignored in the literature, e.g.,
existing benchmark suites, such as COCO/BBOB, allow the optimizer to evaluate
infeasible solutions. This paper presents an initial study on the
strict-box-constrained benchmarking suite (SBOX-COST), which is a variant of
the well-known BBOB benchmark suite that enforces box-constraints by returning
an invalid evaluation value for infeasible solutions. Specifically, we want to
understand the performance difference between BBOB and SBOX-COST as a function
of two initialization methods and six constraint-handling strategies all tested
with modular CMA-ES. We find that, contrary to what may be expected, handling
box-constraints by saturation is not always better than not handling them at
all. However, across all BBOB functions, saturation is better than not
handling, and the difference increases with the number of dimensions. Strictly
enforcing box-constraints also has a clear negative effect on the performance
of classical CMA-ES (with uniform random initialization and no constraint
handling), especially as problem dimensionality increases