Using Bad Learners to find Good Configurations

Finding the optimally performing configuration of a software system for a given setting is often challenging. Recent approaches address this challenge by learning performance models based on a sample set of configurations. However, building an accurate performance model can be very expensive (and is often infeasible in practice). The central insight of this paper is that exact performance values (e.g. the response time of a software system) are not required to rank configurations and to identify the optimal one. As shown by our experiments, models that are cheap to learn but inaccurate (with respect to the difference between actual and predicted performance) can still be used rank configurations and hence find the optimal configuration. This novel \emph{rank-based approach} allows us to significantly reduce the cost (in terms of number of measurements of sample configuration) as well as the time required to build models. We evaluate our approach with 21 scenarios based on 9 software systems and demonstrate that our approach is beneficial in 16 scenarios; for the remaining 5 scenarios, an accurate model can be built by using very few samples anyway, without the need for a rank-based approach.Comment: 11 pages, 11 figure

Clinton Hall, Astor Place

The stereograph features a black and white image of a row of buildings. A horse and buggy sit on the curbside atop a brick roadway. The albumen print is mounted on an orange card with rounded edges.https://scholarsjunction.msstate.edu/fvw-artifacts/1330/thumbnail.jp

Clinton Hall, Astor Place

The stereograph features a black and white image of a row of buildings. A horse and buggy sit on the curbside atop a brick roadway. The albumen print is mounted on an orange card with rounded edges.https://scholarsjunction.msstate.edu/fvw-photographs/1045/thumbnail.jp

Carousal [sic], Prospect Park, Brooklyn, N.Y.

The stereograph features a black and white image of a crowd full of men, women, and children standing beneath a carousel. The albumen print is mounted on a gray card with rounded corners.https://scholarsjunction.msstate.edu/fvw-photographs/1044/thumbnail.jp

Interior of Heimbold\u27s Drug Store Broadway

The stereograph features a black and white image of the interior of a store. The floor is a checkered design, and a counter surrounds the parameter of the store. Shelves full of various supplies stand behind the counters, and globe chandeliers hang from the ceiling. The albumen print is mounted on an orange card with rounded edges.https://scholarsjunction.msstate.edu/fvw-photographs/1018/thumbnail.jp

4th Ave Looking Up

The stereograph features a black and white image of a brick roadway. Sidewalks and buildings line either side of the street. The albumen print is mounted on an orange card with rounded edges.https://scholarsjunction.msstate.edu/fvw-photographs/1037/thumbnail.jp

Carousal [sic], Prospect Park, Brooklyn, N.Y.

The stereograph features a black and white image of a crowd full of men, women, and children standing beneath a carousel. The albumen print is mounted on a gray card with rounded corners.https://scholarsjunction.msstate.edu/fvw-artifacts/1329/thumbnail.jp

Drinking Fountain, Central Park, New York

The stereograph features a black and white image of a drinking fountain off the side of a sidewalk. A wall of greenery grows in the background. The photograph is mounted on a yellow card with rounded edges.https://scholarsjunction.msstate.edu/fvw-artifacts/1502/thumbnail.jp

Drinking Fountain, Central Park, New York

The stereograph features a black and white image of a drinking fountain off the side of a sidewalk. A wall of greenery grows in the background. The photograph is mounted on a yellow card with rounded edges.https://scholarsjunction.msstate.edu/fvw-photographs/1188/thumbnail.jp