Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Feature Selection in Software Product Lines

Software design is a process of trading off competing objectives. If the user objective space is rich, then we should use optimizers that can fully exploit that richness. For example, this study configures software product lines (expressed as feature models) using various search-based software engineering methods. Our main result is that as we increase the number of optimization objectives, the methods in widespread use (e.g. NSGA-II, SPEA2) perform much worse than IBEA (Indicator-Based Evolutionary Algorithm). IBEA works best since it makes most use of user preference knowledge. Hence it does better on the standard measures (hypervolume and spread) but it also generates far more products with 0 violations of domain constraints. We also present significant improvements to IBEA\u27s performance by employing three strong heuristic techniques that we call PUSH, PULL, and seeding. The PUSH technique forces the evolutionary search to respect certain rules and dependencies defined by the feature models, while the PULL technique gives higher weight to constraint satisfaction as an optimization objective and thus achieves a higher percentage of fully-compliant configurations within shorter runtimes. The seeding technique helps in guiding very large feature models to correct configurations very early in the optimization process. Our conclusion is that the methods we apply in search-based software engineering need to be carefully chosen, particularly when studying complex decision spaces with many optimization objectives. Also, we conclude that search methods must be customized to fit the problem at hand. Specifically, the evolutionary search must respect domain constraints

How to Evaluate Solutions in Pareto-based Search-Based Software Engineering? A Critical Review and Methodological Guidance

With modern requirements, there is an increasing tendency of considering multiple objectives/criteria simultaneously in many Software Engineering (SE) scenarios. Such a multi-objective optimization scenario comes with an important issue -- how to evaluate the outcome of optimization algorithms, which typically is a set of incomparable solutions (i.e., being Pareto non-dominated to each other). This issue can be challenging for the SE community, particularly for practitioners of Search-Based SE (SBSE). On one hand, multi-objective optimization could still be relatively new to SE/SBSE researchers, who may not be able to identify the right evaluation methods for their problems. On the other hand, simply following the evaluation methods for general multi-objective optimization problems may not be appropriate for specific SE problems, especially when the problem nature or decision maker's preferences are explicitly/implicitly available. This has been well echoed in the literature by various inappropriate/inadequate selection and inaccurate/misleading use of evaluation methods. In this paper, we first carry out a systematic and critical review of quality evaluation for multi-objective optimization in SBSE. We survey 717 papers published between 2009 and 2019 from 36 venues in seven repositories, and select 95 prominent studies, through which we identify five important but overlooked issues in the area. We then conduct an in-depth analysis of quality evaluation indicators/methods and general situations in SBSE, which, together with the identified issues, enables us to codify a methodological guidance for selecting and using evaluation methods in different SBSE scenarios.Comment: This paper has been accepted by IEEE Transactions on Software Engineering, available as full OA: https://ieeexplore.ieee.org/document/925218

Modelo de medición de la productividad para fábricas de software

La productividad en las fábricas de software es dado por el esfuerzo realizado para la producción del software, siendo muy importante porque permite que las organizaciones logren una mayor eficiencia y eficacia en sus actividades. Uno de los pilares de la competitividad es la productividad, la cual está relacionada al esfuerzo requerido para cumplir con las tareas asignadas, sin embargo, no existe una forma estándar de medirla. En este trabajo, se presenta un modelo basado en Análisis Envoltorio de Datos (DEA, por las siglas del inglés Data Envelopment Analysis) para evaluar la eficiencia relativa de las fábricas de software y sus proyectos, a fin de medir la productividad en la Componente de Producción de Software de la Fábrica de Software a través de las actividades que se realizan en sus diferentes unidades de trabajo. El modelo propuesto consta de dos fases, en la cual se evalúa, respectivamente, la productividad de la fábrica de software y la productividad de los proyectos que esta realiza. Pruebas numéricas sobre 6 fábricas de software con 160 proyectos implementados en el Perú muestran que el modelo propuesto permite determinar las fábricas de software y los proyectos más eficientes.Tesi