An Empirical Analysis of Vulnerabilities in Python Packages for Web Applications

This paper examines software vulnerabilities in common Python packages used particularly for web development. The empirical dataset is based on the PyPI package repository and the so-called Safety DB used to track vulnerabilities in selected packages within the repository. The methodological approach builds on a release-based time series analysis of the conditional probabilities for the releases of the packages to be vulnerable. According to the results, many of the Python vulnerabilities observed seem to be only modestly severe; input validation and cross-site scripting have been the most typical vulnerabilities. In terms of the time series analysis based on the release histories, only the recent past is observed to be relevant for statistical predictions; the classical Markov property holds.Comment: Forthcoming in: Proceedings of the 9th International Workshop on Empirical Software Engineering in Practice (IWESEP 2018), Nara, IEE

Considerations about quality in model-driven engineering

The final publication is available at Springer via http://dx.doi.org/10.1007/s11219-016-9350-6The virtue of quality is not itself a subject; it depends on a subject. In the software engineering field, quality means good software products that meet customer expectations, constraints, and requirements. Despite the numerous approaches, methods, descriptive models, and tools, that have been developed, a level of consensus has been reached by software practitioners. However, in the model-driven engineering (MDE) field, which has emerged from software engineering paradigms, quality continues to be a great challenge since the subject is not fully defined. The use of models alone is not enough to manage all of the quality issues at the modeling language level. In this work, we present the current state and some relevant considerations regarding quality in MDE, by identifying current categories in quality conception and by highlighting quality issues in real applications of the model-driven initiatives. We identified 16 categories in the definition of quality in MDE. From this identification, by applying an adaptive sampling approach, we discovered the five most influential authors for the works that propose definitions of quality. These include (in order): the OMG standards (e.g., MDA, UML, MOF, OCL, SysML), the ISO standards for software quality models (e.g., 9126 and 25,000), Krogstie, Lindland, and Moody. We also discovered families of works about quality, i.e., works that belong to the same author or topic. Seventy-three works were found with evidence of the mismatch between the academic/research field of quality evaluation of modeling languages and actual MDE practice in industry. We demonstrate that this field does not currently solve quality issues reported in industrial scenarios. The evidence of the mismatch was grouped in eight categories, four for academic/research evidence and four for industrial reports. These categories were detected based on the scope proposed in each one of the academic/research works and from the questions and issues raised by real practitioners. We then proposed a scenario to illustrate quality issues in a real information system project in which multiple modeling languages were used. For the evaluation of the quality of this MDE scenario, we chose one of the most cited and influential quality frameworks; it was detected from the information obtained in the identification of the categories about quality definition for MDE. We demonstrated that the selected framework falls short in addressing the quality issues. Finally, based on the findings, we derive eight challenges for quality evaluation in MDE projects that current quality initiatives do not address sufficiently.F.G, would like to thank COLCIENCIAS (Colombia) for funding this work through the Colciencias Grant call 512-2010. This work has been supported by the Gene-ralitat Valenciana Project IDEO (PROMETEOII/2014/039), the European Commission FP7 Project CaaS (611351), and ERDF structural funds.Giraldo-Velásquez, FD.; España Cubillo, S.; Pastor López, O.; Giraldo, WJ. (2016). Considerations about quality in model-driven engineering. Software Quality Journal. 1-66. https://doi.org/10.1007/s11219-016-9350-6S166(1985). Iso information processing—documentation symbols and conventions for data, program and system flowcharts, program network charts and system resources charts. ISO 5807:1985(E) (pp. 1–25).(2011). Iso/iec/ieee systems and software engineering – architecture description. ISO/IEC/IEEE 42010:2011(E) (Revision of ISO/IEC 42010:2007 and IEEE Std 1471-2000) (pp. 1–46).Abran, A., Moore, J.W., Bourque, P., Dupuis, R., & Tripp, L.L. (2013). Guide to the Software Engineering Body of Knowledge (SWEBOK) version 3 public review. IEEE. ISO Technical Report ISO/IEC TR 19759.Agner, L.T.W., Soares, I.W., Stadzisz, P.C., & Simão, J.M. (2013). A brazilian survey on {UML} and model-driven practices for embedded software development. Journal of Systems and Software, 86(4), 997–1005. {SI} : Software Engineering in Brazil: Retrospective and Prospective Views.Amstel, M.F.V. (2010). The right tool for the right job: assessing model transformation quality. pages 69–74. Affiliation: Eindhoven University of Technology, P.O. Box 513, 5600 MB, Eindhoven, Netherlands. Cited By (since 1996):1.Aranda, J., Damian, D., & Borici, A. (2012). Transition to model-driven engineering: what is revolutionary, what remains the same?. In Proceedings of the 15th international conference on model driven engineering languages and systems, MODELS’12 (pp. 692–708). Berlin, Heidelberg: Springer.Arendt, T., & Taentzer, G. (2013). A tool environment for quality assurance based on the eclipse modeling framework. Automated Software Engineering, 20(2), 141–184.Atkinson, C., Bunse, C., & Wüst, J. (2003). Driving component-based software development through quality modelling, volume 2693. Cited By (since 1996):3.Baker, P., Loh, S., & Weil, F. (2005). Model-driven engineering in a large industrial context—motorola case study. In Briand, L., & Williams, C. (Eds.) Model Driven Engineering Languages and Systems, volume 3713 of Lecture Notes in Computer Science (pp. 476–491). Berlin, Heidelberg: Springer.Barišić, A., Amaral, V., Goulão, M., & Barroca, B. (2011). Quality in use of domain-specific languages: a case study. In Proceedings of the 3rd ACM SIGPLAN workshop on evaluation and usability of programming languages and tools, PLATEAU ’11 (pp. 65–72). New York: ACM.Becker, J., Bergener, P., Breuker, D., & Rackers, M. (2010). Evaluating the expressiveness of domain specific modeling languages using the bunge-wand-weber ontology. In 2010 43rd Hawaii international conference on system sciences (HICSS) (pp. 1–10).Bertrand Portier, L.A. (2009). Model driven development misperceptions and challenges.Bézivin, J., & Kurtev, I. (2005). Model-based technology integration with the technical space concept. In Proceedings of the Metainformatics Symposium: Springer.Brambilla, M. (2016). How mature is of model-driven engineering as an engineering discipline @ONLINE.Brambilla, M., & Fraternali, P. (2014). Large-scale model-driven engineering of web user interaction: The webml and webratio experience. Science of Computer Programming, 89 Part B(0), 71 – 87. Special issue on Success Stories in Model Driven Engineering.Brown, A. (2009). Simple and practical model driven architecture (mda) @ONLINE.Bruel, J.-M., Combemale, B., Ober, I., & Raynal, H. (2015). Mde in practice for computational science. Procedia Computer Science, 51, 660–669.Budgen, D., Burn, A.J., Brereton, O.P., Kitchenham, B.A., & Pretorius, R. (2011). Empirical evidence about the uml: a systematic literature review. Software: Practice and Experience, 41(4), 363–392.Burden, H., Heldal, R., & Whittle, J. (2014). Comparing and contrasting model-driven engineering at three large companies. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM ’14 (pp. 14:1–14:10). New York: ACM.Cabot, J. Has mda been abandoned (by the omg)?Cabot, J. (2009). Modeling will be commonplace in three years time @ONLINE.Cachero, C., Poels, G., Calero, C., & Marhuenda, Y. (2007). Towards a Quality-Aware Engineering Process for the Development of Web Applications. Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 07/462, Ghent University, Faculty of Economics and Business Administration.Challenger, M., Kardas, G., & Tekinerdogan, B. (2015). A systematic approach to evaluating domain-specific modeling language environments for multi-agent systems. Software Quality Journal, 1–41.Chaudron, M.V., Heijstek, W., & Nugroho, A. (2012). How effective is uml modeling? Software & Systems Modeling, 11(4), 571–580. J2: Softw Syst Model.Chenouard, R., Granvilliers, L., & Soto, R. (2008). Model-driven constraint programming. pages 236–246. Affiliation: CNRS, LINA, Universit de Nantes, France; Affiliation: Pontificia Universidad Catlica de, Valparaiso, Chile. Cited By (since 1996):8.Clark, T., & Muller, P.-A. (2012). Exploiting model driven technology: a tale of two startups. Software and Systems Modeling, 11(4), 481–493.Corneliussen, L. (2008). What do you think of model-driven software development?Costal, D., Gómez, C., & Guizzardi, G. (2011). Formal semantics and ontological analysis for understanding subsetting, specialization and redefinition of associations in uml. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6998 LNCS:189–203. cited By (since 1996)3.Cruz-Lemus, J.A., Maes, A., Género, M., Poels, G., & Piattini, M. (2010). The impact of structural complexity on the understandability of uml statechart diagrams. Information Sciences, 180(11), 2209–2220. Cited By (since 1996):14.Cuadrado, J.S., Izquierdo, J.L.C., & Molina, J.G. (2014). Applying model-driven engineering in small software enterprises. Science of Computer Programming, 89 Part B(0), 176 – 198. Special issue on Success Stories in Model Driven Engineering.Da Silva, A.R. (2015). Model-driven engineering: a survey supported by the unified conceptual model. Computer Languages Systems and Structures, 43, 139–155.Da Silva Teixeira, D.G.M., Quirino, G.K., Gailly, F., De Almeida Falbo, R., Guizzardi, G., & Perini Barcellos, M. (2016). PoN-S: a Systematic Approach for Applying the Physics of Notation (PoN), (pp. 432–447). Cham: Springer International Publishing.Davies, I., Green, P., Rosemann, M., Indulska, M., & Gallo, S. (2006). How do practitioners use conceptual modeling in practice? Data and Knowledge Engineering, 58(3), 358 – 380. Including the special issue : {ER} 2004ER 2004.Davies, J., Milward, D., Wang, C.-W., & Welch, J. (2015). Formal model-driven engineering of critical information systems. Science of Computer Programming, 103(0), 88 – 113. Selected papers from the First International Workshop on Formal Techniques for Safety-Critical Systems (FTSCS 2012).De Oca, I.M.-M., Snoeck, M., Reijers, H.A., & Rodríguez-Morffi, A. (2015). A systematic literature review of studies on business process modeling quality. Information and Software Technology, 58, 187–205.DenHaan, J. (2009). 8 reasons why model driven development is dangerous @ONLINE.DenHaan, J. (2010). Model driven engineering vs the commando pattern @ONLINE.DenHaan, J. (2011a). Why aren’t we all doing model driven development yet @ONLINE.DenHaan, J. (2011b). Why there is no future model driven development @ONLINE.Di Ruscio, D., Iovino, L., & Pierantonio, A. (2013). Managing the coupled evolution of metamodels and textual concrete syntax specifications. cited By (since 1996)0.Dijkman, R.M., Dumas, M., & Ouyang, C. (2008). Semantics and analysis of business process models in {BPMN}. Information and Software Technology, 50(12), 1281–1294.Domínguez-Mayo, F.J., Escalona, M.J., Mejías, M., Ramos, I., & Fernández, L. (2011). A framework for the quality evaluation of mdwe methodologies and information technology infrastructures. International Journal of Human Capital and Information Technology Professionals, 2(4), 11–22.Domínguez-Mayo, F.J., Escalona, M.J., Mejías, M., & Torres, A.H. (2010). A quality model in a quality evaluation framework for mdwe methodologies. pages 495–506. Affiliation: Departamento de Lenguajes y Sistemas Informíticos, University of Seville, Seville, Spain., Cited By (since 1996):1.Dubray, J.-J. (2011). Why did mde miss the boat?.Escalona, M.J., Gutiérrez, J.J., Pérez-Pérez, M., Molina, A., Domínguez-Mayo, E., & Domínguez-Mayo, F.J. (2011). Measuring the Quality of Model-Driven Projects with NDT-Quality, (pp. 307–317). New York: Springer.Espinilla, M., Domínguez-Mayo, F.J., Escalona, M.J., Mejías, M., Ross, M., & Staples, G. (2011). A Method Based on AHP to Define the Quality Model of QuEF (Vol. 123, pp. 685–694). Berlin, Heidelberg: Springer.Fabra, J., Castro, V.D., Álvarez, P., & Marcos, E. (2012). Automatic execution of business process models: exploiting the benefits of model-driven engineering approaches. Journal of Systems and Software, 85(3), 607–625. Novel approaches in the design and implementation of systems/software architecture.Falkenberg, E.D., Hesse, W., Lindgreen, P., Nilsson, B.E., Oei, J.L.H., Rolland, C., Stamper, R.K., Assche, F.J.M.V., Verrijn-Stuart, A.A., & Voss, K. (1996). Frisco: a framework of information system concepts. Technical report, The IFIP WG 8. 1 Task Group FRISCO.Fettke, P., Houy, C., Vella, A.-L., & Loos, P. (2012). Towards the Reconstruction and Evaluation of Conceptual Model Quality Discourses – Methodical Framework and Application in the Context of Model Understandability, volume 113 of Lecture Notes in Business Information Processing, chapter 28, pages 406–421, Springer, Berlin, Heidelberg.Finnie, S. (2015). Modeling community: Are we missing something?Fournier, C. (2008). Is uml [email protected], R., & Rumpe, B. (2007). Model-driven development of complex software: a research roadmap. In Future of Software Engineering, 2007, FOSE ’07 (pp. 37–54).Gallego, M., Giraldo, F.D., & Hitpass, B. (2015). Adapting the pbec-otss software selection approach for bpm suites: an application case. In 2015 34th International Conference of the Chilean Computer Science Society (SCCC) (pp. 1–10).Galvão, I., & Goknil, A. (2007). Survey of traceability approaches in model-driven engineering. cited By (since 1996)22.Giraldo, F., España, S., Giraldo, W., & Pastor, O. (2015). Modelling language quality evaluation in model-driven information systems engineering: a roadmap. In 2015 IEEE 9th International Conference on Research Challenges in Information Science (RCIS) (pp. 64–69).Giraldo, F., España, S., & Pastor, O. (2014). Analysing the concept of quality in model-driven engineering literature: a systematic review. In 2014 IEEE Eighth International Conference on Research Challenges in Information Science (RCIS) (pp. 1–12).Giraldo, F.D., España, S., & Pastor, O. (2016). Evidences of the mismatch between industry and academy on modelling language quality evaluation. arXiv: 1606.02025 .González, C., & Cabot, J. (2014). Formal verification of static software models in mde: a systematic review. Information and Software Technology, 56(8), 821–838. cited By (since 1996)0.González, C.A., Büttner, F., Clarisó, R., & Cabot, J. (2012). Emftocsp: a tool for the lightweight verification of emf models. pages 44–50. Affiliation: cole des Mines de Nantes, INRIA, LINA, Nantes, France; Affiliation: Universitat Oberta de Catalunya, Barcelona, Spain. Cited By (since 1996):1.Gorschek, T., Tempero, E., & Angelis, L. (2014). On the use of software design models in software development practice: an empirical investigation. Journal of Systems and Software, 95(0), 176– 193.Goulão, M., Amaral, V., & Mernik, M. (2016). Quality in model-driven engineering: a tertiary study. Software Quality Journal, 1–33.Grobshtein, Y., & Dori, D. (2011). Generating sysml views from an opm model: design and evaluation. Systems Engineering, 14(3), 327–340.Haan, J.d. (2008). 8 reasons why model-driven approaches (will) fail.Harel, D., & Rumpe, B. (2000). Modeling languages: Syntax, semantics and all that stuff, part i: The basic stuff, Israel. Technical report Jerusalem Israel.Harel, D., & Rumpe, B. (2004). Meaningful modeling: what’s the semantics of semantics? Computer, 37(10), 64–72.Hebig, R., & Bendraou, R. (2014). On the need to study the impact of model driven engineering on software processes. In Proceedings of the 2014 International Conference on Software and System Process, ICSSP 2014 (pp. 164–168). New York: ACM.Heidari, F., & Loucopoulos, P. (2014). Quality evaluation framework (qef): modeling and evaluating quality of business processes. International Journal of Accounting Information Systems, 15(3), 193–223. Business Process Modeling.Heymans, P., Schobbens, P.Y., Trigaux, J.C., Bontemps, Y., Matulevicius, R., & Classen, A. (2008). Evaluating formal properties of feature diagram languages. Software, IET, 2(3), 281–302. ID 2.Hindawi, M., Morel, L., Aubry, R., & Sourrouille, J.-L. (2009). Description and Implementation of a UML Style Guide (Vol. 5421, pp. 291–302). Berlin: Springer.Hoang, D. (2012). Current limitations of mdd and its implications @ONLINE.Hodges, W. (2013). Model theory Zalta, E.N. (Ed.) The Stanford Encyclopedia of Philosophy. Fall 2013 edition.Hutchinson, J., Rouncefield, M., & Whittle, J. (2011a). Model-driven engineering practices in industry. In Proceedings of the 33rd International Conference on Software Engineering, ICSE’11 (pp. 633–642). New York: ACM.Hutchinson, J., Whittle, J., & Rouncefield, M. (2014). Model-driven engineering practices in industry: social, organizational and managerial factors that lead to success or failure. Science of Computer Programming, 89 Part B(0), 144–161. Special issue on Success Stories in Model Driven Engineering.Hutchinson, J., Whittle, J., Rouncefield, M., & Kristoffersen, S. (2011b). Empirical assessment of mde in industry. In Proceedings of the 33rd International Conference on Software Engineering, ICSE’11 (pp. 471–480). New York: ACM.Igarza, I.M.H., Boada, D.H.G., & Valdés, A.P. (2012). Una introducción al desarrollo de software dirigido por modelos. Serie Científica, 5(3).ISO/IEC (2001). ISO/IEC 9126. Software engineering—Product quality. ISO/IEC.Izurieta, C., Rojas, G., & Griffith, I. (2015). Preemptive management of model driven technical debt for improving software quality. In Proceedings of the 11th International ACM SIGSOFT Conference on Quality of Software Architectures, QoSA’15 (pp. 31–36). New York: ACM.Jalali, S., & Wohlin, C. (2012). Systematic literature studies: Database searches vs. backward snowballing. In Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM’12 (pp. 29–38). New York: ACM.Kahraman, G., & Bilgen, S. (2013). A framework for qualitative assessment of domain-specific languages. Software & Systems Modeling, 1–22.Kessentini, M., Langer, P., & Wimmer, M. (2013). Searching models, modeling search: On the synergies of sbse and mde (pp. 51–54).Kitchenham, B., & Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. Technical Report EBSE 2007-001, Keele University and Durham University Joint Report.Kitchenham, B., Pfleeger, S., Pickard, L., Jones, P., Hoaglin, D., El Emam, K., & Rosenberg, J. (2002). Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28(8), 721–734.Klinke, M. (2008). Do you use mda/mdd/mdsd, any kind of model-driven approach? Will it be the future?Köhnlein, J. (2013). Eclipse diagram editors from a user’s perspective.Kolovos, D.S., Paige, R.F., & Polack, F.A. (2008). The grand challenge of scalability for model driven engineering. In Models in Software Engineering (pp. 48–53): Springer.Kolovos, D.S., Rose, L.M., Matragkas, N., Paige, R.F., Guerra, E., Cuadrado, J.S., De Lara, J., Ráth, I., Varró, D., Tisi, M., & Cabot, J. (2013). A research roadmap towards achieving scalability in model driven engineering. In Proceedings of the Workshop on Scalability in Model Driven Engineering, BigMDE’13 (pp. 2:1–2:10). New York: ACM.Krill, P. (2016). Uml to be ejected from microsoft visual studio (infoworld).Krogstie, J. (2012a). Model-based development and evolution of information systems: a quality approach, Springer Publishing Company, Incorporated.Krogstie, J. (2012b). Quality of modelling languages, (pp. 249–280). London: Springer.Krogstie, J. (2012c). Quality of models, (pp. 205–247). London: Springer.Krogstie, J. (2012d). Specialisations of SEQUAL, (pp. 281–326). London: Springer.Krogstie, J., Lindland, O.I., & Sindre, G. (1995). Defining quality aspects for conceptual models. In Proceedings of the IFIP International Working Conference on Information System Concepts: Towards a Consolidation of Views (pp. 216–231). London: Chapman & Hall, Ltd.Kruchten, P. (2000). The rational unified process: an introduction, 2nd edn. Boston: Addison-Wesley Longman Publishing Co., Inc.Kruchten, P., Nord, R., & Ozkaya, I. (2012). Technical debt: from metaphor to theory and practice. Software, IEEE, 29(6), 18–21.Kulkarni, V., Reddy, S., & Rajbhoj, A. (2010). Scaling up model driven engineering – experience and lessons learnt. In Petriu, D., Rouquette, N., & Haugen, y. (Eds.) Model Driven Engineering Languages and Systems, volume 6395 of Lecture Notes in Computer Science (pp. 331–345). Berlin, Heidelberg: Springer.Laguna, M.A., & Marqués, J.M. (2010). Uml support for designing software product lines: the package merge mechanism, 16(17), 2313–2332.Lange, C. (2007a). Model size matters. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 4364 LNCS:211–216. cited By (since 1996)1.Lange, C., & Chaudron, M. (2005). Managing Model Quality in UML-Based Software Development. In 13th IEEE International Workshop on Technology and Engineering Practice, 2005 (pp. 7–16).Lange, C., Chaudron, M.R.V., Muskens, J., Somers, L.J., & Dortmans, H.M. (2003). An empirical investigation in quantifying inconsistency and incompleteness of uml designs. In Incompleteness of UML Designs, Proceedings Workshop on Consistency Problems in UML-based Software Development, 6th International Conference on Unified Modeling Language, UML, 2003.Lange, C., DuBois, B., Chaudron, M., & Demeyer, S. (2006). An experimental investigation of uml modeling conventions. In Nierstrasz, O., Whittle, J., Harel, D., & Reggio, G. (Eds.) Model Driven Engineering Languages and Systems, volume 4199 of Lecture Notes in Computer Science (pp. 27–41). Berlin, Heidelberg: Springer.Lange, C.F.J., & Chaudron, M.R.V. (2006). Effe

A systematic review of quality attributes and measures for software product lines

[EN] It is widely accepted that software measures provide an appropriate mechanism for understanding, monitoring, controlling, and predicting the quality of software development projects. In software product lines (SPL), quality is even more important than in a single software product since, owing to systematic reuse, a fault or an inadequate design decision could be propagated to several products in the family. Over the last few years, a great number of quality attributes and measures for assessing the quality of SPL have been reported in literature. However, no studies summarizing the current knowledge about them exist. This paper presents a systematic literature review with the objective of identifying and interpreting all the available studies from 1996 to 2010 that present quality attributes and/or measures for SPL. These attributes and measures have been classified using a set of criteria that includes the life cycle phase in which the measures are applied; the corresponding quality characteristics; their support for specific SPL characteristics (e. g., variability, compositionality); the procedure used to validate the measures, etc. We found 165 measures related to 97 different quality attributes. The results of the review indicated that 92% of the measures evaluate attributes that are related to maintainability. In addition, 67% of the measures are used during the design phase of Domain Engineering, and 56% are applied to evaluate the product line architecture. However, only 25% of them have been empirically validated. In conclusion, the results provide a global vision of the state of the research within this area in order to help researchers in detecting weaknesses, directing research efforts, and identifying new research lines. In particular, there is a need for new measures with which to evaluate both the quality of the artifacts produced during the entire SPL life cycle and other quality characteristics. There is also a need for more validation (both theoretical and empirical) of existing measures. In addition, our results may be useful as a reference guide for practitioners to assist them in the selection or the adaptation of existing measures for evaluating their software product lines. © 2011 Springer Science+Business Media, LLC.This research has been funded by the Spanish Ministry of Science and Innovation under the MULTIPLE (Multimodeling Approach For Quality-Aware Software Product Lines) project with ref. TIN2009-13838.Montagud Gregori, S.; Abrahao Gonzales, SM.; Insfrán Pelozo, CE. (2012). A systematic review of quality attributes and measures for software product lines. Software Quality Journal. 20(3-4):425-486. https://doi.org/10.1007/s11219-011-9146-7S425486203-4Abdelmoez, W., Nassar, D. M., Shereschevsky, M., Gradetsky, N., Gunnalan, R., Ammar, H. H., et al. (2004). Error propagation in software architectures. In 10th international symposium on software metrics (METRICS), Chicago, Illinois, USA.Ajila, S. A., & Dumitrescu, R. T. (2007). Experimental use of code delta, code churn, and rate of change to understand software product line evolution. Journal of Systems and Software, 80, 74–91.Aldekoa, G., Trujillo, S., Sagardui, G., & Díaz, O. (2006). Experience measuring maintainability in software product lines. In XV Jornadas de Ingeniería del Software y Bases de Datos (JISBD). Barcelona.Aldekoa, G., Trujillo, S., Sagardui, G., & Díaz, O. (2008). Quantifying maintanibility in feature oriented product lines, Athens, Greece, pp. 243–247.Alves de Oliveira Junior, E., Gimenes, I. M. S., & Maldonado, J. C. (2008). A metric suite to support software product line architecture evaluation. In XXXIV Conferencia Latinamericana de Informática (CLEI), Santa Fé, Argentina, pp. 489–498.Alves, V., Niu, N., Alves, C., & Valença, G. (2010). Requirements engineering for software product lines: A systematic literature review. Information & Software Technology, 52(8), 806–820.Bosch, J. (2000). Design and use of software architectures: Adopting and evolving a product line approach. USA: ACM Press/Addison-Wesley Publishing Co.Briand, L. C., Differing, C. M., & Rombach, D. (1996a). Practical guidelines for measurement-based process improvement. Software Process-Improvement and Practice, 2, 253–280.Briand, L. C., Morasca, S., & Basili, V. R. (1996b). Property based software engineering measurement. IEEE Transactions on Software Eng., 22(1), 68–86.Calero, C., Ruiz, J., & Piattini, M. (2005). Classifying web metrics using the web quality model. Online Information Review, 29(3): 227–248.Chen, L., Ali Babar, M., & Ali, N. (2009). Variability management in software product lines: A systematic review. In 13th international software product lines conferences (SPLC), San Francisco, USA.Clements, P., & Northrop, L. (2002). Software product lines. 2003. Software product lines practices and patterns. Boston, MA: Addison-Wesley.Crnkovic, I., & Larsson, M. (2004). Classification of quality attributes for predictability in component-based systems. Journal of Econometrics, pp. 231–250.Conference Rankings of Computing Research and Education Association of Australasia (CORE). (2010). Available in http://core.edu.au/index.php/categories/conference%20rankings/1 .Davis, A., Dieste, Ó., Hickey, A., Juristo, N., & Moreno, A. M. (2006). Effectiveness of requirements elicitation techniques: Empirical results derived from a systematic review. In 14th IEEE international conference requirements engineering, pp. 179–188.de Souza Filho, E. D., de Oliveira Cavalcanti, R., Neiva, D. F. S., Oliveira, T. H. B., Barachisio Lisboa, L., de Almeida E. S., & de Lemos Meira, S. R. (2008). Evaluating domain design approaches using systematic review. In 2nd European conference on software architecture, Cyprus, pp. 50–65.Ejiogu, L. (1991). Software engineering with formal metrics. QED Publishing.Engström, E., & Runeson, P. (2011). Software product line testing—A systematic mapping study. Information & Software Technology, 53(1), 2–13.Etxeberria, L., Sagarui, G., & Belategi, L. (2008). Quality aware software product line engineering. Journal of the Brazilian Computer Society, 14(1), Campinas Mar.Ganesan, D., Knodel, J., Kolb, R., Haury, U., & Meier, G. (2007). Comparing costs and benefits of different test strategies for a software product line: A study from Testo AG. In 11th international software product line conference, Kyoto, Japan, pp. 74–83, September 2007.Gómez, O., Oktaba, H., Piattini, M., & García, F. (2006). A systematic review measurement in software engineering: State-of-the-art in measures. In First international conference on software and data technologies (ICSOFT), Setúbal, Portugal, pp. 11–14.IEEE standard for a software quality metrics methodology, IEEE Std 1061-1998, 1998.Inoki, M., & Fukazawa, Y. (2007). Software product line evolution method based on Kaizen approach. In 22nd annual ACM symposium on applied computing, Korea.Insfran, E., & Fernandez, A. (2008). A systematic review of usability evaluation in Web development. 2nd international workshop on web usability and accessibility (IWWUA’08), New Zealand, LNCS 5176, Springer, pp. 81–91.ISO/IEC 25010. (2008). Systems and software engineering. Systems and software Quality Requirements and Evaluation (SQuaRE). System and software quality models.ISO/IEC 9126. (2000). Software engineering. Product Quality.Johansson, E., & Höst, R. (2002). Tracking degradation in software product lines through measurement of design rule violations. In 14th International conference on software engineering and knowledge engineering, Ischia, Italy, pp. 249–254.Journal Citation Reports of Thomson Reuters. (2010). Available in http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/ .Khurum, M., & Gorschek, T. (2009). A systematic review of domain analysis solutions for product lines. The Journal of Systems and Software.Kim, T., Ko, I. Y., Kang, S. W., & Lee, D. H. (2008). Extending ATAM to assess product line architecture. In 8th IEEE international conference on computer and information technology, pp. 790–797.Kitchenham, B. (2007). Guidelines for performing systematic literature reviews in software engineering. Version 2.3, EBSE Technical Report, Keele University, UK.Kitchenham, B., Pfleeger, S., & Fenton, N. (1995). Towards a framework for software measurement validation. IEEE Transactions on Software Engineering, 21(12).Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.Mendes, E. (2005). A systematic review of Web engineering research. International symposium on empirical software engineering. Noosa Heads, Australia.Meyer, M. H., & Dalal, D. (2002). Managing platform architectures and manufacturing processes for non assembled products. Journal of Product Innovation Management, 19(4), 277–293.Montagud, S., & Abrahão, S. (2009). Gathering Current knowledge about quality evaluation in software product lines. In 13th international software product lines conferences (SPLC), San Francisco, USA.Montagud, S., & Abrahão, S. (2009). A SQuaRE-bassed quality evaluation method for software product lines. Master’s thesis, December 2009 (in Spanish).Needham, D., & Jones, S. (2006). A software fault tree metric. In 22nd international conference on software maintenance (ICSM), Philadelphia, Pennsylvania, USA.Niemelä, E., & Immonen, A. (2007). Capturing quality requirements of product family architecture. Information and Software Technology, 49(11–12), 1107–1120.Odia, O. E. (2007). Testing in software product lines. Master Thesis Software Engineering of School of Engineering, Bleking Institute of Technology. Thesis no. MSE-2007:16, Sweden.Olumofin, F. G., & Mišić, V. B. (2007). A holistic architecture assessment method for software product lines. Information and Software Technology, 49, 309–323.Pérez Lamancha, B., Polo Usaola, M., & Piattini Velthius, M. (2009). Software product line testing—a systematic review. ICSOFT, (1), 23–30.Poels, G., & Dedene, G. (2000). Distance-based software measurement: necessary and sufficient properties for software measures. Information and Software Technology, 42(I), 35–46.Prehofer, C., van Gurp, J., & Bosch, J. (2008). Compositionality in software platforms. In Emerging methods, technologies and process management in software engineering. Wiley.Rahman, A. (2004). Metrics for the structural assessment of product line architecture. Master Thesis on Software Engineering, Thesis no. MSE-2004:24. School of Engineering, Blekinge Institute of Technology, Sweden.Sethi, K., Cai, Y., Wong, S., Garcia, A., & Sant’Anna, C. (2009). From retrospect to prospect: Assessing modularity and stability from software architecture. Joint working IEEE/IFIP conference on software architecture, 2009 & European conference on software architecture. WICSA/ECSA.Shaik, I., Abdelmoez, W,. Gunnalan, R., Shereshevsky, M., Zeid, A., Ammar, H. H., et al. (2005). Change propagation for assessing design quality of software architectures. 5th working IEEE/IFIP conference on software architecture (WICSA’05).Siegmund, N., Rosenmüller, M., Kuhlemann, M., Kästner, C., & Saake, G. (2008). Measuring non-functional properties in software product lines for product derivation. In 15th Asia-Pacific software engineering conference, Beijing, China.Sun Her, J., Hyeok Kim, J., Hun Oh, S., Yul Rhew, S., & Dong Kim, S. (2007). A framework for evaluating reusability of core asset in product line engineering. Information and Software Technology, 49, 740–760.Svahnberg, M., & Bosch, J. (2000). Evolution in software product lines. In 3rd international workshop on software architectures for products families (IWSAPF-3). Las Palmas de Gran Canaria.Van der Hoek, A., Dincel, E., & Medidović, N. (2003). Using services utilization metrics to assess the structure of product line architectures. In 9th international software metrics symposium (METRICS), Sydney, Australia.Van der Linden, F., Schmid, K., & Rommes, E. (2007). Software product lines in action. Springer.Whitmire, S. (1997). Object oriented design measurement. John Wiley & Sons.Wnuk, K., Regnell, B., & Karlsson, L. (2009). What happened to our features? Visualization and understanding of scope change dynamics in a large-scale industrial setting. In 17th IEEE international requirements engineering conference.Yoshimura, K., Ganesan, D., & Muthig, D. (2006). Assessing merge potential of existing engine control systems into a product line. In International workshop on software engineering for automative systems, Shangai, China, pp. 61–67.Zhang, T., Deng, L., Wu, J., Zhou, Q., & Ma, C. (2008). Some metrics for accessing quality of product line architecture. In International conference on computer science and software engineering (CSSE), Wuhan, China, pp. 500–503

The Scalability-Efficiency/Maintainability-Portability Trade-off in Simulation Software Engineering: Examples and a Preliminary Systematic Literature Review

Large-scale simulations play a central role in science and the industry. Several challenges occur when building simulation software, because simulations require complex software developed in a dynamic construction process. That is why simulation software engineering (SSE) is emerging lately as a research focus. The dichotomous trade-off between scalability and efficiency (SE) on the one hand and maintainability and portability (MP) on the other hand is one of the core challenges. We report on the SE/MP trade-off in the context of an ongoing systematic literature review (SLR). After characterizing the issue of the SE/MP trade-off using two examples from our own research, we (1) review the 33 identified articles that assess the trade-off, (2) summarize the proposed solutions for the trade-off, and (3) discuss the findings for SSE and future work. Overall, we see evidence for the SE/MP trade-off and first solution approaches. However, a strong empirical foundation has yet to be established; general quantitative metrics and methods supporting software developers in addressing the trade-off have to be developed. We foresee considerable future work in SSE across scientific communities.Comment: 9 pages, 2 figures. Accepted for presentation at the Fourth International Workshop on Software Engineering for High Performance Computing in Computational Science and Engineering (SEHPCCSE 2016

Sensemaking Practices in the Everyday Work of AI/ML Software Engineering

This paper considers sensemaking as it relates to everyday software engineering (SE) work practices and draws on a multi-year ethnographic study of SE projects at a large, global technology company building digital services infused with artificial intelligence (AI) and machine learning (ML) capabilities. Our findings highlight the breadth of sensemaking practices in AI/ML projects, noting developers' efforts to make sense of AI/ML environments (e.g., algorithms/methods and libraries), of AI/ML model ecosystems (e.g., pre-trained models and "upstream"models), and of business-AI relations (e.g., how the AI/ML service relates to the domain context and business problem at hand). This paper builds on recent scholarship drawing attention to the integral role of sensemaking in everyday SE practices by empirically investigating how and in what ways AI/ML projects present software teams with emergent sensemaking requirements and opportunities

Structured Review of the Evidence for Effects of Code Duplication on Software Quality

This report presents the detailed steps and results of a structured review of code clone literature. The aim of the review is to investigate the evidence for the claim that code duplication has a negative effect on code changeability. This report contains only the details of the review for which there is not enough place to include them in the companion paper published at a conference (Hordijk, Ponisio et al. 2009 - Harmfulness of Code Duplication - A Structured Review of the Evidence)

Empirical Investigation on Agile Methods Usage: Issues Identified from Early Adopters in Malaysia

Agile Methods are a set of software practices that can help to produce products faster and at the same time deliver what customers want. Despite the benefits that Agile methods can deliver, however, we found few studies from the Southeast Asia region, particularly Malaysia. As a result, less empirical evidence can be obtained in the country making its implementation harder. To use a new method, experience from other practitioners is critical, which describes what is important, what is possible and what is not possible concerning Agile. We conducted a qualitative study to understand the issues faced by early adopters in Malaysia where Agile methods are still relatively new. The initial study involves 13 participants including project managers, CEOs, founders and software developers from seven organisations. Our study has shown that social and human aspects are important when using Agile methods. While technical aspects have always been considered to exist in software development, we found these factors to be less important when using Agile methods. The results obtained can serve as guidelines to practitioners in the country and the neighbouring regions

Technical Debt Prioritization: State of the Art. A Systematic Literature Review

Background. Software companies need to manage and refactor Technical Debt issues. Therefore, it is necessary to understand if and when refactoring Technical Debt should be prioritized with respect to developing features or fixing bugs. Objective. The goal of this study is to investigate the existing body of knowledge in software engineering to understand what Technical Debt prioritization approaches have been proposed in research and industry. Method. We conducted a Systematic Literature Review among 384 unique papers published until 2018, following a consolidated methodology applied in Software Engineering. We included 38 primary studies. Results. Different approaches have been proposed for Technical Debt prioritization, all having different goals and optimizing on different criteria. The proposed measures capture only a small part of the plethora of factors used to prioritize Technical Debt qualitatively in practice. We report an impact map of such factors. However, there is a lack of empirical and validated set of tools. Conclusion. We observed that technical Debt prioritization research is preliminary and there is no consensus on what are the important factors and how to measure them. Consequently, we cannot consider current research conclusive and in this paper, we outline different directions for necessary future investigations