Reproducibility and replicability of software defect prediction studies

© 2018 Elsevier B.V. Context: Replications are an important part of scientific disciplines. Replications test the credibility of original studies and can separate true results from those that are unreliable.Objective: In this paper we investigate the replication of defect prediction studies and identify the characteristics of replicated studies. We further assess how defect prediction replications are performed and the consistency of replication findings.Method: Our analysis is based on tracking the replication of 208 defect prediction studies identified by a highly cited Systematic Literature Review (SLR) [1]. We identify how often each of these 208 studies has been replicated and determine the type of replication carried out. We identify quality, citation counts, publication venue, impact factor, and data availability from all 208 SLR defect prediction papers to see if any of these factors are associated with the frequency with which they are replicated.Results: Only 13 (6%) of the 208 studies are replicated. Replication seems related to original papers appearing in the Transactions of Software Engineering (TSE) journal. The number of citations an original paper had was also an indicator of replications. In addition, studies conducted using closed source data seems to have more replications than those based on open source data. Where a paper has been replicated, 11 (38%) out of 29 studies revealed different results to the original study.Conclusion: Very few defect prediction studies are replicated. The lack of replication means that it remains unclear how reliable defect prediction is. We provide practical steps for improving the state of replication

Software defect prediction: do different classifiers find the same defects?

Open Access: This article is distributed under the terms of the Creative Commons Attribution 4.0 International License CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.During the last 10 years, hundreds of different defect prediction models have been published. The performance of the classifiers used in these models is reported to be similar with models rarely performing above the predictive performance ceiling of about 80% recall. We investigate the individual defects that four classifiers predict and analyse the level of prediction uncertainty produced by these classifiers. We perform a sensitivity analysis to compare the performance of Random Forest, Naïve Bayes, RPart and SVM classifiers when predicting defects in NASA, open source and commercial datasets. The defect predictions that each classifier makes is captured in a confusion matrix and the prediction uncertainty of each classifier is compared. Despite similar predictive performance values for these four classifiers, each detects different sets of defects. Some classifiers are more consistent in predicting defects than others. Our results confirm that a unique subset of defects can be detected by specific classifiers. However, while some classifiers are consistent in the predictions they make, other classifiers vary in their predictions. Given our results, we conclude that classifier ensembles with decision-making strategies not based on majority voting are likely to perform best in defect prediction.Peer reviewedFinal Published versio

Analysis of stress-strain state of bucket wheel excavator revolving platform structure: Fundament of efficient reconstruction

Rotorni bageri čine okosnicu sistema mehanizacije površinskih kopova Njihova eksploatacija u ekstremno teškim uslovima prouzrokuje otkaze delova noseće konstrukcije (plastične deformacije, prsline, pukotine, lomovi). Kao konkretni primeri, u radu su najpre prikazani izgledi tipičnih oštećenja konstrukcija obrtnih platformi bagera TAKRAF SRs 1200×24/4 (400 kW) + VR. Potom su izloženi postupci proračuna njihove konstrukcije čiju osnovu čine dve etape: formiranje konačno elementnih mreža i analiza opterećenja. Sprovođenjem izložene procedure proračuna, oblikovano je konstrukciono rešenje primenjeno prilikom rekonstrukcije obrtnih platformi dva bagera pomenutog tipa tokom 2005. godine. O valjanosti pristupa problemu i uspešnosti rešenja rekonstrukcije uverljivo govori podatak da su, nakon sanacije i rekonstrukcije, bageri otkopali više od 5 000 000 m3 otkrivke, a da pri tome, u zonama rekonstrukcije, nisu konstatovani bilo kakvi defekti konstrukcija obrtnih platformi.Bucket wheel excavators (BWE) are a part and parcel of the mechanization system in open pits. Their exploitation in heavy duty conditions causes failures of supporting structural parts (plastic deformations, cracks fractures, break-ages). As a real example, the paper first presents appearance of typical damages of revolving platform structures for BWE TAKRAF SRs 1200×24/4 (400 kW) + VR. Subsequently, techniques for calculating their structure are presented, whose basis is made in two stages: set up of finite element mesh and analysis of loads. The structural solution is developed by conducting the presented calculation procedure, and is applied during reconstruction of revolving platforms (2005) on two BWEs mentioned before. Validity of problem approach and efficiency of reconstruction solution is found in the fact that after reconstruction and repair BWE have excavated more than 5 000 000 m3 of barren soil, while at reconstruction zones no defects of revolving platform structures have been detected

Analysis of stress-strain state of bucket wheel excavator revolving platform structure: Fundament of efficient reconstruction

Rotorni bageri čine okosnicu sistema mehanizacije površinskih kopova Njihova eksploatacija u ekstremno teškim uslovima prouzrokuje otkaze delova noseće konstrukcije (plastične deformacije, prsline, pukotine, lomovi). Kao konkretni primeri, u radu su najpre prikazani izgledi tipičnih oštećenja konstrukcija obrtnih platformi bagera TAKRAF SRs 1200×24/4 (400 kW) + VR. Potom su izloženi postupci proračuna njihove konstrukcije čiju osnovu čine dve etape: formiranje konačno elementnih mreža i analiza opterećenja. Sprovođenjem izložene procedure proračuna, oblikovano je konstrukciono rešenje primenjeno prilikom rekonstrukcije obrtnih platformi dva bagera pomenutog tipa tokom 2005. godine. O valjanosti pristupa problemu i uspešnosti rešenja rekonstrukcije uverljivo govori podatak da su, nakon sanacije i rekonstrukcije, bageri otkopali više od 5 000 000 m3 otkrivke, a da pri tome, u zonama rekonstrukcije, nisu konstatovani bilo kakvi defekti konstrukcija obrtnih platformi.Bucket wheel excavators (BWE) are a part and parcel of the mechanization system in open pits. Their exploitation in heavy duty conditions causes failures of supporting structural parts (plastic deformations, cracks fractures, break-ages). As a real example, the paper first presents appearance of typical damages of revolving platform structures for BWE TAKRAF SRs 1200×24/4 (400 kW) + VR. Subsequently, techniques for calculating their structure are presented, whose basis is made in two stages: set up of finite element mesh and analysis of loads. The structural solution is developed by conducting the presented calculation procedure, and is applied during reconstruction of revolving platforms (2005) on two BWEs mentioned before. Validity of problem approach and efficiency of reconstruction solution is found in the fact that after reconstruction and repair BWE have excavated more than 5 000 000 m3 of barren soil, while at reconstruction zones no defects of revolving platform structures have been detected

Urban Growth and Urbanization of Sofiа, Belgrade and Rome: the Interaction between Urban Planning and the Market

The paper explores the problems of sustainable and resilient development of the cities in Eastern and South-east Europe on the examples of Sofia, Belgrade and Rome and their urban regions. The research draws comparisons to the forms, patterns and mechanisms of their development.Editors: Atanas Kovachev, Aleksandar D Slaev, Diliana Daskalov