Diversity Maximization Speedup for Localizing Faults in Single-Fault and Multi-Fault Programs

Advance online 06/09/2014</p

A critical evaluation of spectrum-based fault localization techniques on a large-scale software system

In the past, spectrum-based fault localization (SBFL) techniques have been developed to pinpoint a fault location in a program given a set of failing and successful test executions. Most of the algorithms use similarity coefficients and have only been evaluated on established but small benchmark programs from the Software-artifact Infrastructure Repository (SIR). In this paper, we evaluate the feasibility of applying 33 state-of-the-art SBFL techniques to a large real-world project, namely ASPECTJ. From an initial set of 350 faulty version from the iBugs repository of ASPECTJ we manually classified 88 bugs where SBFL techniques are suitable. Notably, only 11 bugs of these bugs can be found after examining the 1000 most suspicious lines and on average 250 source code files need to be inspected per bug. Based on these results, the study showcases the limitations of current SBFL techniques on a larger program

Learning to Combine Multiple Ranking Metrics for Fault Localization

International audienceFault localization is an inevitable step in software debugging. Spectrum-based fault localization consists in computing a ranking metric on execution traces to identify faulty source code. Existing empirical studies on fault localization show that there is no optimal ranking metric for all faults in practice. In this paper, we propose Multric, a learning-based approach to combining multiple ranking metrics for effective fault localization. In Multric, a suspiciousness score of a program entity is a combination of existing ranking metrics. Multric consists two major phases: learning and ranking. Based on training faults, Multric builds a ranking model by learning from pairs of faulty and non-faulty source code elements. When a new fault appears, Multric computes the final ranking with the learned model. Experiments are conducted on 5386 seeded faults in ten open-source Java programs. We empirically compare Multric against four widely-studied metrics and three recently-proposed one. Our experimental results show that Multric localizes faults more effectively than state-of-art metrics, such as Tarantula, Ochiai, and Ample

Mutation-Based Graph Inference for Fault Localization

update for oadoi on Nov 02 2018International audienceWe present a new fault localization algorithm, called Vautrin, built on an approximation of causality based on call graphs. The approximation of causality is done using software mutants. The key idea is that if a mutant is killed by a test, certain call graph edges within a path between the mutation point and the failing test are likely causal. We evaluate our approach on the fault localization benchmark by Steimann et al. totaling 5,836 faults. The causal graphs are extracted from 88,732 nodes connected by 119,531 edges. Vautrin improves the fault localization effectiveness for all subjects of the benchmark. Considering the wasted effort at the method level, a classical fault localization evaluation metric, the improvement ranges from 3% to 55%, with an average improvement of 14%

Test case prioritization technique based on string distance metrics

Numerous test case prioritization (TCP) approaches have been introduced to enhance the test viability in software testing activity with the goal to maximize early average percentage fault detection (APFD). There are different approaches and the process for each approach varies. Furthermore, these approaches are not well documented within the single TCP approach. Based on current studies, having an approach that has high coverage effectiveness (CE) and APFD rate, remains a challenge in TCP. The string-based approach is known to have a single string distance based metric to differentiate test cases that can improve the CE results. However, to differentiate precisely the test cases, the string distances require enhancement. Therefore, a TCP technique based on string distance metric was developed to improve CE and APFD rate. In this research, to differentiate precisely the test cases and counter the string distances problem, an enhanced string distances based metric with a string weight based metric was introduced. Then, the metric was executed under designed process for string-based approach for complete evaluation. Experimental results showed that the enhanced string metric had the highest APFD with 98.56% and highest CE with 69.82% in Siemen dataset, cstcas. Besides, the technique yielded the highest APFD with 76.38% in Robotic Wheelchair System (RWS) case study. As a conclusion, the enhanced TCP technique with weight based metric has prioritised the test case based on their occurrences which helped to differentiate precisely the test cases, and improved the overall scores of APFD and CE

Architectures for dependable modern microprocessors

Η εξέλιξη των ολοκληρωμένων κυκλωμάτων σε συνδυασμό με τους αυστηρούς χρονικούς περιορισμούς καθιστούν την επαλήθευση της ορθής λειτουργίας των επεξεργαστών μία εξαιρετικά απαιτητική διαδικασία. Με κριτήριο το στάδιο του κύκλου ζωής ενός επεξεργαστή, από την στιγμή κατασκευής των πρωτοτύπων και έπειτα, οι τεχνικές ελέγχου ορθής λειτουργίας διακρίνονται στις ακόλουθες κατηγορίες: (1) Silicon Debug: Τα πρωτότυπα ολοκληρωμένα κυκλώματα ελέγχονται εξονυχιστικά, (2) Manufacturing Testing: ο τελικό ποιοτικός έλεγχος και (3) In-field verification: Περιλαμβάνει τεχνικές, οι οποίες διασφαλίζουν την λειτουργία του επεξεργαστή σύμφωνα με τις προδιαγραφές του. Η διδακτορική διατριβή προτείνει τα ακόλουθα: (1) Silicon Debug: Η εργασία αποσκοπεί στην επιτάχυνση της διαδικασίας ανίχνευσης σφαλμάτων και στον αυτόματο εντοπισμό τυχαίων προγραμμάτων που δεν περιέχουν νέα -χρήσιμη- πληροφορία σχετικά με την αίτια ενός σφάλματος. Η κεντρική ιδέα αυτής της μεθόδου έγκειται στην αξιοποίηση της έμφυτης ποικιλομορφίας των αρχιτεκτονικών συνόλου εντολών και στην δυνατότητα από-διαμόρφωσης τμημάτων του κυκλώματος, (2) Manufacturing Testing: προτείνεται μία μέθοδο για την βελτιστοποίηση του έλεγχου ορθής λειτουργίας των πολυνηματικών και πολυπύρηνων επεξεργαστών μέσω της χρήση λογισμικού αυτοδοκιμής, (3) Ιn-field verification: Αναλύθηκε σε βάθος η επίδραση που έχουν τα μόνιμα σφάλματα σε μηχανισμούς αύξησης της απόδοσης. Επιπρόσθετα, προτάθηκαν τεχνικές για την ανίχνευση και ανοχή μόνιμων σφαλμάτων υλικού σε μηχανισμούς πρόβλεψης διακλάδωσης.Technology scaling, extreme chip integration and the compelling requirement to diminish the time-to-market window, has rendered microprocessors more prone to design bugs and hardware faults. Microprocessor validation is grouped into the following categories, based on where they intervene in a microprocessor’s lifecycle: (a) Silicon debug: the first hardware prototypes are exhaustively validated, (b) Μanufacturing testing: the final quality control during massive production, and (c) In-field verification: runtime error detection techniques to guarantee correct operation. The contributions of this thesis are the following: (1) Silicon debug: We propose the employment of deconfigurable microprocessor architectures along with a technique to generate self-checking random test programs to avoid the simulation step and triage the redundant debug sessions, (2) Manufacturing testing: We propose a self-test optimization strategy for multithreaded, multicore microprocessors to speedup test program execution time and enhance the fault coverage of hard errors; and (3) In-field verification: We measure the effect of permanent faults performance components. Then, we propose a set of low-cost mechanisms for the detection, diagnosis and performance recovery in the front-end speculative structures. This thesis introduces various novel methodologies to address the validation challenges posed throughout the life-cycle of a chip

Test case prioritization approaches in regression testing: A systematic literature review

Context Software quality can be assured by going through software testing process. However, software testing phase is an expensive process as it consumes a longer time. By scheduling test cases execution order through a prioritization approach, software testing efficiency can be improved especially during regression testing. Objective It is a notable step to be taken in constructing important software testing environment so that a system's commercial value can increase. The main idea of this review is to examine and classify the current test case prioritization approaches based on the articulated research questions. Method Set of search keywords with appropriate repositories were utilized to extract most important studies that fulfill all the criteria defined and classified under journal, conference paper, symposiums and workshops categories. 69 primary studies were nominated from the review strategy. Results There were 40 journal articles, 21 conference papers, three workshop articles, and five symposium articles collected from the primary studies. As for the result, it can be said that TCP approaches are still broadly open for improvements. Each approach in TCP has specified potential values, advantages, and limitation. Additionally, we found that variations in the starting point of TCP process among the approaches provide a different timeline and benefit to project manager to choose which approaches suite with the project schedule and available resources. Conclusion Test case prioritization has already been considerably discussed in the software testing domain. However, it is commonly learned that there are quite a number of existing prioritization techniques that can still be improved especially in data used and execution process for each approach

Weighted string distance approach based on modified clustering technique for optimizing test case prioritization

Numerous test case prioritization (TCP) approaches have been introduced to enhance the test viability in software testing activity with the goal to maximize early average percentage fault detection (APFD). String based approach had shown that applying a single string distance-based metric to differentiate the test cases can improve the APFD and coverage rate (CR) results. However, to precisely differentiate the test cases in regression testing, the string approach still requires an enhancement as it lacks priority criteria. Therefore, a study on how to effectively cluster and prioritize test cases through string-based approach is conducted. To counter the string distances problem, weighted string distances is introduced. A further enhancement was made by tuning the weighted string metric with K-Means clustering and prioritization using Firefly Algorithm (FA) technique for the TCP approach to become more flexible in manipulating available information. Then, the combination of the weighted string distances along with clustering and prioritization is executed under the designed process for a new weighted string distances-based approach for complete evaluation. The experimental results show that all the weighted string distances obtained better results compared to its single string metric with average APFD values 95.73% and CR values 61.80% in cstcas Siemen dataset. As for the proposed weighted string distances approach with clustering techniques for regression testing, the combination obtained better results and flexibility than the conventional string approach. In addition, the proposed approach also passed statistical assessment by obtaining p-value higher than 0.05 in Shapiro-Wilk’s normality test and p-value lower than 0.05 in Tukey Kramer Post Hoc tests. In conclusion, the proposed weighted string distances approach improves the overall score of APFD and CE and provides flexibility in the TCP approach for regression testing environment