An empirical comparison of commercial and open‐source web vulnerability scanners

Web vulnerability scanners (WVSs) are tools that can detect security vulnerabilities in web services. Although both commercial and open-source WVSs exist, their vulnerability detection capability and performance vary. In this article, we report on a comparative study to determine the vulnerability detection capabilities of eight WVSs (both open and commercial) using two vulnerable web applications: WebGoat and Damn vulnerable web application. The eight WVSs studied were: Acunetix; HP WebInspect; IBM AppScan; OWASP ZAP; Skipfish; Arachni; Vega; and Iron WASP. The performance was evaluated using multiple evaluation metrics: precision; recall; Youden index; OWASP web benchmark evaluation; and the web application security scanner evaluation criteria. The experimental results show that, while the commercial scanners are effective in detecting security vulnerabilities, some open-source scanners (such as ZAP and Skipfish) can also be effective. In summary, this study recommends improving the vulnerability detection capabilities of both the open-source and commercial scanners to enhance code coverage and the detection rate, and to reduce the number of false-positives

A Stratification and Sampling Model for Bellwether Moving Window

An effective method for finding the relevant number (window size) and the elapsed time (window age) of recently completed projects has proven elusive in software effort estimation. Although these two parameters significantly affect the prediction accuracy, there is no effective method to stratify and sample chronological projects to improve prediction performance of software effort estimation models. Exemplary projects (Bellwether) representing the training set have been empirically validated to improve the prediction accuracy in the domain of software defect prediction. However, the concept of Bellwether and its effect have not been empirically proven in software effort estimation as a method of selecting exemplary/relevant projects with defined window size and age. In view of this, we introduce a novel method for selecting relevant and recently completed projects referred to as Bellwether moving window for improving the software effort prediction accuracy. We first sort and cluster a pool of N projects and apply statistical stratification based on Markov chain modeling to select the Bellwether moving window. We evaluate the proposed approach using the baseline Automatically Transformed Linear Model on the ISBSG dataset. Results show that (1) Bellwether effect exist in software effort estimation dataset, (2) the Bellwether moving window with a window size of 82 to 84 projects and window age of 1.5 to 2 years resulted in an improved prediction accuracy than the traditional approach