Some Approaches for Software Defect Prediction

Käesoleva töö peamiseks eesmärgiks on anda üldisem ülevaade protsessidest tarkvara vigade hindamise mudelites, mis kasutavad masinõppe klassifikaatoreid, ja analüüsida mõningaid hindamiseskperimentide tulemusi, mis on läbi viidud antud töös refereeritud uurimistöödes. Lisaks on antud lühike selgitus antud töös vaadeldavates tarkvara vigade hindamise mudelites kasutatud algoritmidest ja tuuakse välja ning seletatakse lahti mõned hinnangumõõdikud, mida kasutatakse tarkvara vigade hindamise mudelite hindamistäpsuste mõõtmiseks. Tuuakse välja ka üldine ülevaade vaadeldavates tarkvara vigade hindamise mudelites toimuvatest protsessidest.The main idea of this thesis is to give a general overview of the processes within the soft-ware defect prediction models using machine learning classifiers and to provide analysis to some of the results of the evaluation experiments conducted in the research papers covered in this work. Additionally, a brief explanation of the algorithms used within the software defect prediction models covered in this work is given and some of the evaluation measures used to evaluate the prediction accuracy of software defect prediction models are listed and explained. Also, a general overview of the processes within a handful of specific software defect prediction models is provided

Endless Data

Small and Medium Enterprises (SMEs), as well as micro teams, face an uphill task when delivering software to the Cloud. While rapid release methods such as Continuous Delivery can speed up the delivery cycle: software quality, application uptime and information management remain key concerns. This work looks at four aspects of software delivery: crowdsourced testing, Cloud outage modelling, collaborative chat discourse modelling, and collaborative chat discourse segmentation. For each aspect, we consider business related questions around how to improve software quality and gain more significant insights into collaborative data while respecting the rapid release paradigm

A New Approach for Predicting Security Vulnerability Severity in Attack Prone Software Using Architecture and Repository Mined Change Metrics

Billions of dollars are lost every year to successful cyber attacks that are fundamentally enabled by software vulnerabilities. Modern cyber attacks increasingly threaten individuals, organizations, and governments, causing service disruption, inconvenience, and costly incident response. Given that such attacks are primarily enabled by software vulnerabilities, this work examines the efficacy of using change metrics, along with architectural burst and maintainability metrics, to predict modules and files that might be analyzed or tested further to excise vulnerabilities prior to release. The problem addressed by this research is the residual vulnerability problem, or vulnerabilities that evade detection and persist in released software. Many modern software projects are over a million lines of code, and composed of reused components of varying maturity. The sheer size of modern software, along with the reuse of existing open source modules, complicates the questions of where to look, and in what order to look, for residual vulnerabilities. Traditional code complexity metrics, along with newer frequency based churn metrics (mined from software repository change history), are selected specifically for their relevance to the residual vulnerability problem. We compare the performance of these complexity and churn metrics to architectural level change burst metrics, automatically mined from the git repositories of the Mozilla Firefox Web Browser, Apache HTTP Web Server, and the MySQL Database Server, for the purpose of predicting attack prone files and modules. We offer new empirical data quantifying the relationship between our selected metrics and the severity of vulnerable files and modules, assessed using severity data compiled from the NIST National Vulnerability Database, and cross-referenced to our study subjects using unique identifiers defined by the Common Vulnerabilities and Exposures (CVE) vulnerability catalog. Specifically, we evaluate our metrics against the severity scores from CVE entries associated with known-vulnerable files and modules. We use the severity scores according to the Base Score Metric from the Common Vulnerability Scoring System (CVSS), corresponding to applicable CVE entries extracted from the NIST National Vulnerability Database, which we associate with vulnerable files and modules via automated and semi-automated techniques. Our results show that architectural level change burst metrics can perform well in situations where more traditional complexity metrics fail as reliable estimators of vulnerability severity. In particular, results from our experiments on Apache HTTP Web Server indicate that architectural level change burst metrics show high correlation with the severity of known vulnerable modules, and do so with information directly available from the version control repository change-set (i.e., commit) history

Deep Learning: Our Miraculous Year 1990-1991

In 2020, we will celebrate that many of the basic ideas behind the deep learning revolution were published three decades ago within fewer than 12 months in our "Annus Mirabilis" or "Miraculous Year" 1990-1991 at TU Munich. Back then, few people were interested, but a quarter century later, neural networks based on these ideas were on over 3 billion devices such as smartphones, and used many billions of times per day, consuming a significant fraction of the world's compute.Comment: 37 pages, 188 references, based on work of 4 Oct 201

Physics-Informed Computer Vision: A Review and Perspectives

Incorporation of physical information in machine learning frameworks are opening and transforming many application domains. Here the learning process is augmented through the induction of fundamental knowledge and governing physical laws. In this work we explore their utility for computer vision tasks in interpreting and understanding visual data. We present a systematic literature review of formulation and approaches to computer vision tasks guided by physical laws. We begin by decomposing the popular computer vision pipeline into a taxonomy of stages and investigate approaches to incorporate governing physical equations in each stage. Existing approaches in each task are analyzed with regard to what governing physical processes are modeled, formulated and how they are incorporated, i.e. modify data (observation bias), modify networks (inductive bias), and modify losses (learning bias). The taxonomy offers a unified view of the application of the physics-informed capability, highlighting where physics-informed learning has been conducted and where the gaps and opportunities are. Finally, we highlight open problems and challenges to inform future research. While still in its early days, the study of physics-informed computer vision has the promise to develop better computer vision models that can improve physical plausibility, accuracy, data efficiency and generalization in increasingly realistic applications

Residual Compressive Strength Prediction of Carbon/Epoxy Laminates Subjected to Low Velocity Impact Damage

Low energy impact damage to a composite structure is difficult to detect and can have profound effects on compressive strengths. Low energy impact damage is sometimes termed as barely visible impact damage (BVID). Detecting BVID is only possible by implementing nondestructive testing (NDT) techniques. Depending upon the support conditions, material system, laminate thickness, lay-up orientation, and impactor geometry, velocity, and hardness, the types of damage associated with BVID include delaminations, longitudinal and transverse matrix cracks, and in some cases, fiber breaks. Material properties such as the strengths of the matrix, fibers, fiber/matrix interface, and more important for BVID, ply interface properties in a multi-ply laminate, are all parameters that determine impact resistance. After the composite structure experiences BVID, the depletion of the structural strength is determined as result of compression after impact (CAI) material testing. The primary emphasis of this research is to predict structural compressive strength after low energy/low velocity impact using neural networks. After the composite structure absorbs BVID, it is common to determine structural strength depletion based on impact energy. Because impact energy is seldom known in real world applications, it is more reasonable to determine ultimate strength based on amount of damage present. The technique used in this research to assess the damage and predict ultimate strength includes ultrasonic testing (UT), to generate an image representing the damage, and neural networks to predict future performance. Using the pixel data from the ultrasonic C-scan image of the impact damage, in conjunction with CAI testing, and analyzing it with a backpropagation neural network, correlations on ultimate compressive strength can be made. This analysis demonstrates the ability of a neural network to predict the ultimate compressive strengths of impact damaged composite structures using UT data

Recent Advances in Signal Processing

The signal processing task is a very critical issue in the majority of new technological inventions and challenges in a variety of applications in both science and engineering fields. Classical signal processing techniques have largely worked with mathematical models that are linear, local, stationary, and Gaussian. They have always favored closed-form tractability over real-world accuracy. These constraints were imposed by the lack of powerful computing tools. During the last few decades, signal processing theories, developments, and applications have matured rapidly and now include tools from many areas of mathematics, computer science, physics, and engineering. This book is targeted primarily toward both students and researchers who want to be exposed to a wide variety of signal processing techniques and algorithms. It includes 27 chapters that can be categorized into five different areas depending on the application at hand. These five categories are ordered to address image processing, speech processing, communication systems, time-series analysis, and educational packages respectively. The book has the advantage of providing a collection of applications that are completely independent and self-contained; thus, the interested reader can choose any chapter and skip to another without losing continuity

Multiple Surface Pipeline Leak Detection Using Real-Time Sensor Data Analysis

Pipelines enable the largest volume of both intra and international transportation of oil and gas and play critical roles in the energy sufficiency of countries. The biggest drawback with the use of pipelines for oil and gas transportation is the problem of oil spills whenever the pipelines lose containment. The severity of the oil spill on the environment is a function of the volume of the spill and this is a function of the time taken to detect the leak and contain the spill from the pipeline. A single leak on the Enbridge pipeline spilled 3.3 million liters into the Kalamazoo river while a pipeline rupture in North Dakota which went undetected for 143 days spilled 29 million gallons into the environment.Several leak detection systems (LDS) have been developed with the capacity for rapid detection and localization of pipeline leaks, but the characteristics of these LDS limit their leak detection capability. Machine learning provides an opportunity to develop faster LDS, but it requires access to pipeline leak datasets that are proprietary in nature and not readily available. Current LDS have difficulty in detecting low-volume/low-pressure spills located far away from the inlet and outlet pressure sensors. Some reasons for this include the following, leak induced pressure variation generated by these leaks is dissipated before it gets to the inlet and outlet pressure sensors, another reason is that the LDS are designed for specific minimum detection levels which is a percentage of the flow volume of the pipeline, so when the leak falls below the LDS minimum detection value, the leak will not be detected. Perturbations generated by small volume leaks are often within the threshold values of the pipeline\u27s normal operational envelop as such the LDS disregards these perturbations. These challenges have been responsible for pipeline leaks going on for weeks only to be detected by third-party persons in the vicinity of the leaks. This research has been able to develop a framework for the generation of pipeline datasets using the PIPESIM software and the RAND function in Python. The topological data of the pipeline right of way, the pipeline network design specification, and the fluid flow properties are the required information for this framework. With this information, leaks can be simulated at any point on the pipeline and the datasets generated. This framework will facilitate the generation of the One-class dataset for the pipeline which can be used for the development of LDS using machine learning. The research also developed a leak detection topology for detecting low-volume leaks. This topology comprises of the installation of a pressure sensor with remote data transmission capacity at the midpoint of the line. The sensor utilizes the exception-based transmission scheme where it only transmits when the new data differs from the existing data value. This will extend the battery life of the sensor. The installation of the sensor at the midpoint of the line was found to increase the sensitivity of the LDS to leak-induced pressure variations which were traditionally dissipated before getting to the Inlet/outlet sensors. The research also proposed the development of a Leak Detection as a Service (LDaaS) platform where the pressure data from the inlet and the midpoint sensors are collated and subjected to a specially developed leak detection algorithm for the detection of pipeline leaks. This leak detection topology will enable operators to detect low-volume/low-pressure leaks that would have been missed by the existing leak detection system and deploy the oil spill response plans quicker thus reducing the volume of oil spilled into the environment. It will also provide a platform for regulators to monitor the leak alerts as they are generated and enable them to evaluate the oil spill response plans of the operators

Automated Quality Assessment of Natural Language Requirements

High demands on quality and increasing complexity are major challenges in the development of industrial software in general. The development of automotive software in particular is subject to additional safety, security, and legal demands. In such software projects, the specification of requirements is the first concrete output of the development process and usually the basis for communication between manufacturers and development partners. The quality of this output is therefore decisive for the success of a software development project. In recent years, many efforts in academia and practice have been targeted towards securing and improving the quality of requirement specifications. Early improvement approaches concentrated on the assistance of developers in formulating their requirements. Other approaches focus on the use of formal methods; but despite several advantages, these are not widely applied in practice today. Most software requirements today are informal and still specified in natural language. Current and previous research mainly focuses on quality characteristics agreed upon by the software engineering community. They are described in the standard ISO/IEC/IEEE 29148:2011, which offers nine essential characteristics for requirements quality. Several approaches focus additionally on measurable indicators that can be derived from text. More recent publications target the automated analysis of requirements by assessing their quality characteristics and by utilizing methods from natural language processing and techniques from machine learning. This thesis focuses in particular on the reliability and accuracy in the assessment of requirements and addresses the relationships between textual indicators and quality characteristics as defined by global standards. In addition, an automated quality assessment of natural language requirements is implemented by using machine learning techniques. For this purpose, labeled data is captured through assessment sessions. In these sessions, experts from the automotive industry manually assess the quality characteristics of natural language requirements.% as defined in ISO 29148. The research is carried out in cooperation with an international engineering and consulting company and enables us to access requirements from automotive software development projects of safety and comfort functions. We demonstrate the applicability of our approach for real requirements and present promising results for an industry-wide application

A Machine Learning approach for damage detection and localisation in Wind Turbine Gearbox Bearings

Increasing demand for renewable sources requires more cost-effective solutions to mitigate the cost of maintenance and produce more energy. Preventive maintenance is the most normally adopted scheme in industry for maintenance but despite being well accepted has severe limitations. Its inability to intelligently schedule maintenance at the right time and prevent unexpected breakdowns are the main downsides of this approach and consequently leads to several problems such as unnecessary maintenances. This strategy does not justify the additional costs and thereby represents a negative aspect for renewable energy resource companies that try to generate cost-competitive energy. These challenges are progressively leading towards the predictive maintenance approach to overcome these aforementioned issues. Wind Turbine Gearbox Bearings have received a lot of attention due to the high incidence failure rates provoked by the harsh operational and environmental conditions. Current techniques only reach a level one of diagnostics commonly known as the Novelty Detection stage and normally requires the expertise of a skilled operator to interpret data and infer damage from it. A data-driven approach by using Machine Learning methods has been used to tackle the damage detection and location stage in bearing components. The damage location was performed by using non-destructive methods such as the Acoustic Emission technique — these measurements were used as features to locate damage around the bearing component once the damage was detected. The implementation of this stages also led to the exploration of damage generation due to overload defects and proposed a methodology to simulate these defects in bearings — the study of this concept was implemented in a scaled-down experiment where damage detection and localisation was performed. Due to the importance of the implementation of a damage location stage, damage in AE sensors was also explored in this work. Features extracted from impedance curves allowed to train Machine Learning methods to trigger a novelty when a bonding scenario occurred. This ultimately allowed the identification of unhealthy sensors in the network that could potentially generate spurious results in the damage predictions stage