DeepEvolution: A Search-Based Testing Approach for Deep Neural Networks

The increasing inclusion of Deep Learning (DL) models in safety-critical systems such as autonomous vehicles have led to the development of multiple model-based DL testing techniques. One common denominator of these testing techniques is the automated generation of test cases, e.g., new inputs transformed from the original training data with the aim to optimize some test adequacy criteria. So far, the effectiveness of these approaches has been hindered by their reliance on random fuzzing or transformations that do not always produce test cases with a good diversity. To overcome these limitations, we propose, DeepEvolution, a novel search-based approach for testing DL models that relies on metaheuristics to ensure a maximum diversity in generated test cases. We assess the effectiveness of DeepEvolution in testing computer-vision DL models and found that it significantly increases the neuronal coverage of generated test cases. Moreover, using DeepEvolution, we could successfully find several corner-case behaviors. Finally, DeepEvolution outperformed Tensorfuzz (a coverage-guided fuzzing tool developed at Google Brain) in detecting latent defects introduced during the quantization of the models. These results suggest that search-based approaches can help build effective testing tools for DL systems

Mathematics in Software Reliability and Quality Assurance

This monograph concerns the mathematical aspects of software reliability and quality assurance and consists of 11 technical papers in this emerging area. Included are the latest research results related to formal methods and design, automatic software testing, software verification and validation, coalgebra theory, automata theory, hybrid system and software reliability modeling and assessment

Software Test Case Generation Tools and Techniques: A Review

Software Industry is evolving at a very fast pace since last two decades. Many software developments, testing and test case generation approaches have evolved in last two decades to deliver quality products and services. Testing plays a vital role to ensure the quality and reliability of software products. In this paper authors attempted to conduct a systematic study of testing tools and techniques. Six most popular e-resources called IEEE, Springer, Association for Computing Machinery (ACM), Elsevier, Wiley and Google Scholar to download 738 manuscripts out of which 125 were selected to conduct the study. Out of 125 manuscripts selected, a good number approx. 79% are from reputed journals and around 21% are from good conference of repute. Testing tools discussed in this paper have broadly been divided into five different categories: open source, academic and research, commercial, academic and open source, and commercial & open source. The paper also discusses several benchmarked datasets viz. Evosuite 10, SF100 Corpus, Defects4J repository, Neo4j, JSON, Mocha JS, and Node JS to name a few. Aim of this paper is to make the researchers aware of the various test case generation tools and techniques introduced in the last 11 years with their salient features

A two-stage structural optimisation and thermal discretisation of non-convective structured insulators: applications in granular-solid structures by additive manufacturing technology

A systematic design procedure for characterising the strength and insulation requirements of a modular unit structure from additive manufacturing has been presented. The proposed 'two-stage' method consists of structural optimisation and thermal 'discretisation', through use of the Metamorphic Development (MD) and Discretisation by Partitioning Method (DbPM), respectively. A structural layout optimisation method of a consolidated granular-solid structure for strength requirements is demonstrated. The reliability of the layout optimized design solution tested using experiments and finite element analysis (PEA) are reproduced with reasonable accuracy. Layout optimisation yielded 40% savings in build material, whilst satisfying the targeted deflection. [Continues.

Deep geothermal exploration by means of electromagnetic methods: New insights from the Larderello geothermal field (Italy)

The main target of this research is the improvement of the knowledge on the deep structures of the Larderello-Travale geothermal field (Tuscany, Italy), with a focus on the Lago Boracifero sector, particularly on the heat source of the system, the tectonics and its relation with the hydrothermal circulation. In the frame of the PhD program and of the IMAGE project (Integrated Methods for Advanced Geothermal Exploration; EU FP7), we acquired new magnetotelluric (MT) and Time Domain EM (TDEM) data in a key sector of the field (Lago Boracifero). These data integrate the MT datasets previously acquired in the frame of exploration and scientific projects. This study is based also on a integrated modelling, which included and organized in Petrel (Schlumberger) environment, a large quantity of geological and geophysical data. We also propose an integrated approach to improve the reliability of the 2D MT inversion models, by using external information from the integrated model of the field as well as an innovative probabilistic analysis of the MT data. We present our attempt to treat the 1D magnetotelluric inverse problem with a probabilistic approach, by adopting the Particle Swarm Optimization (PSO), a heuristic method based on the concept of the adaptive behaviour to solve complex problems. The user-friendly software “GlobalEM” was implemented for the analysis and probabilistic optimization of MT data. The results from theoretical and measured MT data are promising, also for the possibility to implement different schemes of constrained optimization as well as joint optimization (e.g. MT and TDEM). The analysis of the a-posteriori distribution of the results can be of help to understand the reliability of the model. The 2D MT inversion models and the integrated study of the Larderello-Travale geothermal field improved the knowledge about the deep structures of the system, with a relevant impact on the conceptual geothermal model. In Micaschist and Gneiss complexes we observed a generally high electrical resistivity response locally interrupted by low resistivity anomalies that are well correlated with the most productive sectors of the field. A still partial melted igneous intrusion beneath the Lago Boracifero sector was detected based on the interpretation of the low resistivity anomalies located at a mid-crustal level (> 6 km). New insights on the tectonics are proposed in this research. The fundamental role of a large tectonic structure, i.e. the Cornia Fault, located along the homonymous river, was highlighted. In our opinion, this fault played an important role in the geothermal evolution of the Lago Boracifero sector, favouring both the hydrothermal circulation and the emplacement of magma bodies. In our opinion, the system can be ascribed to a “young convective and intrusive” field feed by a complex composite batholite

Towards Debugging and Testing Deep Learning Systems

Au cours des dernières années, l’apprentissage profond, en anglais Deep Learning (DL) a fait d’énormes progrès, en atteignant et dépassant même parfois le niveau de performance des humains pour différentes tâches, telles que la classification des images et la reconnaissance vocale. Grâce à ces progrès, nous constatons une large adoption du DL dans des applications critiques, telles que la conduite autonome de véhicules, la prévention et la détection du crime, et le traitement médical. Cependant, malgré leurs progrès spectaculaires, les systèmes de DL, tout comme les logiciels traditionnels, présentent souvent des comportements erronés en raison de l’existence de défauts cachés ou d’inefficacités. Ces comportements erronés peuvent être à l’origine d’accidents catastrophiques. Ainsi, l’assurance de la qualité des logiciels (SQA), y compris la fiabilité et la robustesse, pour les systèmes de DL devient une préoccupation majeure. Les tests traditionnels pour les modèles de DL consistent à mesurer leurs performances sur des données collectées manuellement ; ils dépendent donc fortement de la qualité des données de test qui, souvent, n’incluent pas de données d’entrée rares, comme en témoignent les récents accidents de voitures avec conduite autonome (exemple Tesla/Uber). Les techniques de test avancées sont très demandées pour améliorer la fiabilité des systèmes de DL. Néanmoins, les tests des systèmes de DL posent des défis importants, en raison de leur nature non-déterministe puisqu’ils suivent un paradigme axé sur les données (la tâche cible est apprise statistiquement) et leur manque d’oracle puisqu’ils sont conçus principalement pour fournir la réponse. Récemment, les chercheurs en génie logiciel ont commencé à adapter des concepts du domaine du test logiciel tels que la couverture des cas de tests et les pseudo-oracles, pour résoudre ces difficultés. Malgré les résultats prometteurs obtenus de cette rénovation des méthodes existantes de test logiciel, le domaine du test des systèmes de DL est encore immature et les méthodes proposées à ce jour ne sont pas très efficaces. Dans ce mémoire, nous examinons les solutions existantes proposées pour tester les systèmes de DL et proposons quelques nouvelles techniques. Nous réalisons cet objectif en suivant une approche systématique qui consiste à : (1) étudier les problèmes et les défis liés aux tests des logiciels de DL; (2) souligner les forces et les faiblesses des techniques de test logiciel adaptées aux systèmes de DL; (3) proposer de nouvelles solutions de test pour combler certaines lacunes identifiées dans la littérature, et potentiellement aider à améliorer l’assurance qualité des systèmes de DL.----------ABSTRACT: Over the past few years, Deep Learning (DL) has made tremendous progress, achieving or surpassing human-level performance for different tasks such as image classification and speech recognition. Thanks to these advances, we are witnessing a wide adoption of DL in safetycritical applications such as autonomous driving cars, crime prevention and detection, and medical treatment. However, despite their spectacular progress, DL systems, just like traditional software systems, often exhibit erroneous corner-cases behaviors due to the existence of latent defects or inefficiencies, and which can lead to catastrophic accidents. Thus, software quality assurance (SQA), including reliability and robustness, for DL systems becomes a big concern. Traditional testing for DL models consists of measuring their performance on manually collected data ; so it heavily depends on the quality of the test data that often fails to include rare inputs, as evidenced by recent autonomous-driving car accidents (e.g., Tesla/Uber). Advanced testing techniques are in high demand to improve the trustworthiness of DL systems. Nevertheless, DL testing poses significant challenges stemming from the non-deterministic nature of DL systems (since they follow a data-driven paradigm ; the target task is learned statistically) and their lack of oracle (since they are designed principally to provide the answer). Recently, software researchers have started adapting concepts from the software testing domain such as test coverage and pseudo-oracles to tackle these difficulties. Despite some promising results obtained from adapting existing software testing methods, current software testing techniques for DL systems are still quite immature. In this thesis, we examine existing testing techniques for DL systems and propose some new techniques. We achieve this by following a systematic approach consisting of : (1) investigating DL software issues and testing challenges ; (2) outlining the strengths and weaknesses of the software-based testing techniques adapted for DL systems ; and (3) proposing novel testing solutions to fill some of the identified literature gaps, and potentially help improving the SQA of DL systems

Implementation Approach of Unit and Integration Testing Method Based on Recent Advancements in Functional Software Testing

Finding bugs and flaws, detecting invalid or inaccurate functionality, and analyzing and certifying the entire softwareproduct all require software testing. We looked at unit testing and integration testing in this project since they are two fundamental stages of software testing and are significantly associated. For both unit and integration testing, a sufficient number of testing methodologies and approaches have been assessed and contrasted, with each implementation system, algorithm, and technique being thoroughly scrutinized. Some of them are effective in finding as many hidden defects as possible while also reducing testing complexity, time, and expense. In this context, we chose sOrTES, a stochastic scheduling support tool that would be utilized for manual integration test cases. The chosen strategy is the most appropriate since empirical evidence reveals that it can prevent around 40% of testing failures while also increasing requirement coverage by 9.6%

MUSME 2011 4 th International Symposium on Multibody Systems and Mechatronics

El libro de actas recoge las aportaciones de los autores a través de los correspondientes artículos a la Dinámica de Sistemas Multicuerpo y la Mecatrónica (Musme). Estas disciplinas se han convertido en una importante herramienta para diseñar máquinas, analizar prototipos virtuales y realizar análisis CAD sobre complejos sistemas mecánicos articulados multicuerpo. La dinámica de sistemas multicuerpo comprende un gran número de aspectos que incluyen la mecánica, dinámica estructural, matemáticas aplicadas, métodos de control, ciencia de los ordenadores y mecatrónica. Los artículos recogidos en el libro de actas están relacionados con alguno de los siguientes tópicos del congreso: Análisis y síntesis de mecanismos ; Diseño de algoritmos para sistemas mecatrónicos ; Procedimientos de simulación y resultados ; Prototipos y rendimiento ; Robots y micromáquinas ; Validaciones experimentales ; Teoría de simulación mecatrónica ; Sistemas mecatrónicos ; Control de sistemas mecatrónicosUniversitat Politècnica de València (2011). MUSME 2011 4 th International Symposium on Multibody Systems and Mechatronics. Editorial Universitat Politècnica de València. http://hdl.handle.net/10251/13224Archivo delegad

Mathematical Problems in Rock Mechanics and Rock Engineering

With increasing requirements for energy, resources and space, rock engineering projects are being constructed more often and are operated in large-scale environments with complex geology. Meanwhile, rock failures and rock instabilities occur more frequently, and severely threaten the safety and stability of rock engineering projects. It is well-recognized that rock has multi-scale structures and involves multi-scale fracture processes. Meanwhile, rocks are commonly subjected simultaneously to complex static stress and strong dynamic disturbance, providing a hotbed for the occurrence of rock failures. In addition, there are many multi-physics coupling processes in a rock mass. It is still difficult to understand these rock mechanics and characterize rock behavior during complex stress conditions, multi-physics processes, and multi-scale changes. Therefore, our understanding of rock mechanics and the prevention and control of failure and instability in rock engineering needs to be furthered. The primary aim of this Special Issue “Mathematical Problems in Rock Mechanics and Rock Engineering” is to bring together original research discussing innovative efforts regarding in situ observations, laboratory experiments and theoretical, numerical, and big-data-based methods to overcome the mathematical problems related to rock mechanics and rock engineering. It includes 12 manuscripts that illustrate the valuable efforts for addressing mathematical problems in rock mechanics and rock engineering