Metamorphic relations for enhancing system understanding and use

Modern information technology paradigms, such as online services and off-the-shelf products, often involve a wide variety of users with different or even conflicting objectives. Every software output may satisfy some users, but may also fail to satisfy others. Furthermore, users often do not know the internal working mechanisms of the systems. This situation is quite different from bespoke software, where developers and users usually know each other. This paper proposes an approach to help users to better understand the software that they use, and thereby more easily achieve their objectives—even when they do not fully understand how the system is implemented. Our approach borrows the concept of metamorphic relations from the field of metamorphic testing (MT), using it in an innovative way that extends beyond MT. We also propose a "symmetry" metamorphic relation pattern and a "change direction" metamorphic relation input pattern that can be used to derive multiple concrete metamorphic relations. Empirical studies reveal previously unknown failures in some of the most popular applications in the world, and show how our approach can help users to better understand and better use the systems. The empirical results provide strong evidence of the simplicity, applicability, and effectiveness of our methodology

Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testing

We have recently witnessed tremendous success of Machine Learning (ML) in practical applications. Computer vision, speech recognition and language translation have all seen a near human level performance. We expect, in the near future, most business applications will have some form of ML. However, testing such applications is extremely challenging and would be very expensive if we follow today's methodologies. In this work, we present an articulation of the challenges in testing ML based applications. We then present our solution approach, based on the concept of Metamorphic Testing, which aims to identify implementation bugs in ML based image classifiers. We have developed metamorphic relations for an application based on Support Vector Machine and a Deep Learning based application. Empirical validation showed that our approach was able to catch 71% of the implementation bugs in the ML applications.Comment: Published at 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2018

Robust Watermarking using Hidden Markov Models

Software piracy is the unauthorized copying or distribution of software. It is a growing problem that results in annual losses in the billions of dollars. Prevention is a difficult problem since digital documents are easy to copy and distribute. Watermarking is a possible defense against software piracy. A software watermark consists of information embedded in the software, which allows it to be identified. A watermark can act as a deterrent to unauthorized copying, since it can be used to provide evidence for legal action against those responsible for piracy.In this project, we present a novel software watermarking scheme that is inspired by the success of previous research focused on detecting metamorphic viruses. We use a trained hidden Markov model (HMM) to detect a specific copy of software. We give experimental results that show our scheme is robust. That is, we can identify the original software even after it has been extensively modified, as might occur as part of an attack on the watermarking scheme

Malicious cryptography techniques for unreversable (malicious or not) binaries

Fighting against computer malware require a mandatory step of reverse engineering. As soon as the code has been disassemblied/decompiled (including a dynamic analysis step), there is a hope to understand what the malware actually does and to implement a detection mean. This also applies to protection of software whenever one wishes to analyze them. In this paper, we show how to amour code in such a way that reserse engineering techniques (static and dymanic) are absolutely impossible by combining malicious cryptography techniques developped in our laboratory and new types of programming (k-ary codes). Suitable encryption algorithms combined with new cryptanalytic approaches to ease the protection of (malicious or not) binaries, enable to provide both total code armouring and large scale polymorphic features at the same time. A simple 400 Kb of executable code enables to produce a binary code and around $2^{140}$ mutated forms natively while going far beyond the old concept of decryptor.Comment: 17 pages, 2 figures, accepted for presentation at H2HC'1

Test Case Purification for Improving Fault Localization

Finding and fixing bugs are time-consuming activities in software development. Spectrum-based fault localization aims to identify the faulty position in source code based on the execution trace of test cases. Failing test cases and their assertions form test oracles for the failing behavior of the system under analysis. In this paper, we propose a novel concept of spectrum driven test case purification for improving fault localization. The goal of test case purification is to separate existing test cases into small fractions (called purified test cases) and to enhance the test oracles to further localize faults. Combining with an original fault localization technique (e.g., Tarantula), test case purification results in better ranking the program statements. Our experiments on 1800 faults in six open-source Java programs show that test case purification can effectively improve existing fault localization techniques

Literacy and Thinking Tools for Science Teachers

Literacy and thinking tools, such as Venn diagrams, are construction tools for the mind. Just as carpenters use tools to construct a piece of furniture, literate thinkers learning science can use tools to construct new scientific understandings. Like tools used by a carpenter, some literacy and thinking tools are purpose-built for science education; Josephine used a Venn diagram tool because she wanted to compare her pet bird to a bald eagle. Just as a screwdriver is built to slot into the head of a screw and rotate it, you can use literacy and thinking tools for subject- and text-specific purposes. In this chapter, we examine some characteristics of literacy and thinking tools (Whitehead, 2001, 2004). A list of these tools, together with the chapters associ-ated with their use, is provided in Table 2:1