Gray-box combinatorial interaction testing

The enourmous size of configuration spaces in highly configurable softwares pose challenges to testing. Typically exhaustive testing is neither an option nor a way. Combinatorial interaction techiques are a systematic way to test such enourmous configuration spaces by a systematic way of sampling the space, employed through covering arrays. A t-way covering array is a sampled subset of configurations which contains all t-way option setting combinations. Testing through t-way covering arrays is proven to be highly e ective at revealing failures caused by interaction of t or fewer options. Although, traditional covering arrays are e ective however, we’ve observed that they su er in the presence of complex interactions among configuration options, referred as tangled options. A tangled configuration option is described as either a configuration option with complex structure and/or nested in hierarchy of configuration options. In this thesis, we conjecture the e ectiveness of CIT in the presence of tangled options can greatly be improved, by analyzing the system’s source code. The analysis of source code reveals the interaction of configuration options with each other, this information can be used to determine which additional option setting combinations and the conditions under which these combinations must be tested. Gray-box testing methods rely on partial structural information of the system during testing. We’ve statically analyzed the source code of subject applications to extract the structure and hierachy of configuration options. Each configuration option has been structurally tested according to a test criterion against a t-way covering array and subsequently their t-way interactions. The criterion revealed the missing coverage of options which were employed to drive the additional testcase generation phase to acheive complete coverage. We present a number of novel CIT coverage criteria for t-wise interaction testing of configuration options. In this thesis, we’ve conducted a series of large scale experiments on 18 di erent real-world highly configurable software applications from di erent application domains to evaluate the proposed approach. We’ve observed that traditional t-way CAs can provide above 80% coverage for configuration options testing. However, they significantly su er to provide interaction coverage under high t and tangling e ects where coverage is dropped to less than 50%. Our work address these issues and propose a technique to acheive complete coverage

Moving forward with combinatorial interaction testing

Combinatorial interaction testing (CIT) is an efficient and effective method of detecting failures that are caused by the interactions of various system input parameters. In this paper, we discuss CIT, point out some of the difficulties of applying it in practice, and highlight some recent advances that have improved CIT’s applicability to modern systems. We also provide a roadmap for future research and directions; one that we hope will lead to new CIT research and to higher quality testing of industrial systems

A survey of self organisation in future cellular networks

This article surveys the literature over the period of the last decade on the emerging field of self organisation as applied to wireless cellular communication networks. Self organisation has been extensively studied and applied in adhoc networks, wireless sensor networks and autonomic computer networks; however in the context of wireless cellular networks, this is the first attempt to put in perspective the various efforts in form of a tutorial/survey. We provide a comprehensive survey of the existing literature, projects and standards in self organising cellular networks. Additionally, we also aim to present a clear understanding of this active research area, identifying a clear taxonomy and guidelines for design of self organising mechanisms. We compare strength and weakness of existing solutions and highlight the key research areas for further development. This paper serves as a guide and a starting point for anyone willing to delve into research on self organisation in wireless cellular communication networks

ACTS in Need: Automatic Configuration Tuning with Scalability Guarantees

To support the variety of Big Data use cases, many Big Data related systems expose a large number of user-specifiable configuration parameters. Highlighted in our experiments, a MySQL deployment with well-tuned configuration parameters achieves a peak throughput as 12 times much as one with the default setting. However, finding the best setting for the tens or hundreds of configuration parameters is mission impossible for ordinary users. Worse still, many Big Data applications require the support of multiple systems co-deployed in the same cluster. As these co-deployed systems can interact to affect the overall performance, they must be tuned together. Automatic configuration tuning with scalability guarantees (ACTS) is in need to help system users. Solutions to ACTS must scale to various systems, workloads, deployments, parameters and resource limits. Proposing and implementing an ACTS solution, we demonstrate that ACTS can benefit users not only in improving system performance and resource utilization, but also in saving costs and enabling fairer benchmarking

BestConfig: Tapping the Performance Potential of Systems via Automatic Configuration Tuning

An ever increasing number of configuration parameters are provided to system users. But many users have used one configuration setting across different workloads, leaving untapped the performance potential of systems. A good configuration setting can greatly improve the performance of a deployed system under certain workloads. But with tens or hundreds of parameters, it becomes a highly costly task to decide which configuration setting leads to the best performance. While such task requires the strong expertise in both the system and the application, users commonly lack such expertise. To help users tap the performance potential of systems, we present BestConfig, a system for automatically finding a best configuration setting within a resource limit for a deployed system under a given application workload. BestConfig is designed with an extensible architecture to automate the configuration tuning for general systems. To tune system configurations within a resource limit, we propose the divide-and-diverge sampling method and the recursive bound-and-search algorithm. BestConfig can improve the throughput of Tomcat by 75%, that of Cassandra by 63%, that of MySQL by 430%, and reduce the running time of Hive join job by about 50% and that of Spark join job by about 80%, solely by configuration adjustment

The Maunakea Spectroscopic Explorer Book 2018

(Abridged) This is the Maunakea Spectroscopic Explorer 2018 book. It is intended as a concise reference guide to all aspects of the scientific and technical design of MSE, for the international astronomy and engineering communities, and related agencies. The current version is a status report of MSE's science goals and their practical implementation, following the System Conceptual Design Review, held in January 2018. MSE is a planned 10-m class, wide-field, optical and near-infrared facility, designed to enable transformative science, while filling a critical missing gap in the emerging international network of large-scale astronomical facilities. MSE is completely dedicated to multi-object spectroscopy of samples of between thousands and millions of astrophysical objects. It will lead the world in this arena, due to its unique design capabilities: it will boast a large (11.25 m) aperture and wide (1.52 sq. degree) field of view; it will have the capabilities to observe at a wide range of spectral resolutions, from R2500 to R40,000, with massive multiplexing (4332 spectra per exposure, with all spectral resolutions available at all times), and an on-target observing efficiency of more than 80%. MSE will unveil the composition and dynamics of the faint Universe and is designed to excel at precision studies of faint astrophysical phenomena. It will also provide critical follow-up for multi-wavelength imaging surveys, such as those of the Large Synoptic Survey Telescope, Gaia, Euclid, the Wide Field Infrared Survey Telescope, the Square Kilometre Array, and the Next Generation Very Large Array.Comment: 5 chapters, 160 pages, 107 figure