Towards more effective testing of communications-critical large scale systems

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.A publication based on the research from this thesis has been published and can be cited as Nabulsi, MA and Hierons, RM (2014), A new test framework for communications-critical large scale systems, IEEE Software, In press. The published version can be accessed via the link below.None of today’s large scale systems could function without the reliable availability of a varied range of network communications capabilities. Whilst software, hardware and communications technologies have been advancing throughout the past two decades, the methods commonly used by industry for testing large scale systems which incorporate critical communications interfaces have not kept pace. This thesis argues for the need for a specifically tailored framework to achieve effective testing of communications-critical large scale systems (CCLSS). The thesis initially discusses how generic test approaches are leading to inefficient and costly test activities in industry. The thesis then presents the form and features of an alternative CCLSS domain-specific test framework, develops its ideas further into a detailed and structured test approach for one of its layers, and then provides a detailed example of how this framework can be applied using a real-life case study. The thesis concludes with a qualitative as well a simulation-based evaluation of the framework’s benefits observed during the case study and an evaluation by expert external participants considering whether similar benefits can be realised if the framework is adopted for the testing of other comparable systems. Requirements data from a second CCLSS is included in the evaluation by external participants as a second smaller case study

Simulation combined model-based testing method for train control systems

A Train Control System (TCS) is utilised to guard the operational safety of the trains in railway systems. Therefore, functional testing is applied to verify consistency between the TCS and specification requirements. Traditional functional testing in TCSs is mainly based on manually designed test cases, which is becoming unsuitable for testing increasingly complex TCSs. Therefore, Model-Based Testing (MBT) methods have been introduced into TCS functional testing, to improve the efficiency and coverage of TCS testing, with application difficulties. To overcome the difficulties of applying MBT methods to test TCSs, the author introduces simulation combined MBT which combines an MBT method with simulation. Modelling method and implementation method for the proposed approach were explained in detail. Two case studies were undertaken to explore the effectiveness of the testing platform developed. The testing results obtained prove that the testing platform can be utilised to implement the functional testing of TCSs. To prove that the MBT platform is effective in detecting errors in the SUT, validation and verification was undertaken, which include validation of specification requirements and verification of the MBT platform. The testing performance is proven to be better than existing MBT methods in terms of coverage and efficiency

Model based approach to software testing

Ohjelmistotestaus vaikeutuu päivä päivältä koska ohjelmistojen suunnittelu- ja testausvaiheet ovat selkeästi toisistaan erillisiä toimintoja, jonka johdosta testitapaukset eivät vastaa ohjelmiston oikeaa toiminnallisuutta. Eräs tapa ratkaista tämä ongelma on ottaa käyttöön malli joka kuvaa ohjelmiston toimintaa esimerkiksi käyttöliittymätasolla. Mallista voidaan mallipohjaisella testauksella tuottaa automaattisesti tarvittavat testitapaukset. Mallista voidaan tuottaa automaattisesti myös esimerkiksi lähdekoodia. Tämän työn tavoitteina oli löytää tapa, jolla mallipohjaisen lähestymistavan voisi tuoda organisaation käyttöön helposti, kehittää mallipohjainen testaustyökalu, jotta lähestymistapaa voitaisiin testata sekä todistaa että lähestymistapa tarjoaa taloudellisia hyötyjä. Työ on jaettu kahteen osaan. Ensin, kirjallisuusosassa, mallipohjainen lähestymistapa käydään läpi. Toisessa osassa esitellään ehdotus lähestymistavan tuomiseksi organisaation, käydään läpi ehdotukseen perustuvan mallipohjaisen testaustyökalun implementaatio sekä verrataan mallipohjaista testausta perinteisiin testausmenetelmiin. Tehty vertailu todisti että mallipohjainen testaus tarjoaa suuria taloudellisia hyötyjä. Tekniikkaa käyttämällä säästettiin paljon testaukseen käytettyjä miestyötunteja.Software testing is becoming more and more difficult task every day because in the current software engineering cycle the design and testing activities are separated, which leads to a situation where test cases are not in harmony with the actual application. One way to solve this problem is to take a model of the application into use, which can be, e.g., a user interface model and from which the test cases can be derived automatically. This technique is known as model based testing. The model can also be used for, e.g., automatic code generation. The main objectives of this thesis were to find a suitable way to deploy the model based approach into an organization, to develop a model based testing tool in order to test the approach and to show proof of financial benefits of the approach. The thesis is divided into two parts. First, in the literature part, the model based approach is explored thoroughly. In the second part a suggestion for the deployment of model based approach into an organization is presented, a model based testing tool implementation based on that suggestion is reviewed and a comparison between traditional testing methods and model based testing is conducted. The comparison between the testing methods showed that model based testing brings financial benefits to the organization. By using model based testing, a lot of time was saved in the test execution compared to the traditional testing methods