Ensuring interoperability between network elements in next generation networks

Next Generation Networks (NGNs), based on the Internet Protocol (IP), implement several services such as IP-based telephony and are beginning to replace the classic telephony systems. Due to the development and implementation of new powerful services these systems are becoming increasingly complex. Implementing these new services (typically software-based network elements) is often accompanied by unexpected and erratic behaviours which can manifest as interoperability problems. The reason for this caused by insufficient testing at the developing companies. The testing of such products is by nature a costly and time-consuming exercise and therefore cut down to what is considered the maximum acceptable level. Ensuring the interoperability between network elements is a known challenge. However, there exists no concept of which testing methods should be utilised to achieve an acceptable level of quality. The objective of this thesis was to improve the interoperability between network elements in NGNs by creating a testing scheme comprising of three diverse testing methods: conformance testing, interoperability testing and posthoc analysis. In the first project a novel conformance testing methodology for developing sets of conformance test cases for service specifications in NGNs was proposed. This methodology significantly improves the chance of interoperability and provides a considerable enhancement to the currently used interoperability tests. It was evaluated by successfully applying it to the Presence Service. The second report proposed a post-hoc methodology which enables the identification of the ultimate causes for interoperability problems in a NGN in daily operation. The new methods were implemented in the tool IMPACT (IP-Based Multi Protocol Posthoc Analyzer and Conformance Tester), which stores all exchanged messages between network elements in a database. Using SQL queries, the causes for errors can be found efficiently. Overall the presented testing scheme improves significantly the chance that network elements interoperate successfully by providing new methods. Beyond that, the quality of the software product is raised by mapping these methods to phases in a process model and providing well defined steps on which test method is the best suited at a certain stage

Standardized event pair based test generation method using TSS&TP

In the software engineering test development takes significant resources. A general method for the creation of appropriate test suites could solve the problems of the often ad-hoc and time-consuming test generation process. The recent method uses formal specifications to support systematic derivation of complete test suites. From the formal specification using a special procedure a formalized document, the so-called Test Suite Structure (TSS) and Test Purposes (TP) can be created. With the help of this document developers can easily, automatically implement the test suites. The TSS&TP document also enables the persons who perform the tests to understand the test criteria and the steps, even if they do not actually know the protocol itself. We present a thorough picture of our test derivation method and show its efficiency on the Wireless Transaction Protocol (WTP) of the Wireless Application Protocol family (WAP). During our work in the validation phase we also found some operational flaws in the protocol specification

Assessing and Improving Interoperability of Distributed Systems

Interoperabilität von verteilten Systemen ist eine Grundlage für die Entwicklung von neuen und innovativen Geschäftslösungen. Sie erlaubt es existierende Dienste, die auf verschiedenen Systemen angeboten werden, so miteinander zu verknüpfen, dass neue oder erweiterte Dienste zur Verfügung gestellt werden können. Außerdem kann durch diese Integration die Zuverlässigkeit von Diensten erhöht werden. Das Erreichen und Bewerten von Interoperabilität stellt jedoch eine finanzielle und zeitliche Herausforderung dar. Zur Sicherstellung und Bewertung von Interoperabilität werden systematische Methoden benötigt. Um systematisch Interoperabilität von Systemen erreichen und bewerten zu können, wurde im Rahmen der vorliegenden Arbeit ein Prozess zur Verbesserung und Beurteilung von Interoperabilität (IAI) entwickelt. Der IAI-Prozess beinhaltet drei Phasen und kann die Interoperabilität von verteilten, homogenen und auch heterogenen Systemen bewerten und verbessern. Die Bewertung erfolgt dabei durch Interoperabilitätstests, die manuell oder automatisiert ausgeführt werden können. Für die Automatisierung von Interoperabilitätstests wird eine neue Methodik vorgestellt, die einen Entwicklungsprozess für automatisierte Interoperabilitätstestsysteme beinhaltet. Die vorgestellte Methodik erleichtert die formale und systematische Bewertung der Interoperabilität von verteilten Systemen. Im Vergleich zur manuellen Prüfung von Interoperabilität gewährleistet die hier vorgestellte Methodik eine höhere Testabdeckung, eine konsistente Testdurchführung und wiederholbare Interoperabilitätstests. Die praktische Anwendbarkeit des IAI-Prozesses und der Methodik für automatisierte Interoperabilitätstests wird durch drei Fallstudien belegt. In der ersten Fallstudie werden Prozess und Methodik für Internet Protocol Multimedia Subsystem (IMS) Netzwerke instanziiert. Die Interoperabilität von IMS-Netzwerken wurde bisher nur manuell getestet. In der zweiten und dritten Fallstudie wird der IAI-Prozess zur Beurteilung und Verbesserung der Interoperabilität von Grid- und Cloud-Systemen angewendet. Die Bewertung und Verbesserung dieser Interoperabilität ist eine Herausforderung, da Grid- und Cloud-Systeme im Gegensatz zu IMS-Netzwerken heterogen sind. Im Rahmen der Fallstudien werden Möglichkeiten für Integrations- und Interoperabilitätslösungen von Grid- und Infrastructure as a Service (IaaS) Cloud-Systemen sowie von Grid- und Platform as a Service (PaaS) Cloud-Systemen aufgezeigt. Die vorgestellten Lösungen sind in der Literatur bisher nicht dokumentiert worden. Sie ermöglichen die komplementäre Nutzung von Grid- und Cloud-Systemen, eine vereinfachte Migration von Grid-Anwendungen in ein Cloud-System sowie eine effiziente Ressourcennutzung. Die Interoperabilitätslösungen werden mit Hilfe des IAI-Prozesses bewertet. Die Durchführung der Tests für Grid-IaaS-Cloud-Systeme erfolgte manuell. Die Interoperabilität von Grid-PaaS-Cloud-Systemen wird mit Hilfe der Methodik für automatisierte Interoperabilitätstests bewertet. Interoperabilitätstests und deren Beurteilung wurden bisher in der Grid- und Cloud-Community nicht diskutiert, obwohl sie eine Basis für die Entwicklung von standardisierten Schnittstellen zum Erreichen von Interoperabilität zwischen Grid- und Cloud-Systemen bieten.Achieving interoperability of distributed systems offers means for the development of new and innovative business solutions. Interoperability allows the combination of existing services provided on different systems, into new or extended services. Such an integration can also increase the reliability of the provided service. However, achieving and assessing interoperability is a technical challenge that requires high effort regarding time and costs. The reasons are manifold and include differing implementations of standards as well as the provision of proprietary interfaces. The implementations need to be engineered to be interoperable. Techniques that assess and improve interoperability systematically are required. For the assurance of reliable interoperation between systems, interoperability needs to be assessed and improved in a systematic manner. To this aim, we present the Interoperability Assessment and Improvement (IAI) process, which describes in three phases how interoperability of distributed homogeneous and heterogeneous systems can be improved and assessed systematically. The interoperability assessment is achieved by means of interoperability testing, which is typically performed manually. For the automation of interoperability test execution, we present a new methodology including a generic development process for a complete and automated interoperability test system. This methodology provides means for a formalized and systematic assessment of systems' interoperability in an automated manner. Compared to manual interoperability testing, the application of our methodology has the following benefits: wider test coverage, consistent test execution, and test repeatability. We evaluate the IAI process and the methodology for automated interoperability testing in three case studies. Within the first case study, we instantiate the IAI process and the methodology for Internet Protocol Multimedia Subsystem (IMS) networks, which were previously assessed for interoperability only in a manual manner. Within the second and third case study, we apply the IAI process to assess and improve the interoperability of grid and cloud computing systems. Their interoperability assessment and improvement is challenging, since cloud and grid systems are, in contrast to IMS networks, heterogeneous. We develop integration and interoperability solutions for grids and Infrastructure as a Service (IaaS) clouds as well as for grids and Platform as a Service (PaaS) clouds. These solutions are unique and foster complementary usage of grids and clouds, simplified migration of grid applications into the cloud, as well as efficient resource utilization. In addition, we assess the interoperability of the grid-cloud interoperability solutions. While the tests for grid-IaaS clouds are performed manually, we applied our methodology for automated interoperability testing for the assessment of interoperability to grid-PaaS cloud interoperability successfully. These interoperability assessments are unique in the grid-cloud community and provide a basis for the development of standardized interfaces improving the interoperability between grids and clouds

Executable system architecting using systems modeling language in conjunction with Colored Petri Nets - a demonstration using the GEOSS network centric system

Models and simulation furnish abstractions to manage complexities allowing engineers to visualize the proposed system and to analyze and validate system behavior before constructing it. Unified Modeling Language (UML) and its systems engineering extension, Systems Modeling Language (SysML), provide a rich set of diagrams for systems specification. However, the lack of executable semantics of such notations limits the capability of analyzing and verifying defined specifications. This research has developed an executable system architecting framework based on SysML-CPN transformation, which introduces dynamic model analysis into SysML modeling by mapping SysML notations to Colored Petri Net (CPN), a graphical language for system design, specification, simulation, and verification. A graphic user interface was also integrated into the CPN model to enhance the model-based simulation. A set of methodologies has been developed to achieve this framework. The aim is to investigate system wide properties of the proposed system, which in turn provides a basis for system reconfiguration --Abstract, page iii

Extended Finite State Machine based test generation for an OpenFlow switch

Implementations of an OpenFlow (OF) switch, a crucial Software Defined Networking (SDN) component, are prone to errors caused by developer mistakes or/and ambiguous requirements stated in the OF documents. The paper is devoted to test derivation for related OF switch implementations. A model based test generation strategy is proposed. It relies on an Extended Finite State Machine (EFSM) specification that describes the functional behaviour of the switch-to-controller communication while potential faults/misconfigurations are expressed via corresponding mutation operators. We propose a method for deriving a test suite that contains distinguishing sequences for the specification EFSM and corresponding mutants. The proposed approach is implemented as a testbed to automatically derive and execute the test suites against different versions of an OF implementation. Preliminary experimental evaluation has shown the effectiveness of the proposed approach. Further on, the derived test suites have been able to detect a number of functional inconsistencies such as erroneous responses to the Flow Mod adding rules with specific 'action' values in an available Open vSwitch implementatio

A conformance test framework for the DeviceNet fieldbus

The DeviceNet fieldbus technology is introduced and discussed. DeviceNet is an open standard fieldbus which uses the proven Controller Area Network technology. As an open standard fieldbus, the device conformance is extremely important to ensure smooth operation. The error management in DeviceNet protocol is highlighted and an error injection technique is devised to test the implementation under test for the correct error-recovery conformance. The designed Error Frame Generator prototype allows the error management and recovery of DeviceNet implementations to be conformance tested. The Error Frame Generator can also be used in other Controller Area Network based protocols. In addition, an automated Conformance Test Engine framework has been defined for realising the conformance testing of DeviceNet implementations. Automated conformance test is used to achieve consistent and reliable test results, apart from the benefits in time and personnel savings. This involves the investigations and feasibility studies in adapting the ISO 9646 conformance test standards for use in DeviceNet fieldbus. The Unique Input/Output sequences method is used for the generation of DeviceNet conformance tests. The Unique Input/Output method does not require a fully specified protocol specification and gives shorter test sequences, since only specific state information is needed. As conformance testing addresses only the protocol verification, it is foreseen that formal method validation of the DeviceNet protocol must be performed at some stage to validate the DeviceNet specification

Test Sequence Generation From Formally Verified SysML Models

est generation has been acknowledged as a cost-prone activity reducing productivity and time to market. Theexpected benefits of Model Based Systems Engineering includeautomated generation of test sequences from models. The paperproposes verification solutions for the System Modeling Lan-guage (SysML). In particular, the paper shows how to linktest generation to formal verification. The proposed algorithmsare implemented by the free software TTool. Two case studiessupport discussion on conformance and interoperability testing,respectively

Symbolic Programming of Distributed Cyber-Physical Systems

Cyber-Physical Systems (CPSs) tightly integrate physical world phenomena and cyber aspects of computational units. The composition of physical, computational and communication systems demands different levels and types of abstraction as well as novel programming methodologies allowing for homogeneous programming, knowledge representation and exchange on heterogeneous devices. Current modeling approaches, frameworks and architectures result fairly inadequate to the task, especially when resource-constrained devices are involved. This work proposes symbolic computation as an effective solution to program resource constrained CPS devices with code maintaining strict ties to high-level specifications expressed in natural language while supporting interoperability among heterogeneous devices. Design, architectural, programming, and deployment aspects of CPSs are addressed through a single formalism unifying the specification of both cyber and physical parts of CPSs. In particular, programming patterns are modeled as sequences of words adhering to natural language syntax and semantics. Given a software under test (SUT), i.e. an input program expressed as a natural language sentence, formal specifications are used to generate oracles for sentence verification and to generate input test cases. The choice of natural language inspired programming supplies a mechanism for the development of the same software on different hardware platforms, ensuring interoperability among heterogeneous devices. Formal specifications also permit to generate stress tests in order to verify that program components behave as expected in repeated execution. In order to make high-level symbolic programs run on real hardware devices with no loss of expressivity during the translation of high-level specifications into an executable implementation, this work proposes a novel software architecture, Distributed Computing for Constrained Devices (DC4CD), as a supporting platform. The proposed architecture enables symbolic processing and distributed computing on devices with very limited energy, communication and processing capabilities that can be integrated into CPSs. In particular, DC4CD has been extensively used to test the symbolic distributed programming methodology on Wireless Sensor Networks (WSNs) that include nodes with actuation abilities. The platform offers networking abstractions for the exchange of symbolic code among peer devices and allows designers to change at runtime, even wirelessly on deployed nodes, not only the application code but also system code.Cyber-Physical Systems (CPSs) tightly integrate physical world phenomena and cyber aspects of computational units. The composition of physical, computational and communication systems demands different levels and types of abstraction as well as novel programming methodologies allowing for homogeneous programming, knowledge representation and exchange on heterogeneous devices. Current modeling approaches, frameworks and architectures result fairly inadequate to the task, especially when resource-constrained devices are involved. This work proposes symbolic computation as an effective solution to program resource constrained CPS devices with code maintaining strict ties to high-level specifications expressed in natural language while supporting interoperability among heterogeneous devices. Design, architectural, programming, and deployment aspects of CPSs are addressed through a single formalism unifying the specification of both cyber and physical parts of CPSs. In particular, programming patterns are modeled as sequences of words adhering to natural language syntax and semantics. Given a software under test (SUT), i.e. an input program expressed as a natural language sentence, formal specifications are used to generate oracles for sentence verification and to generate input test cases. The choice of natural language inspired programming supplies a mechanism for the development of the same software on different hardware platforms, ensuring interoperability among heterogeneous devices. Formal specifications also permit to generate stress tests in order to verify that program components behave as expected in repeated execution. In order to make high-level symbolic programs run on real hardware devices with no loss of expressivity during the translation of high-level specifications into an executable implementation, this work proposes a novel software architecture, Distributed Computing for Constrained Devices (DC4CD), as a supporting platform. The proposed architecture enables symbolic processing and distributed computing on devices with very limited energy, communication and processing capabilities that can be integrated into CPSs. In particular, DC4CD has been extensively used to test the symbolic distributed programming methodology on Wireless Sensor Networks (WSNs) that include nodes with actuation abilities. The platform offers networking abstractions for the exchange of symbolic code among peer devices and allows designers to change at runtime, even wirelessly on deployed nodes, not only the application code but also system code