Performance Testing of Distributed Component Architectures

Performance characteristics, such as response time, throughput andscalability, are key quality attributes of distributed applications. Current practice,however, rarely applies systematic techniques to evaluate performance characteristics.We argue that evaluation of performance is particularly crucial in early developmentstages, when important architectural choices are made. At first glance, thiscontradicts the use of testing techniques, which are usually applied towards the endof a project. In this chapter, we assume that many distributed systems are builtwith middleware technologies, such as the Java 2 Enterprise Edition (J2EE) or theCommon Object Request Broker Architecture (CORBA). These provide servicesand facilities whose implementations are available when architectures are defined.We also note that it is the middleware functionality, such as transaction and persistenceservices, remote communication primitives and threading policy primitives,that dominates distributed system performance. Drawing on these observations, thischapter presents a novel approach to performance testing of distributed applications.We propose to derive application-specific test cases from architecture designs so thatthe performance of a distributed application can be tested based on the middlewaresoftware at early stages of a development process. We report empirical results thatsupport the viability of the approach

The Early Assessment of System Performance in Distributed Real-time Systems

Distributed real-time process control systems are notoriously difficult to develop. They frequently overrun time schedules and break cost constraints. The problems are compounded where there are multiple development teams and stakeholders. Conventional model-driven development has been examined to see if it can be extended to resolve some of these problems. It may be possible to use early system design stages to identify performance issues which would otherwise not be identified until late in the development of the system. A functional model is proposed, in addition to those conventionally used for model-driven development, based on loosely coupled functional elements, to represent the behaviour of each system component. The model complements existing requirements and design specifications and addresses the combination of individual component abstractions to produce a complete system specification. The functional model enables the accurate prediction of system performance prior to the detailed design of each component. The thesis examines how performance can be calculated and modelled. An animator tool and associated code generator are used to predict system and component performance in a distributed aircraft navigation system. The use of the animator to support the system design prior to the generation of the component contract specifications and interface control documents provides a means of assessing performance which is accessible to domain experts and system designers alike. The model also enables the effects of requirements changes and component design issues on the system design to be assessed in terms of the system design to provide system wide solutions. This performance assessment model and animator compliments the existing 'fix-it-later' approach, reducing the chances of performance failure detected late during the system development process when they are most expensive to fix

Development of a Security Methodology for Cooperative Information Systems: The CooPSIS Project

Since networks and computing systems are vital components of today\u27s life, it is of utmost importance to endow them with the capability to survive physical and logical faults, as well as malicious or deliberate attacks. When the information system is obtained by federating pre-existing local systems, a methodology is needed to integrate security policies and mechanisms under a uniform structure. Therefore, in building distributed information systems, a methodology for analysis, design and implementation of security requirements of data and processes is essential for obtaining mutual trust between cooperating organizations. Moreover, when the information system is built as a cooperative set of e-services, security is related to the type of data, to the sensitivity context of the cooperative processes and to the security characteristics of the communication paradigms. The CoopSIS (Cooperative Secure Information Systems) project aims to develop methods and tools for the analysis, design, implementation and evaluation of secure and survivable distributed information systems of cooperative type, in particular with experimentation in the Public Administration Domain. This paper presents the basic issues of a methodology being conceived to build a trusted cooperative environment, where data sensitivity parameters and security requirements of processes are taken into account. The milestones phases of the security development methodology in the context of this project are illustrated

Independent verification of specification models for large software systems at the early phases of development lifecycle

One of the major challenges facing the software industry, in general and IV&V (Independent Verification and Validation) analysts in particular, is to find ways for analyzing dynamic behavior of requirement specifications of large software systems early in the development lifecycle. Such analysis can significantly improve the performance and reliability of the developed systems. This dissertation addresses the problem of developing an IV&V framework for extracting semantics of dynamic behavior from requirement specifications based on: (1) SART (Structured Analysis with Realtime) models, and (2) UML (Unified Modeling Language) models.;For SART, the framework presented here shows a direct mapping from SART specification models to CPN (Colored Petrinets) models. The semantics of the SART hierarchy at the individual levels are preserved in the mapping. This makes it easy for the analyst to perform the analysis and trace back to the corresponding SART model. CPN was selected because it supports rigorous dynamic analysis. A large scale case study based on a component of NASA EOS system was performed for a proof of the concept.;For UML specifications, an approach based on metamodels is presented. A special type of metamodel, called dynamic metamodel (DMM), is introduced. This approach holds several advantages over the direct mapping of UML to CPN. The mapping rules for generating DMM are not CPN specific, hence they would not change if a language other than CPN is used. Also it makes it more flexible to develop DMM because other types of models can be added to the existing UML models. A simple example of a pacemaker is used to illustrate the concepts of DMM

A Survey on Trust Metrics for Autonomous Robotic Systems

This paper surveys the area of Trust Metrics related to security for autonomous robotic systems. As the robotics industry undergoes a transformation from programmed, task oriented, systems to Artificial Intelligence-enabled learning, these autonomous systems become vulnerable to several security risks, making a security assessment of these systems of critical importance. Therefore, our focus is on a holistic approach for assessing system trust which requires incorporating system, hardware, software, cognitive robustness, and supplier level trust metrics into a unified model of trust. We set out to determine if there were already trust metrics that defined such a holistic system approach. While there are extensive writings related to various aspects of robotic systems such as, risk management, safety, security assurance and so on, each source only covered subsets of an overall system and did not consistently incorporate the relevant costs in their metrics. This paper attempts to put this prior work into perspective, and to show how it might be extended to develop useful system-level trust metrics for evaluating complex robotic (and other) systems

Architectural Analysis of Systems Based on the Publisher-Subscriber Style

Architectural styles impose constraints on both the topology and the interaction behavior of involved parties. In this paper, we propose an approach for analyzing implemented systems based on the publisher-subscriber architectural style. From the style definition, we derive a set of reusable questions and show that some of them can be answered statically whereas others are best answered using dynamic analysis. The paper explains how the results of static analysis can be used to orchestrate dynamic analysis. The proposed method was successfully applied on the NASA's Goddard Mission Services Evolution Center (GMSEC) software product line. The results show that the GMSEC has a) a novel reusable vendor-independent middleware abstraction layer that allows the NASA's missions to configure the middleware of interest without changing the publishers' or subscribers' source code, and b) some high priority bugs due to behavioral discrepancies, which were eluded during testing and code reviews, among different implementations of the same APIs for different vendors

Software Safety Progress in NASA

NASA has developed guidelines for development and analysis of safety-critical software. These guidelines have been documented in a Guidebook for Safety Critical Software Development and Analysis. The guidelines represent a practical 'how to' approach, to assist software developers and safety analysts in cost effective methods for software safety. They provide guidance in the implementation of the recent NASA Software Safety Standard NSS-1740.13 which was released as 'Interim' version in June 1994, scheduled for formal adoption late 1995. This paper is a survey of the methods in general use, resulting in the NASA guidelines for safety critical software development and analysis