Improved metrics collection and correlation for the CERN cloud storage test framework

Storage space is one of the most important ingredients that the European Organization for Nuclear Research (CERN) needs for its experiments and operation. Part of the Data & Storage Services (IT-DSS) group’s work at CERN is focused on testing and evaluating the cloud storage system that is provided by the openlab partner Huawei, Huawei Universal Disk Storage System (UDS). As a whole, the system consists of both software and hardware. The objective of the Huawei-CERN partnership is to investigate the performance of the cloud storage system. Among the interesting questions are the system’s scalability, reliability and ability to store and retrieve files. During the tests, possible bugs and malfunctions can be discovered and corrected. Different versions of the storage software that runs inside the storage system can also be compared to each other. The nature of testing and benchmarking a storage system gives rise to several small tasks that can be done during a short summer internship. In order to test the storage system a test framework developed by the DSS group is used. The framework consists of various types of file transfer tests, client and server monitoring programs and log file analysis programs. Part of the work done was additions to the existing framework and part was developing new tools. Metrics collection was the central theme. Metrics are to be understood as system statistics, such as memory consumption or processor usage. Memory usage and disk reads/writes were added to the existing client real-time monitoring framework. CPU and memory usage, network traffic (bytes received/sent) and the number of processes running are collected from a client computer before and after a daily test. Two other additions are visualization for storage system log files, as well as a new monitoring tool for the storage system. This report is divided into parts describing each part of the framework that was improved or added, the problem and the final solution. A short description of the code and the architecture are also included

Mixed-reality Automotive Testing with SENSORIS

Highly automated and autonomous vehicles become more and more widespread changing the classical way of testing and validation. Traditionally, the automotive industry has pursued testing rather in real-world or in pure virtual simulation environments. As a new possibility, mixed-reality testing has also appeared enabling an efficient combination of real and simulated elements of testing. Furthermore, vehicles from different OEMs will have a common interface to communicate with a test system. The paper presents a mixed-reality test framework for visualizing perception sensor feeds real-time in the Unity 3D game engine. Thereby, the digital twin of the tested vehicle and its environment are realized in the simulation. The communication between the sensors of the tested vehicle and the central computer running the test is realized via the standard SENSORIS interface. The paper outlines the hardware and software requirements towards such a system in detail. To show the viability of the system a vehicle in the loop test has been carried out

NASA Data Acquisition System Software Development for Rocket Propulsion Test Facilities

Current NASA propulsion test facilities include Stennis Space Center in Mississippi, Marshall Space Flight Center in Alabama, Plum Brook Station in Ohio, and White Sands Test Facility in New Mexico. Within and across these centers, a diverse set of data acquisition systems exist with different hardware and software platforms. The NASA Data Acquisition System (NDAS) is a software suite designed to operate and control many critical aspects of rocket engine testing. The software suite combines real-time data visualization, data recording to a variety formats, short-term and long-term acquisition system calibration capabilities, test stand configuration control, and a variety of data post-processing capabilities. Additionally, data stream conversion functions exist to translate test facility data streams to and from downstream systems, including engine customer systems. The primary design goals for NDAS are flexibility, extensibility, and modularity. Providing a common user interface for a variety of hardware platforms helps drive consistency and error reduction during testing. In addition, with an understanding that test facilities have different requirements and setups, the software is designed to be modular. One engine program may require real-time displays and data recording; others may require more complex data stream conversion, measurement filtering, or test stand configuration management. The NDAS suite allows test facilities to choose which components to use based on their specific needs. The NDAS code is primarily written in LabVIEW, a graphical, data-flow driven language. Although LabVIEW is a general-purpose programming language; large-scale software development in the language is relatively rare compared to more commonly used languages. The NDAS software suite also makes extensive use of a new, advanced development framework called the Actor Framework. The Actor Framework provides a level of code reuse and extensibility that has previously been difficult to achieve using LabVIEW. Th

Arcadia, a software development environment research project

The research objectives of the Arcadia project are two-fold: discovery and development of environment architecture principles and creation of novel software development tools, particularly powerful analysis tools, which will function within an environment built upon these architectural principles.Work in the architecture area is concerned with providing the framework to support integration while also supporting the often conflicting goal of extensibility. Thus, this area of research is directed toward achieving external integration by providing a consistent, uniform user interface, while still admitting customization and addition of new tools and interface functions. In an effort to also attain internal integration, research is aimed at developing mechanisms for structuring and managing the tools and data objects that populate a software development environment, while facilitating the insertion of new kinds of tools and new classes of objects.The unifying theme of work in the tools area is support for effective analysis at every stage of a software development project. Research is directed toward tools suitable for analyzing pre-implementation descriptions of software, software itself, and towards the production of testing and debugging tools. In many cases, these tools are specifically tailored for applicability to concurrent, distributed, or real-time software systems.The initial focus of Arcadia research is on creating a prototype environment, embodying the architectural principles, which supports Ada1 software development. This prototype environment is itself being developed in Ada.Arcadia is being developed by a consortium of researchers from the University of California at Irvine, the University of Colorado at Boulder, the University of Massachusetts at Amherst, TRW, Incremental Systems Corporation, and The Aerospace Corporation. This paper delineates the research objectives and describes the approaches being taken, the organization of the research endeavor, and current status of the work

Geographically distributed hybrid testing & collaboration between geotechnical centrifuge and structures laboratories

© 2018, Institute of Engineering Mechanics (IEM). All rights reserved. Distributed Hybrid Testing (DHT) is an experimental technique designed to capitalise on advances in modern networking infrastructure to overcome traditional laboratory capacity limitations. By coupling the heterogeneous test apparatus and computational resources of geographically distributed laboratories, DHT provides the means to take on complex, multi-disciplinary challenges with new forms of communication and collaboration. To introduce the opportunity and practicability afforded by DHT, here an exemplar multi-site test is addressed in which a dedicated fibre network and suite of custom software is used to connect the geotechnical centrifuge at the University of Cambridge with a variety of structural dynamics loading apparatus at the University of Oxford and the University of Bristol. While centrifuge time-scaling prevents real-time rates of loading in this test, such experiments may be used to gain valuable insights into physical phenomena, test procedure and accuracy. These and other related experiments have led to the development of the real-time DHT technique and the creation of a flexible framework that aims to facilitate future distributed tests within the UK and beyond. As a further example, a real-time DHT experiment between structural labs using this framework for testing across the Internet is also presented

An architecture for embedded system communication

Time is a major constraint in the development of most embedded systems. In many cases, the development of embedded software is directly dependent on the development of the embedded systems. This calls for a development framework that enables embedded software and hardware to be developed in parallel. In an attempt to solve the problem, a concept prototype hardware-in-the-loop (HIL) simulation methodology has been proposed and implemented at the Ohio State University for the TMS320LF2407A DSP board. We build on top of that HIL system by rewriting the low level device drivers that allow data and control information to be set simultaneously, thus, creating a software abstraction layer over various devices available on the DSP board. The device drivers allow data access at the processor and the pin level for the devices on the DSP board. This abstraction simulates external devices in a transparent manner using a device driver library that provides the same programming interface to the device simulators as to real devices. Also, it allows for the testing of both real and simulated hardware connected to the DSP board as a part of the embedded system. The main advantages of the framework are rapid prototyping, unit testing and monitoring. We also modify the existing serial line protocol and perform a comparison between the new and the existing protocol and show that the new protocol is efficient for large data transport. This protocol allows for the effective utilization of serial line bandwidth when the DSP board is used for signal processing or voice based applications. We present the virtual testbed as a software development tool. We conclude by exploring the future directions for the applications

Design of a graphical framework for simple prototyping of pluvial flooding cellular automata algorithms

CCWI 2011: Computing and Control for the Water Industry, 5-7 September 2011, University of Exeter, UKCellular automata (CA) algorithms can be used for quickly describing models of complex systems using simple rules. CADDIES is a new EPSRC and industry-sponsored project that aims to use the computational speed of CA algorithms to produce operationally useful real/near-real time pluvial urban flood models for both 1D-sewer and 2D-surface (dual-drainage) flows. In this paper, the design of a graphical software framework for the CADDIES project is presented. This is intended to simplify the development, testing and use of CA algorithms, and to facilitate the handling of the peripheral tasks of data management and display; allowing the research users to focus on the central tasks of optimisation of CA models and algorithms themselves

Identifying Defects Related to the Order in which Messages are Received in Message-Passing Systems

Improving the quality of software artifacts and products is an essential activity for everyone working on the development of software. Testing is one approach to reveal defects and faults in software. In recent years, message-passing systems have grown to a significant degree due to the rise of distributed systems, embedded systems, and so forth. In message-passing systems, components communicate with each other through sending and receiving messages. This message-passing mechanism introduces new opportunities for testing programs due to the fact that the time a message is delivered is not guaranteed, so the order in which messages are delivered is also not guaranteed. This non-determinism introduces interleaving and parallelization and subsequently a new source of software defects like race conditions. In this thesis, we have explained a new approach to testing a given component for identifying software faults related to the order in which messages are received by that component. We reorder messages coming to a certain component and deliver them in a different distinct ordering each time. We have three different methods for achieving message reordering: Blocking, Buffering, and Adaptive Buffering. We evaluate the effectiveness of our new testing methods using four metrics: Ordering Coverage, Coverage Rate, Slowdown Overhead, and Memory Overhead. We have implemented our Reordering Framework on QNX Neutrino 6.5.0 and compared our reordering methods with each other and with the naive random case using our experiments. We have also showed that our testing approach applies to real programs and can reveal real bugs in software

Development and Evaluation of a Real-Time Framework for a Portable Assistive Hearing Device

Testing and verification of digital hearing aid devices, and the embedded software and algorithms can prove to be a challenging task especially taking into account time-to-market considerations. This thesis describes a PC based, real-time, highly configurable framework for the evaluation of audio algorithms. Implementation of audio processing algorithms on such a platform can provide hearing aid designers and manufacturers the ability to test new and existing processing techniques and collect data about their performance in real-life situations, and without the need to develop a prototype device. The platform is based on the Eurotech Catalyst development kit and the Fedora Linux OS, and it utilizes the JACK audio engine to facilitate reliable real-time performance Additionally, we demonstrate the capabilities of this platform by implementing an audio processing chain targeted at improving speech intelligibility for people suffering from auditory neuropathy. Evaluation is performed for both noisy and noise-free environments. Subjective evaluation of the results, using normal hearing listeners and an auditory neuropathy simulator, demonstrates improvement in some conditions

Modular and Safe Event-Driven Programming

Asynchronous event-driven systems are ubiquitous across domains such as device drivers, distributed systems, and robotics. These systems are notoriously hard to get right as the programmer needs to reason about numerous control paths resulting from the complex interleaving of events (or messages) and failures. Unsurprisingly, it is easy to introduce subtle errors while attempting to fill in gaps between high-level system specifications and their concrete implementations.This dissertation proposes new methods for programming safe event-driven asynchronous systems.In the first part of the thesis, we present ModP, a modular programming framework for compositional programming and testing of event-driven asynchronous systems.The ModP module system supports a novel theory of compositional refinement for assume-guarantee reasoning of dynamic event-driven asynchronous systems. We build a complex distributed systems software stack using ModP.Our results demonstrate that compositional reasoning can help scale model-checking (both explicit and symbolic) to large distributed systems.ModP is transforming the way asynchronous software is built at Microsoft and Amazon Web Services (AWS). Microsoft uses ModP for implementing safe device drivers and other software in the Windows kernel.AWS uses ModP for compositional model checking of complex distributed systems. While ModP simplifies analysis of such systems, the state space of industrial-scale systems remains extremely large.In the second part of this thesis, we present scalable verification and systematic testing approaches to further mitigate this state-space explosion problem.First, we introduce the concept of a delaying explorer to perform prioritized exploration of the behaviors of an asynchronous reactive program. A delaying explorer stratifies the search space using a custom strategy (tailored towards finding bugs faster), and a delay operation that allows deviation from that strategy. We show that prioritized search with a delaying explorer performs significantly better than existing approaches for finding bugs in asynchronous programs.Next, we consider the challenge of verifying time-synchronized systems; these are almost-synchronous systems as they are neither completely asynchronous nor synchronous.We introduce approximate synchrony, a sound and tunable abstraction for verification of almost-synchronous systems. We show how approximate synchrony can be used for verification of both time-synchronization protocols and applications running on top of them.Moreover, we show how approximate synchrony also provides a useful strategy to guide state-space exploration during model-checking.Using approximate synchrony and implementing it as a delaying explorer, we were able to verify the correctness of the IEEE 1588 distributed time-synchronization protocol and, in the process, uncovered a bug in the protocol that was well appreciated by the standards committee.In the final part of this thesis, we consider the challenge of programming a special class of event-driven asynchronous systems -- safe autonomous robotics systems.Our approach towards achieving assured autonomy for robotics systems consists of two parts: (1) a high-level programming language for implementing and validating the reactive robotics software stack; and (2) an integrated runtime assurance system to ensure that the assumptions used during design-time validation of the high-level software hold at runtime.Combining high-level programming language and model-checking with runtime assurance helps us bridge the gap between design-time software validation that makes assumptions about the untrusted components (e.g., low-level controllers), and the physical world, and the actual execution of the software on a real robotic platform in the physical world. We implemented our approach as DRONA, a programming framework for building safe robotics systems.We used DRONA for building a distributed mobile robotics system and deployed it on real drone platforms. Our results demonstrate that DRONA (with the runtime-assurance capabilities) enables programmers to build an autonomous robotics software stack with formal safety guarantees.To summarize, this thesis contributes new theory and tools to the areas of programming languages, verification, systematic testing, and runtime assurance for programming safe asynchronous event-driven across the domains of fault-tolerant distributed systems and safe autonomous robotics systems