On the Experimental Evaluation of Vehicular Networks: Issues, Requirements and Methodology Applied to a Real Use Case

One of the most challenging fields in vehicular communications has been the experimental assessment of protocols and novel technologies. Researchers usually tend to simulate vehicular scenarios and/or partially validate new contributions in the area by using constrained testbeds and carrying out minor tests. In this line, the present work reviews the issues that pioneers in the area of vehicular communications and, in general, in telematics, have to deal with if they want to perform a good evaluation campaign by real testing. The key needs for a good experimental evaluation is the use of proper software tools for gathering testing data, post-processing and generating relevant figures of merit and, finally, properly showing the most important results. For this reason, a key contribution of this paper is the presentation of an evaluation environment called AnaVANET, which covers the previous needs. By using this tool and presenting a reference case of study, a generic testing methodology is described and applied. This way, the usage of the IPv6 protocol over a vehicle-to-vehicle routing protocol, and supporting IETF-based network mobility, is tested at the same time the main features of the AnaVANET system are presented. This work contributes in laying the foundations for a proper experimental evaluation of vehicular networks and will be useful for many researchers in the area.Comment: in EAI Endorsed Transactions on Industrial Networks and Intelligent Systems, 201

Performance testing of distributed computational resources in the software development phase

A grid software harmonization is possible through adoption of standards i.e. common protocols and interfaces. In the development phase of standard implementation, the performance testing of grid subsystems can detect hidden software issues which are not detectable using other testing procedures. A simple software solution was proposed which consists of a communication layer, resource consumption agents hosted in computational resources (clients or servers), a database of the performance results and a web interface to visualize the results. Communication between agents, monitoring the resources and main control Python script (supervisor) is possible through the communication layer based on the secure XML-RPC protocol. The resource monitoring agent is a key element of performance testing which provides information about all monitored processes including their child processes. The agent is a simple Python script based on the Python psutil library. The second agent, provided after the resource monitored phase, records data from the resources in the central MySQL database. The results can be queried and visualized using a web interface. The database and data visualization scripts could be considered for a service thus the testers do not need install them to run own tests

Interoperability-Guided Testing of QUIC Implementations using Symbolic Execution

The main reason for the standardization of network protocols, like QUIC, is to ensure interoperability between implementations, which poses a challenging task. Manual tests are currently used to test the different existing implementations for interoperability, but given the complex nature of network protocols, it is hard to cover all possible edge cases. State-of-the-art automated software testing techniques, such as Symbolic Execution (SymEx), have proven themselves capable of analyzing complex real-world software and finding hard to detect bugs. We present a SymEx-based method for finding interoperability issues in QUIC implementations, and explore its merit in a case study that analyzes the interoperability of picoquic and QUANT. We find that, while SymEx is able to analyze deep interactions between different implementations and uncovers several bugs, in order to enable efficient interoperability testing, implementations need to provide additional information about their current protocol state.Comment: 6 page

Enhancements to Secure Bootstrapping of Smart Appliances

In recent times, there has been a proliferation of smart IoT devices that make our everyday life more convenient, both at home and at work environment. Most of these smart devices are connected to cloud-based online services, and they typically reuse the existing Wi-Fi network infrastructure for Internet connectivity. Hence, it is of paramount importance to ensure that these devices establish a robust security association with the Wi-Fi networks and cloud-based servers. The initial process by which a device establishes a robust security association with the network and servers is known as secure bootstrapping. The bootstrapping process results in the derivation of security keys and other connection parameters required by the security associations. Since the smart IoT devices often possess minimal user-interface, there is a need for bootstrapping methods with which the users can effortlessly connect their smart IoT devices to the networks and services. Nimble out-of-band authentication for Extensible Authentication Protocol (EAP-NOOB) is one such secure bootstrapping method. It is a new EAP authentication method for IEEE 802.1X/EAP authentication framework. The protocol does not assume or require any pre-configured authentication credentials such as symmetric keys or certificates. In lieu, the authentication credentials along with the user’s ownership of the device are established during the bootstrapping process. The primary goal of this thesis is to study and implement the draft specification of the EAP-NOOB protocol in order to evaluate the working of EAP-NOOB in real-world scenarios. During our implementation and testing of the initial prototype for EAP-NOOB, we discovered several issues in the protocol. In this thesis, we propose a suitable solution for each of the problems identified and also, verify the solutions through implementation and testing. The main results of this thesis work are various enhancements and clarifications to the EAP-NOOB protocol specification. The results consequently aid the standardisation of the protocol at IETF. We also design and implement several additional features for EAP-NOOB to enhance the user experience

Experimental Security Analysis of Connected Pacemakers

Medical devices and their connectivity capabilities are providing a variety of benefits to the healthcare domain, including remote monitoring, automated alerts, and improved patient outcomes. However, these medical devices introduce a range of new potential cyber security risks when connected to the Internet, affecting the patient or the healthcare infrastructure. In this paper, we systematically analyze the security issues of connected pacemakers. In particular, we use a black box testing methodology against a commercial pacemaker device and the network infrastructure. Our main objective is to understand how the data is sent from a bedside monitor in the patient’s home to the backend server hosted by the pacemaker manufacturer, and whether or not this data is protected from a cyber security perspective. To do so, we leveraged several hardware related vulnerabilities found in the bedside monitor to obtain the firmware of the device and then reverse engineered the proprietary communication protocol. We demonstrate how vulnerabilities in this protocol can be leveraged to allow an attacker to perform a man-in-the-middle attack on the pacemaker.publishedVersio

The STRESS Method for Boundary-point Performance Analysis of End-to-end Multicast Timer-Suppression Mechanisms

Evaluation of Internet protocols usually uses random scenarios or scenarios based on designers' intuition. Such approach may be useful for average-case analysis but does not cover boundary-point (worst or best-case) scenarios. To synthesize boundary-point scenarios a more systematic approach is needed.In this paper, we present a method for automatic synthesis of worst and best case scenarios for protocol boundary-point evaluation. Our method uses a fault-oriented test generation (FOTG) algorithm for searching the protocol and system state space to synthesize these scenarios. The algorithm is based on a global finite state machine (FSM) model. We extend the algorithm with timing semantics to handle end-to-end delays and address performance criteria. We introduce the notion of a virtual LAN to represent delays of the underlying multicast distribution tree. The algorithms used in our method utilize implicit backward search using branch and bound techniques and start from given target events. This aims to reduce the search complexity drastically. As a case study, we use our method to evaluate variants of the timer suppression mechanism, used in various multicast protocols, with respect to two performance criteria: overhead of response messages and response time. Simulation results for reliable multicast protocols show that our method provides a scalable way for synthesizing worst-case scenarios automatically. Results obtained using stress scenarios differ dramatically from those obtained through average-case analyses. We hope for our method to serve as a model for applying systematic scenario generation to other multicast protocols.Comment: 24 pages, 10 figures, IEEE/ACM Transactions on Networking (ToN) [To appear

A Study of Concurrency Bugs and Advanced Development Support for Actor-based Programs

The actor model is an attractive foundation for developing concurrent applications because actors are isolated concurrent entities that communicate through asynchronous messages and do not share state. Thereby, they avoid concurrency bugs such as data races, but are not immune to concurrency bugs in general. This study taxonomizes concurrency bugs in actor-based programs reported in literature. Furthermore, it analyzes the bugs to identify the patterns causing them as well as their observable behavior. Based on this taxonomy, we further analyze the literature and find that current approaches to static analysis and testing focus on communication deadlocks and message protocol violations. However, they do not provide solutions to identify livelocks and behavioral deadlocks. The insights obtained in this study can be used to improve debugging support for actor-based programs with new debugging techniques to identify the root cause of complex concurrency bugs.Comment: - Submitted for review - Removed section 6 "Research Roadmap for Debuggers", its content was summarized in the Future Work section - Added references for section 1, section 3, section 4.3 and section 5.1 - Updated citation

Towards the Usage of MBT at ETSI

In 2012 the Specialists Task Force (STF) 442 appointed by the European Telcommunication Standards Institute (ETSI) explored the possibilities of using Model Based Testing (MBT) for test development in standardization. STF 442 performed two case studies and developed an MBT-methodology for ETSI. The case studies were based on the ETSI-standards GeoNetworking protocol (ETSI TS 102 636) and the Diameter-based Rx protocol (ETSI TS 129 214). Models have been developed for parts of both standards and four different MBT-tools have been employed for generating test cases from the models. The case studies were successful in the sense that all the tools were able to produce the test suites having the same test adequacy as the corresponding manually developed conformance test suites. The MBT-methodology developed by STF 442 is based on the experiences with the case studies. It focusses on integrating MBT into the sophisticated standardization process at ETSI. This paper summarizes the results of the STF 442 work.Comment: In Proceedings MBT 2013, arXiv:1303.037