ERIGrid Holistic Test Description for Validating Cyber-Physical Energy Systems

Smart energy solutions aim to modify and optimise the operation of existing energy infrastructure. Such cyber-physical technology must be mature before deployment to the actual infrastructure, and competitive solutions will have to be compliant to standards still under development. Achieving this technology readiness and harmonisation requires reproducible experiments and appropriately realistic testing environments. Such testbeds for multi-domain cyber-physical experiments are complex in and of themselves. This work addresses a method for the scoping and design of experiments where both testbed and solution each require detailed expertise. This empirical work first revisited present test description approaches, developed a newdescription method for cyber-physical energy systems testing, and matured it by means of user involvement. The new Holistic Test Description (HTD) method facilitates the conception, deconstruction and reproduction of complex experimental designs in the domains of cyber-physical energy systems. This work develops the background and motivation, offers a guideline and examples to the proposed approach, and summarises experience from three years of its application.This work received funding in the European Community’s Horizon 2020 Program (H2020/2014–2020) under project “ERIGrid” (Grant Agreement No. 654113)

A heuristic-based approach to code-smell detection

Encapsulation and data hiding are central tenets of the object oriented paradigm. Deciding what data and behaviour to form into a class and where to draw the line between its public and private details can make the difference between a class that is an understandable, flexible and reusable abstraction and one which is not. This decision is a difficult one and may easily result in poor encapsulation which can then have serious implications for a number of system qualities. It is often hard to identify such encapsulation problems within large software systems until they cause a maintenance problem (which is usually too late) and attempting to perform such analysis manually can also be tedious and error prone. Two of the common encapsulation problems that can arise as a consequence of this decomposition process are data classes and god classes. Typically, these two problems occur together – data classes are lacking in functionality that has typically been sucked into an over-complicated and domineering god class. This paper describes the architecture of a tool which automatically detects data and god classes that has been developed as a plug-in for the Eclipse IDE. The technique has been evaluated in a controlled study on two large open source systems which compare the tool results to similar work by Marinescu, who employs a metrics-based approach to detecting such features. The study provides some valuable insights into the strengths and weaknesses of the two approache

CLOCIS:Cloud-based conformance testing framework for IoT devices in the future internet

In recent years, the Internet of Things (IoT) has not only become ubiquitous in daily life but has also emerged as a pivotal technology across various sectors, including smart factories and smart cities. Consequently, there is a pressing need to ensure the consistent and uninterrupted delivery of IoT services. Conformance testing has thus become an integral aspect of IoT technologies. However, traditional methods of IoT conformance testing fall short of addressing the evolving requirements put forth by both industry and academia. Historically, IoT testing has necessitated a visit to a testing laboratory, implying that both the testing systems and testers must be co-located. Furthermore, there is a notable absence of a comprehensive method for testing an array of IoT standards, especially given their inherent heterogeneity. With a surge in the development of diverse IoT standards, crafting an appropriate testing environment poses challenges. To address these concerns, this article introduces a method for remote IoT conformance testing, underpinned by a novel conceptual architecture termed CLOCIS. This architecture encompasses an extensible approach tailored for a myriad of IoT standards. Moreover, we elucidate the methods and procedures integral to testing IoT devices. CLOCIS, predicated on this conceptual framework, is actualized, and to attest to its viability, we undertake IoT conformance testing and present the results. When leveraging CLOCIS, small and medium-sized enterprises (SMEs) and entities in the throes of IoT service development stand to benefit from a reduced time to market and cost-efficient testing procedures. Additionally, this innovation holds promise for IoT standardization communities, enabling them to champion their standards with renewed vigor

7e Nederlandse testdag, Eindhoven, 8 November 2001 : proceedings

These are the proceedings of the seventh edition of the Nederlandse Testdag (a.k.a. Dutch Testing Day), held on November 8, 2001 in Eindhoven, The Netherlands. The increase in the complexity of software and hardware systems was the predominant concern in the software design of the last decades. This increase is still going on today. and mastering this complexity is possible, only by investigating, discussing and evaluating methods and techniques for testing such systems. The Nederlandse Testdag serves as a forum in which researchers from the industry and the academia discuss and present their latest experiences and theories in the area of testing. The initiative for organising the Nederlandse Testdag is, and has always been, the result of the combined efforts of the Dutch academia and the industry. The Nederlandse Testdag is an annual event which was first held in 1995. This year's edition again consists of one invited presentation by Jens Grabowski, on ITCN-3. and six regular presentations, both from the academia and from the industry. The presentations capture a broad field of the entire testing spectrum. In the presentation by Martin Gijsen (CMG), test automation for Graphical User Interface (GUI), dedicated and embedded systems according to the TestFrame methodology is explained. Klaas Mateboer (Collis) presents the test-tool Conclusion. René de Vries (University of Twente) reports on specification testing in practice and illustrates this by means of an example. In the presentation by Loe Feijs (Eindhoven University of Technology), testing is related to game-theory. Marcel Verhoef (Chess) and Bertil Oving (NLR) present their experiences using real-time simulation, UML and VDM to obtain more reliable spacecraft avionics. Finally, Ben van Buitenen (Baan), provides an insight in service pack testing: how to efficiently test customised software components and packages. The organisation of the Nederlandse Testdag is grateful for the sponsorship it received from the Eindhoven University of Technology, the Eindhoven Embedded Systems Institute, and the financial support from Dutch Research School IPA. We are very much indebted to CMG and Telelogic's willingness to sponsor this event financially. Over the years, both companies have profiled themselves as companies investing both time and resources in advancing the current state in testing. Finally, the organisation thanks Marcella de Rooij and EIize Russell for their organisational assistance

7e Nederlandse testdag, Eindhoven, 8 November 2001 : proceedings

These are the proceedings of the seventh edition of the Nederlandse Testdag (a.k.a. Dutch Testing Day), held on November 8, 2001 in Eindhoven, The Netherlands. The increase in the complexity of software and hardware systems was the predominant concern in the software design of the last decades. This increase is still going on today. and mastering this complexity is possible, only by investigating, discussing and evaluating methods and techniques for testing such systems. The Nederlandse Testdag serves as a forum in which researchers from the industry and the academia discuss and present their latest experiences and theories in the area of testing. The initiative for organising the Nederlandse Testdag is, and has always been, the result of the combined efforts of the Dutch academia and the industry. The Nederlandse Testdag is an annual event which was first held in 1995. This year's edition again consists of one invited presentation by Jens Grabowski, on ITCN-3. and six regular presentations, both from the academia and from the industry. The presentations capture a broad field of the entire testing spectrum. In the presentation by Martin Gijsen (CMG), test automation for Graphical User Interface (GUI), dedicated and embedded systems according to the TestFrame methodology is explained. Klaas Mateboer (Collis) presents the test-tool Conclusion. René de Vries (University of Twente) reports on specification testing in practice and illustrates this by means of an example. In the presentation by Loe Feijs (Eindhoven University of Technology), testing is related to game-theory. Marcel Verhoef (Chess) and Bertil Oving (NLR) present their experiences using real-time simulation, UML and VDM to obtain more reliable spacecraft avionics. Finally, Ben van Buitenen (Baan), provides an insight in service pack testing: how to efficiently test customised software components and packages. The organisation of the Nederlandse Testdag is grateful for the sponsorship it received from the Eindhoven University of Technology, the Eindhoven Embedded Systems Institute, and the financial support from Dutch Research School IPA. We are very much indebted to CMG and Telelogic's willingness to sponsor this event financially. Over the years, both companies have profiled themselves as companies investing both time and resources in advancing the current state in testing. Finally, the organisation thanks Marcella de Rooij and EIize Russell for their organisational assistance

Enabling IoT in Manufacturing: from device to the cloud

Industrial automation platforms are experiencing a paradigm shift. With the new technol-ogies and strategies that are being applied to enable a synchronization of the digital and real world, including real-time access to sensorial information and advanced networking capabilities to actively cooperate and form a nervous system within the enterprise, the amount of data that can be collected from real world and processed at digital level is growing at an exponential rate. Indeed, in modern industry, a huge amount of data is coming through sensorial networks em-bedded in the production line, allowing to manage the production in real-time. This dissertation proposes a data collection framework for continuously collecting data from the device to the cloud, enabling resources at manufacturing industries shop floors to be handled seamlessly. The framework envisions to provide a robust solution that besides collecting, transforming and man-aging data through an IoT model, facilitates the detection of patterns using collected historical sensor data. Industrial usage of this framework, accomplished in the frame of the EU C2NET project, supports and automates collaborative business opportunities and real-time monitoring of the production lines

Assessing and Improving Interoperability of Distributed Systems

Interoperabilität von verteilten Systemen ist eine Grundlage für die Entwicklung von neuen und innovativen Geschäftslösungen. Sie erlaubt es existierende Dienste, die auf verschiedenen Systemen angeboten werden, so miteinander zu verknüpfen, dass neue oder erweiterte Dienste zur Verfügung gestellt werden können. Außerdem kann durch diese Integration die Zuverlässigkeit von Diensten erhöht werden. Das Erreichen und Bewerten von Interoperabilität stellt jedoch eine finanzielle und zeitliche Herausforderung dar. Zur Sicherstellung und Bewertung von Interoperabilität werden systematische Methoden benötigt. Um systematisch Interoperabilität von Systemen erreichen und bewerten zu können, wurde im Rahmen der vorliegenden Arbeit ein Prozess zur Verbesserung und Beurteilung von Interoperabilität (IAI) entwickelt. Der IAI-Prozess beinhaltet drei Phasen und kann die Interoperabilität von verteilten, homogenen und auch heterogenen Systemen bewerten und verbessern. Die Bewertung erfolgt dabei durch Interoperabilitätstests, die manuell oder automatisiert ausgeführt werden können. Für die Automatisierung von Interoperabilitätstests wird eine neue Methodik vorgestellt, die einen Entwicklungsprozess für automatisierte Interoperabilitätstestsysteme beinhaltet. Die vorgestellte Methodik erleichtert die formale und systematische Bewertung der Interoperabilität von verteilten Systemen. Im Vergleich zur manuellen Prüfung von Interoperabilität gewährleistet die hier vorgestellte Methodik eine höhere Testabdeckung, eine konsistente Testdurchführung und wiederholbare Interoperabilitätstests. Die praktische Anwendbarkeit des IAI-Prozesses und der Methodik für automatisierte Interoperabilitätstests wird durch drei Fallstudien belegt. In der ersten Fallstudie werden Prozess und Methodik für Internet Protocol Multimedia Subsystem (IMS) Netzwerke instanziiert. Die Interoperabilität von IMS-Netzwerken wurde bisher nur manuell getestet. In der zweiten und dritten Fallstudie wird der IAI-Prozess zur Beurteilung und Verbesserung der Interoperabilität von Grid- und Cloud-Systemen angewendet. Die Bewertung und Verbesserung dieser Interoperabilität ist eine Herausforderung, da Grid- und Cloud-Systeme im Gegensatz zu IMS-Netzwerken heterogen sind. Im Rahmen der Fallstudien werden Möglichkeiten für Integrations- und Interoperabilitätslösungen von Grid- und Infrastructure as a Service (IaaS) Cloud-Systemen sowie von Grid- und Platform as a Service (PaaS) Cloud-Systemen aufgezeigt. Die vorgestellten Lösungen sind in der Literatur bisher nicht dokumentiert worden. Sie ermöglichen die komplementäre Nutzung von Grid- und Cloud-Systemen, eine vereinfachte Migration von Grid-Anwendungen in ein Cloud-System sowie eine effiziente Ressourcennutzung. Die Interoperabilitätslösungen werden mit Hilfe des IAI-Prozesses bewertet. Die Durchführung der Tests für Grid-IaaS-Cloud-Systeme erfolgte manuell. Die Interoperabilität von Grid-PaaS-Cloud-Systemen wird mit Hilfe der Methodik für automatisierte Interoperabilitätstests bewertet. Interoperabilitätstests und deren Beurteilung wurden bisher in der Grid- und Cloud-Community nicht diskutiert, obwohl sie eine Basis für die Entwicklung von standardisierten Schnittstellen zum Erreichen von Interoperabilität zwischen Grid- und Cloud-Systemen bieten.Achieving interoperability of distributed systems offers means for the development of new and innovative business solutions. Interoperability allows the combination of existing services provided on different systems, into new or extended services. Such an integration can also increase the reliability of the provided service. However, achieving and assessing interoperability is a technical challenge that requires high effort regarding time and costs. The reasons are manifold and include differing implementations of standards as well as the provision of proprietary interfaces. The implementations need to be engineered to be interoperable. Techniques that assess and improve interoperability systematically are required. For the assurance of reliable interoperation between systems, interoperability needs to be assessed and improved in a systematic manner. To this aim, we present the Interoperability Assessment and Improvement (IAI) process, which describes in three phases how interoperability of distributed homogeneous and heterogeneous systems can be improved and assessed systematically. The interoperability assessment is achieved by means of interoperability testing, which is typically performed manually. For the automation of interoperability test execution, we present a new methodology including a generic development process for a complete and automated interoperability test system. This methodology provides means for a formalized and systematic assessment of systems' interoperability in an automated manner. Compared to manual interoperability testing, the application of our methodology has the following benefits: wider test coverage, consistent test execution, and test repeatability. We evaluate the IAI process and the methodology for automated interoperability testing in three case studies. Within the first case study, we instantiate the IAI process and the methodology for Internet Protocol Multimedia Subsystem (IMS) networks, which were previously assessed for interoperability only in a manual manner. Within the second and third case study, we apply the IAI process to assess and improve the interoperability of grid and cloud computing systems. Their interoperability assessment and improvement is challenging, since cloud and grid systems are, in contrast to IMS networks, heterogeneous. We develop integration and interoperability solutions for grids and Infrastructure as a Service (IaaS) clouds as well as for grids and Platform as a Service (PaaS) clouds. These solutions are unique and foster complementary usage of grids and clouds, simplified migration of grid applications into the cloud, as well as efficient resource utilization. In addition, we assess the interoperability of the grid-cloud interoperability solutions. While the tests for grid-IaaS clouds are performed manually, we applied our methodology for automated interoperability testing for the assessment of interoperability to grid-PaaS cloud interoperability successfully. These interoperability assessments are unique in the grid-cloud community and provide a basis for the development of standardized interfaces improving the interoperability between grids and clouds

The Oracle Problem When Testing from MSCs

Message Sequence Charts (MSCs) form a popular language in which scenario-based specifications and models can be written. There has been significant interest in automating aspects of testing from MSCs. This paper concerns the Oracle Problem, in which we have an observation made in testing and wish to know whether this is consistent with the specification. We assume that there is an MSC specification and consider the case where we have entirely independent local testers (local observability) and where the observations of the local testers are logged and brought together (tester observability). It transpires that under local observability the Oracle Problem can be solved in low-order polynomial time if we use sequencing, loops and choices but becomes NP-complete if we also allow parallel components; if we place a bound on the number of parallel components then it again can be solved in polynomial time. For tester observability, the problem is NP-complete when we have either loops or choices. However, it can be solved in low-order polynomial time if we have only one loop, no choices, and no parallel components. If we allow parallel components then the Oracle Problem is NP-complete for tester observability even if we restrict to the case where there are at most two processes

A New Model-Based Framework for Testing Security of IOT Systems in Smart Cities Using Attack Trees and Price Timed Automata

International audienceIn this paper we propose a new model-based framework for testing security properties of Internet of Things in Smart Cities. In general a model-based approach consists in extracting test cases from a formal specification either of the system under test or the environment of the considered system in an automatic fashion. Our framework is mainly built on the use of two formalisms namely Attack Trees and Price Timed Automata. An attack tree allows to describe the strategy adopted by the malicious party which intends to violate the security of the considered IOT system. An attack tree is translated into a network of price timed automata. The product of the constructed price timed automata is then computed using the well known UPPAALL platform. The obtained timed automata product serves as input for the adopted test generation algorithm. Moreover our framework takes advantage of the use of the standardized specification and execution testing language TTCN-3. With this respect, the obtained abstract tests are translated into the TTCN-3 format. Finally we propose a cloud-oriented architecture in order to ensure test execution and to collect the generated verdicts