Model-based testing of PLC programs with appropriate conformance relations

International audienceNumerous theoretical results have been obtained in the field of conformance testing, a very promising formal technique to improve dependability of critical systems. Nevertheless, developing on this basis PLC test techniques that produce correct conformance verdicts requires to take into account the real technological features of PLC. This paper proposes conformance relations that meet this objective. Examples illustrate the benefits of the contribution

On the Safe IOCOS relation for Testing Safety PLC Code

In this paper, limitations of the IOCOS testing relation in regard to testing safety PLC code is examined and a modification of the current IOCOS relation, called safe-IOCOS is proposed. In the IOCOS testing relation, an implementation is IOCOS with respect to a specification, if it emits a subset of the specified outputs and a super-set of the specified inputs after the execution of each trace in the specification. However, for testing safety PLC code, the IOCOS relation is not detailed enough as the subset requirement on the respective inputs and outputs could allow some safety behaviors to go untested. These limitations of the IOCOS relation may thus pose threats to humans. So the notion of safe-IOCOS is defined, which strengthens IOCOS to require equality between the implementation and the specification in relation to the inputs and outputs, respectively. An example shows these shortcomings of IOCOS and how the proposed safe-IOCOS relation is better suited for testing safety PLC code

New Challenges in Quality of Services Control Architectures in Next Generation Networks

A mesura que Internet i les xarxes IP s'han anat integrant dins la societat i les corporacions, han anat creixent les expectatives de nous serveis convergents així com les expectatives de qualitat en les comunicacions. Les Next Generation Networks (NGN) donen resposta a les noves necessitats i representen el nou paradigma d'Internet a partir de la convergència IP. Un dels aspectes menys desenvolupats de les NGN és el control de la Qualitat del Servei (QoS), especialment crític en les comunicacions multimèdia a través de xarxes heterogènies i/o de diferents operadors. A més a més, les NGN incorporen nativament el protocol IPv6 que, malgrat les deficiències i esgotament d'adreces IPv4, encara no ha tingut l'impuls definitiu.Aquesta tesi està enfocada des d'un punt de vista pràctic. Així doncs, per tal de poder fer recerca sobre xarxes de proves (o testbeds) que suportin IPv6 amb garanties de funcionament, es fa un estudi en profunditat del protocol IPv6, del seu grau d'implementació i dels tests de conformància i interoperabilitat existents que avaluen la qualitat d'aquestes implementacions. A continuació s'avalua la qualitat de cinc sistemes operatius que suporten IPv6 mitjançant un test de conformància i s'implementa el testbed IPv6 bàsic, a partir del qual es farà la recerca, amb la implementació que ofereix més garanties.El QoS Broker és l'aportació principal d'aquesta tesi: un marc integrat que inclou un sistema automatitzat per gestionar el control de la QoS a través de sistemes multi-domini/multi-operador seguint les recomanacions de les NGN. El sistema automatitza els mecanismes associats a la configuració de la QoS dins d'un mateix domini (sistema autònom) mitjançant la gestió basada en polítiques de QoS i automatitza la negociació dinàmica de QoS entre QoS Brokers de diferents dominis, de forma que permet garantir QoS extrem-extrem sense fissures. Aquesta arquitectura es valida sobre un testbed de proves multi-domini que utilitza el mecanisme DiffServ de QoS i suporta IPv6.L'arquitectura definida en les NGN permet gestionar la QoS tant a nivell 3 (IP) com a nivell 2 (Ethernet, WiFi, etc.) de forma que permet gestionar també xarxes PLC. Aquesta tesi proposa una aproximació teòrica per aplicar aquesta arquitectura de control, mitjançant un QoS Broker, a les noves xarxes PLC que s'estan acabant d'estandarditzar, i discuteix les possibilitats d'aplicació sobre les futures xarxes de comunicació de les Smart Grids.Finalment, s'integra en el QoS Broker un mòdul per gestionar l'enginyeria del tràfic optimitzant els dominis mitjançant tècniques de intel·ligència artificial. La validació en simulacions i sobre un testbed amb routers Cisco demostra que els algorismes genètics híbrids són una opció eficaç en aquest camp.En general, les observacions i avenços assolits en aquesta tesi contribueixen a augmentar la comprensió del funcionament de la QoS en les NGN i a preparar aquests sistemes per afrontar problemes del món real de gran complexitat.A medida que Internet y las redes IP se han ido integrando dentro de la sociedad y las corporaciones, han ido creciendo las expectativas de nuevos servicios convergentes así como las expectativas de calidad en las comunicaciones. Las Next Generation Networks (NGN) dan respuesta a las nuevas necesidades y representan el nuevo paradigma de Internet a partir de la convergencia IP. Uno de los aspectos menos desarrollados de las NGN es el control de la Calidad del Servicio (QoS), especialmente crítico en las comunicaciones multimedia a través de redes heterogéneas y/o de diferentes operadores. Además, las NGN incorporan nativamente el protocolo IPv6 que, a pesar de las deficiencias y agotamiento de direcciones IPv4, aún no ha tenido el impulso definitivo.Esta tesis está enfocada desde un punto de vista práctico. Así pues, con tal de poder hacer investigación sobre redes de prueba (o testbeds) que suporten IPv6 con garantías de funcionamiento, se hace un estudio en profundidad del protocolo IPv6, de su grado de implementación y de los tests de conformancia e interoperabilidad existentes que evalúan la calidad de estas implementaciones. A continuación se evalua la calidad de cinco sistemas operativos que soportan IPv6 mediante un test de conformancia y se implementa el testbed IPv6 básico, a partir del cual se realizará la investigación, con la implementación que ofrece más garantías.El QoS Broker es la aportación principal de esta tesis: un marco integrado que incluye un sistema automatitzado para gestionar el control de la QoS a través de sistemas multi-dominio/multi-operador siguiendo las recomendaciones de las NGN. El sistema automatiza los mecanismos asociados a la configuración de la QoS dentro de un mismo dominio (sistema autónomo) mediante la gestión basada en políticas de QoS y automatiza la negociación dinámica de QoS entre QoS brokers de diferentes dominios, de forma que permite garantizar QoS extremo-extremo sin fisuras. Esta arquitectura se valida sobre un testbed de pruebas multi-dominio que utiliza el mecanismo DiffServ de QoS y soporta IPv6. La arquitectura definida en las NGN permite gestionar la QoS tanto a nivel 3 (IP) o como a nivel 2 (Ethernet, WiFi, etc.) de forma que permite gestionar también redes PLC. Esta tesis propone una aproximación teórica para aplicar esta arquitectura de control, mediante un QoS Broker, a las noves redes PLC que se están acabando de estandardizar, y discute las posibilidades de aplicación sobre las futuras redes de comunicación de las Smart Grids.Finalmente, se integra en el QoS Broker un módulo para gestionar la ingeniería del tráfico optimizando los dominios mediante técnicas de inteligencia artificial. La validación en simulaciones y sobre un testbed con routers Cisco demuestra que los algoritmos genéticos híbridos son una opción eficaz en este campo.En general, las observaciones y avances i avances alcanzados en esta tesis contribuyen a augmentar la comprensión del funcionamiento de la QoS en las NGN y en preparar estos sistemas para afrontar problemas del mundo real de gran complejidad.The steady growth of Internet along with the IP networks and their integration into society and corporations has brought with it increased expectations of new converged services as well as greater demands on quality in communications. The Next Generation Networks (NGNs) respond to these new needs and represent the new Internet paradigm from the IP convergence. One of the least developed aspects in the NGNs is the Quality of Service (QoS) control, which is especially critical in the multimedia communication through heterogeneous networks and/or different operators. Furthermore, the NGNs natively incorporate the IPv6 protocol which, despite its shortcomings and the depletion of IPv4 addresses has not been boosted yet.This thesis has been developed with a practical focus. Therefore, with the aim of carrying out research over testbeds supporting the IPv6 with performance guarantees, an in-depth study of the IPv6 protocol development has been conducted and its degree of implementation and the existing conformance and interoperability tests that evaluate these implementations have been studied. Next, the quality of five implementations has been evaluated through a conformance test and the basic IPv6 testbed has been implemented, from which the research will be carried out. The QoS Broker is the main contribution to this thesis: an integrated framework including an automated system for QoS control management through multi-domain/multi-operator systems according to NGN recommendations. The system automates the mechanisms associated to the QoS configuration inside the same domain (autonomous system) through policy-based management and automates the QoS dynamic negotiation between peer QoS Brokers belonging to different domains, so it allows the guarantee of seamless end-to-end QoS. This architecture is validated over a multi-domain testbed which uses the QoS DiffServ mechanism and supports IPv6.The architecture defined in the NGN allows QoS management at level 3 (IP) as well as at level 2 (e.g. Ethernet, WiFi) so it also facilitates the management of PLC networks. Through the use of a QoS Broker, this thesis proposes a theoretical approach for applying this control architecture to the newly standardized PLC networks, and discusses the possibilities of applying it over the future communication networks of the Smart Grids.Finally, a module for managing traffic engineering which optimizes the network domains through artificial intelligence techniques is integrated in the QoS Broker. The validations by simulations and over a Cisco router testbed demonstrate that hybrid genetic algorithms are an effective option in this area.Overall, the advances and key insights provided in this thesis help advance our understanding of QoS functioning in the NGNs and prepare these systems to face increasingly complex problems, which abound in current industrial and scientific applications

Formal Specification and Verification for Automated Production Systems

Complex industrial control software often drives safety- and mission-critical systems, like automated production plants or control units embedded into devices in automotive systems. Such controllers have in common that they are reactive systems, i.e., that they periodically read sensor stimuli and cyclically execute the same program to produce actuator signals. The correctness of software for automated production is rarely verified using formal techniques. Although, due to the Industrial Revolution 4.0 (IR4.0), the impact and importance of software have become an important role in industrial automation. What is used instead in industrial practice today is testing and simulation, where individual test cases are used to validate an automated production system. Three reasons why formal methods are not popular are: (a) It is difficult to adequately formulate the desired temporal properties. (b) There is a lack of specification languages for reactive systems that are both sufficiently expressive and comprehensible for practitioners. (c) Due to the lack of an environment model the obtained results are imprecise. Nonetheless, formal methods for automated production systems are well studied academically---mainly on the verification of safety properties via model checking. In this doctoral thesis we present the concept of (1) generalized test tables (GTTs), a new specification language for functional properties, and their extension (2) relational test tables (RTTs) for relational properties. The concept includes the syntactical notion, designed for the intuition of engineers, and the semantics, which are based on game theory. We use RTTs for a novel confidential property on reactive systems, the provably forgetting of information. Moreover, for regression verification, an important relational property, we are able to achieve performance improvements by (3) creating a decomposing rule which splits large proofs into small sub-task. We implemented the verification procedures and evaluated them against realistic case studies, e.g., the Pick-and-Place-Unit from the Technical University of Munich. The presented contribution follows the idea of lowering the obstacle of verifying the dependability of reactive systems in general, and automated production systems in particular for the engineer either by introducing a new specification language (GTTs), by exploiting existing programs for the specification (RTTs, regression verification), or by improving the verification performance

Achieving business excellence in software quality management

Many companies have had difficulties in achieving success with software process improvement initiatives or have had adverse experiences in implementing quality systems. With a plethora of standards available and the numerous frameworks to apply best practice, none appears to act as a panacea to guarantee fulfilment or realise a true Return-on-Investment. This thesis proposes a holistic approach to software process improvement, describing a range of supporting tools and methods highlighting a true understanding of the customer base and associated cultures. The research aim was to develop and evaluate a demonstrably effective and efficient software quality management methodology suitable for a technical company such as AWE plc. To be effective the methodology must deliver an improved conformance to the quality standards and deliver real process improvement. To be efficient the methodology must deliver a real Return-on-Investment. Case studies on the implementation of the quality system were carried out at AWE plc. Each case study provided a further opportunity to measure and analyse the success or otherwise of that method or tool for further refinement. Audits, self-assessment, training, system design, marketing, and the people skills associated with a consultation process are all examined in detail. The research methodology has demonstrated its success as case studies show that steady improvement in implementing the software quality system has occurred year on year. This success has been validated by third party ISO 9001 assessments and has led to an enhancement in AWE plc reputation as a centre of software excellence. The approach has overcome cultural resistance and changed working practices. With a philosophy of customer care, consultation, and active engagement, practitioners now adopt best-practice quality management principles. The cost effectiveness of this methodology means its adoption could be considered by any organisation whether large or small

Digital Twin for Legacy Systems: Simulation Model Testing and Validation

In this paper, an approach to incorporate a digitaltwin for legacy production systems is presented. Hardware-in-the-loop setups are routinely used by manufacturing companiesto carry out virtual commissioning. However, manufacturingcompanies having online legacy production systems are stillstruggling to incorporate a digital twin due to the absence ofverified and validated simulation models. Companies that usevirtual commissioning as a part of their engineering tool chain,usually perform offline verification of the simulation model.This approach is typically based on visual inspection and is atedious task as each aspect of the model has to be visuallyvalidated. For legacy systems, only assessing the behaviorvisually in the absence of updated documents can result in anincorrect simulation model, i.e. simulating incorrect behaviorwith respect to the specification. Due to this, such simulationmodels cannot be incorporated in the engineering tool chain,as the simulated results can lead to improper decisions and caneven cause equipment damage. This paper presents a platformand an approach, based on model-based testing, that is a firststep for manufacturing companies to incorporate a validatedsimulation model for existing online production systems thatwill serve as a digital twin

Increasing System Test Coverage in Production Automation Systems

An approach is introduced, which supports a testing technician in the identification of possibly untested behavior of control software of fully integrated automated production systems (aPS). Based on an approach for guided semi-automatic system testing, execution traces are recorded during testing, allowing a subsequent coverage assessment. As the behavior of an aPS is highly dependent on the software, omitted system behavior can be identified and assessed for criticality. Through close cooperation with industry, this approach represents the first coverage assessment approach for system testing in production automation to be applied on real industrial objects and evaluated by industrial experts