Facilitating evolution during design and implementation

The volumes and complexity of data that companies need to handle are increasing at an accelerating rate. In order to compete effectively and ensure their commercial sustainability, it is becoming crucial for them to achieve robust traceability in both their data and the evolving designs of their systems. This is addressed by the CRISTAL software which was originally developed at CERN by UWE, Bristol, for one of the particle detectors at the Large Hadron Collider, and has been subsequently transferred into the commercial world. Companies have been able to demonstrate increased agility, generate additional revenue, and improve the efficiency and cost-effectiveness with which they develop and implement systems in various areas, including business process management (BPM), healthcare and accounting applications. CRISTAL’s ability to manage data and its provenance at the terabyte scale, with full traceability over extended timescales, together with its description-driven approach, has provided the flexible adaptability required to future proof dynamically evolving software for these businesses

Designing Traceability into Big Data Systems

Providing an appropriate level of accessibility and traceability to data or process elements (so-called Items) in large volumes of data, often Cloud-resident, is an essential requirement in the Big Data era. Enterprise-wide data systems need to be designed from the outset to support usage of such Items across the spectrum of business use rather than from any specific application view. The design philosophy advocated in this paper is to drive the design process using a so-called description-driven approach which enriches models with meta-data and description and focuses the design process on Item re-use, thereby promoting traceability. Details are given of the description-driven design of big data systems at CERN, in health informatics and in business process management. Evidence is presented that the approach leads to design simplicity and consequent ease of management thanks to loose typing and the adoption of a unified approach to Item management and usage.Comment: 10 pages; 6 figures in Proceedings of the 5th Annual International Conference on ICT: Big Data, Cloud and Security (ICT-BDCS 2015), Singapore July 2015. arXiv admin note: text overlap with arXiv:1402.5764, arXiv:1402.575

The Deployment of an Enhanced Model-Driven Architecture for Business Process Management

Business systems these days need to be agile to address the needs of a changing world. Business modelling requires business process management to be highly adaptable with the ability to support dynamic workflows, inter-application integration (potentially between businesses) and process reconfiguration. Designing systems with the in-built ability to cater for evolution is also becoming critical to their success. To handle change, systems need the capability to adapt as and when necessary to changes in users requirements. Allowing systems to be self-describing is one way to facilitate this. Using our implementation of a self-describing system, a so-called description-driven approach, new versions of data structures or processes can be created alongside older versions providing a log of changes to the underlying data schema and enabling the gathering of traceable (provenance) data. The CRISTAL software, which originated at CERN for handling physics data, uses versions of stored descriptions to define versions of data and workflows which can be evolved over time and thereby to handle evolving system needs. It has been customised for use in business applications as the Agilium-NG product. This paper reports on how the Agilium-NG software has enabled the deployment of an unique business process management solution that can be dynamically evolved to cater for changing user requirement.Comment: 11 pages, 4 figures, 1 table, 22nd International Database Engineering & Applications Symposium (IDEAS 2018). arXiv admin note: text overlap with arXiv:1402.5764, arXiv:1402.5753, arXiv:1502.0154

Analysis traceability and provenance for HEP

This paper presents the use of the CRISTAL software in the N4U project. CRISTAL was used to create a set of provenance aware analysis tools for the Neuroscience domain. This paper advocates that the approach taken in N4U to build the analysis suite is sufficiently generic to be able to be applied to the HEP domain. A mapping to the PROV model for provenance interoperability is also presented and how this can be applied to the HEP domain for the interoperability of HEP analyses

Analysing the provenance tracking of business process management in the quality domain

This work presents a framework of how Provenance can be combined with BPM. This is achieved through a use case, from the Martine Spécialités enterprise which is using the Agilium BPM system (without the provenance functionality). Using the PROV Data Model, a graphical representation of the provenance tracking of the processes of the enterprise has been done and the functional requirements for combining provenance and BPM have been defined. A survey was submitted to the employees in order to see if adding the provenance functionality is something beneficial for them. The results have shown that provenance is adding value for the decision making during the execution of the processes, and that several benefits can be obtained from this, such as better decision making and time saving

Gestion autonomique d'applications dynamiques sûres et résilientes

Service-Oriented architectures (SOA) are considered the most advanced way to develop and integrate modular and flexible applications.There are many SOA platforms available for software developers and architects; the most evolved of them being SCA and OSGi.An application based on one of these platforms can be assembled with only the components required for the execution of its tasks, which helps decreasing its resource consumption and increasing its maintainability.Furthermore, those platforms allow adding plug-ins at runtime, even if they were not known during the early stages of the development of the application.Thus, they allow updating, extending and adapting the features of the base product or of the technical services required for its execution, continuously and without outage.Those capabilities are applied in the DevOps paradigm and, more generally, to implement the continuous deployment of artifacts.However, the extensibility provided by those platforms can decrease the overall reliability of the system: a strong tendency in software development is the assembly of third-parties components.Such components may be of unknown or even questionable quality.In case of error, deterioration of performance, ... it is difficult to identify the implicated components or combinations of components.It becomes essential for the software producer to determine the responsibility of the various components involved in a malfunction.This thesis aims to provide a platform, Cohorte, to design and implement scalable software products, resilient to malfunctions of unqualified extensions.The components of such products may be developed in various programming languages and be deployed continuously (adding, updating and withdrawal) and without interruption of service.Our proposal adopts the principle of isolating the components considered unstable or insecure.The choice of the components to be isolated may be decided by the development team and the operational team, from their expertise, or determined from a combination of indicators.The latters evolve over time to reflect the reliability of components.For example, components can be considered reliable after a quarantine period; an update may result in deterioration of stability, ...Therefore, it is essential to question the initial choices in isolating components to limit, in the first case, the scope of communications between components and, in the second case, to maintain the reliability of the critical core of the product.Les architectures orientées services (SOA) sont considérées comme le moyen le plus avancé pour réaliser et intégrer rapidement des applications modulaires et flexibles.Dans ce domaine, les plates-formes SOA à disposition des développeurs et des architectes de produits logiciels sont multiples; les deux plus évoluées d'entre elles étant SCA et OSGi.Une application s'appuyant sur l'une de ces plates-formes peut ainsi être assemblée avec le minimum de composants nécessaires à la réalisation de ses tâches, afin de réduire sa consommation de ressources et d'augmenter sa maintenabilité.De plus, ces plates-formes autorisent l'ajout de composants greffons qui n'étaient pas connus lors des phases initiales de la réalisation du produit.Elles permettent ainsi de mettre à jour, d'étendre et d'adapter continuellement les fonctionnalités du produit de base ou des services techniques nécessaires à sa mise en production, sans interruption de service.Ces capacités sont notamment utilisées dans le cadre du paradigme DevOps et, plus généralement, pour mettre en œuvre le déploiement continu d'artefacts.Cependant, l'extensibilité offerte par ces plates-formes peut diminuer la fiabilité globale du système: une tendance forte pour développer un produit est l'assemblage de composants provenant de tierces-parties. De tels composants peuvent être d'une qualité inconnue voire douteuse.En cas d'erreur, de détérioration des performances, etc., il est difficile de diagnostiquer les composants ou combinaisons de composants incriminés.Il devient indispensable pour le producteur d'un logiciel de déterminer la responsabilité des différents composants impliqués dans un dysfonctionnement.Cette thèse a pour objectif de fournir une plate-forme, Cohorte, permettant de concevoir et d'exécuter des produits logiciels extensibles et résilients aux dysfonctionnements d'extensions non qualifiées.Les composants de tels produits pourront être développés dans différents langages de programmation et être déployés (ajout, mise à jour et retrait) en continu et sans interruption de service.Notre proposition adopte pour principe d'isoler les composants considérés comme instables ou peu sûrs.Le choix des composants à isoler peut être décidé par l'équipe de développement et l'équipe opérationnelle, à partir de leur expertise, ou bien déterminé à partir d'une combinaison d'indicateurs.Ces derniers évoluent au cours du temps pour refléter la fiabilité des composants.Par exemple, des composants peuvent être considérés fiables après une période de quarantaine; une mise à jour peut entraîner la dégradation de leur stabilité, etc..Par conséquent, il est indispensable de remettre en cause les choix initiaux dans l'isolation des composants afin, dans le premier cas, de limiter le coup des communications entre composants et, dans le deuxième cas, de maintenir le niveau de fiabilité du noyau critique du produit