Design Criteria to Architect Continuous Experimentation for Self-Driving Vehicles

The software powering today's vehicles surpasses mechatronics as the dominating engineering challenge due to its fast evolving and innovative nature. In addition, the software and system architecture for upcoming vehicles with automated driving functionality is already processing ~750MB/s - corresponding to over 180 simultaneous 4K-video streams from popular video-on-demand services. Hence, self-driving cars will run so much software to resemble "small data centers on wheels" rather than just transportation vehicles. Continuous Integration, Deployment, and Experimentation have been successfully adopted for software-only products as enabling methodology for feedback-based software development. For example, a popular search engine conducts ~250 experiments each day to improve the software based on its users' behavior. This work investigates design criteria for the software architecture and the corresponding software development and deployment process for complex cyber-physical systems, with the goal of enabling Continuous Experimentation as a way to achieve continuous software evolution. Our research involved reviewing related literature on the topic to extract relevant design requirements. The study is concluded by describing the software development and deployment process and software architecture adopted by our self-driving vehicle laboratory, both based on the extracted criteria.Comment: Copyright 2017 IEEE. Paper submitted and accepted at the 2017 IEEE International Conference on Software Architecture. 8 pages, 2 figures. Published in IEEE Xplore Digital Library, URL: http://ieeexplore.ieee.org/abstract/document/7930218

Considerations about Continuous Experimentation for Resource-Constrained Platforms in Self-Driving Vehicles

Autonomous vehicles are slowly becoming reality thanks to the efforts of many academic and industrial organizations. Due to the complexity of the software powering these systems and the dynamicity of the development processes, an architectural solution capable of supporting long-term evolution and maintenance is required. Continuous Experimentation (CE) is an already increasingly adopted practice in software-intensive web-based software systems to steadily improve them over time. CE allows organizations to steer the development efforts by basing decisions on data collected about the system in its field of application. Despite the advantages of Continuous Experimentation, this practice is only rarely adopted in cyber-physical systems and in the automotive domain. Reasons for this include the strict safety constraints and the computational capabilities needed from the target systems. In this work, a concept for using Continuous Experimentation for resource-constrained platforms like a self-driving vehicle is outlined.Comment: Copyright 2017 Springer. Paper submitted and accepted at the 11th European Conference on Software Architecture. 8 pages, 1 figure. Published in Lecture Notes in Computer Science vol 10475 (Springer), https://link.springer.com/chapter/10.1007/978-3-319-65831-5_

Bridging the Experimental Gap: Applying Continuous Experimentation to the Field of Cyber-Physical Systems, in the Example of the Automotive Domain

In the software world frequent updates and fast delivery of new features are needed by companies to bring value to customers and not lag behind competition. When in cyber-physical systems the software functionality dominates in importance the hardware capabilities, the same speed in creating new value is needed by the product owners to differentiate their products and attract customers. The automotive field is an example of a domain that will face this challenge as the industry races to achieve self-driving vehicles, which will necessarily be software-intensive highly complex cyber-physical systems. A software engineering practice capable of accelerating and guiding the software production process using real-world data is Continuous Experimentation. This practice proved to be valuable in software-intensive web-based systems, allowing data-driven software evolution. It involves the use of experiments, which are instrumented versions of the software to be tested, deployed to the actual systems and executed in a limited way alongside the official software version. Valuable data on the future behavior of the prospective feature is collected in this way as it was fed the same real-world data it would encounter once approved and deployed. Additionally, in those cases where an experimental software version can be run as a replacement for the official version, relevant data regarding the system-user interaction can be gathered. In this thesis, the field of cyber-physical systems and the automotive practitioners\u27 perspective on Continuous Experimentation are sampled employing a literature review and a series of case studies. A set of necessary architectural characteristics are defined and possible methods to overcome the issue of resource constraints in cyber-physical systems are proposed in two exploratory studies. Finally, a design study shows and analyses a prototype of a Continuous Experimentation cycle that was designed and executed in a project partnered by Revere, the Chalmers University of Technology\u27s laboratory for vehicle research

On Using Blockchains for Safety-Critical Systems

Innovation in the world of today is mainly driven by software. Companies need to continuously rejuvenate their product portfolios with new features to stay ahead of their competitors. For example, recent trends explore the application of blockchains to domains other than finance. This paper analyzes the state-of-the-art for safety-critical systems as found in modern vehicles like self-driving cars, smart energy systems, and home automation focusing on specific challenges where key ideas behind blockchains might be applicable. Next, potential benefits unlocked by applying such ideas are presented and discussed for the respective usage scenario. Finally, a research agenda is outlined to summarize remaining challenges for successfully applying blockchains to safety-critical cyber-physical systems

Continuous Experimentation and the cyber-physical systems challenge: An overview of the literature and the industrial perspective.

Context: New software development patterns are emerging aiming at accelerating the process of delivering value. One is Continuous Experimentation, which allows to systematically deploy and run instrumented software variants during development phase in order to collect data from the field of application. While currently this practice is used on a daily basis on web-based systems, technical difficulties challenge its adoption in fields where computational resources are constrained, e.g., cyber-physical systems and the automotive industry. Objective: This paper aims at providing an overview of the engagement on the Continuous Experimentation practice in the context of cyber-physical systems.Method: A systematic literature review has been conducted to investigate the link between the practice and the field of application. Additionally, an industrial multiple case study is reported. Results: The study presents the current state-of-the-art regarding Continuous Experimentation in the field of cyber-physical systems. The current perspective of Continuous Experimentation in industry is also reported. Conclusions: The field has not reached maturity yet. More conceptual analyses are found than solution proposals and the state-of-practice is yet to be achieved. However it is expected that in time an increasing number of solutions will be proposed and validated

Continuous Experimentation for Automotive Software on the Example of a Heavy Commercial Vehicle in Daily Operation

As the automotive industry focuses its attention more and more towards the software functionality of vehicles, techniques to deliver new software value at a fast pace are needed. Continuous Experimentation, a practice coming from the web-based systems world, is one of such techniques. It enables researchers and developers to use real-world data to verify their hypothesis and steer the software evolution based on performances and user preferences, reducing the reliance on simulations and guesswork. Several challenges prevent the verbatim adoption of this practice on automotive cyber-physical systems, e.g., safety concerns and limitations from computational resources; nonetheless, the automotive field is starting to take interest in this technique. This work aims at demonstrating and evaluating a prototypical Continuous Experimentation infrastructure, implemented on a distributed computational system housed in a commercial truck tractor that is used in daily operations by a logistic company on public roads. The system comprises computing units and sensors, and software deployment and data retrieval are only possible remotely via a mobile data connection due to the commercial interests of the logistics company. This study shows that the proposed experimentation process resulted in the development team being able to base software development choices on the real-world data collected during the experimental procedure. Additionally, a set of previously identified design criteria to enable Continuous Experimentation on automotive systems was discussed and their validity confirmed in the light of the presented work.Comment: Paper accepted to the 14th European Conference on Software Architecture (ECSA 2020). 16 pages, 5 figure

An architecture for enabling A/B experiments in automotive embedded software

A/B experimentation is a known technique for data-driven product development and has demonstrated its value in web-facing businesses. With the digitalisation of the automotive industry, the focus in the industry is shifting towards software. For automotive embedded software to continuously improve, A/B experimentation is considered an important technique. However, the adoption of such a technique is not without challenge. In this paper, we present an architecture to enable A/B testing in automotive embedded software. The design addresses challenges that are unique to the automotive industry in a systematic fashion. Going from hypothesis to practice, our architecture was also applied in practice for running online experiments on a considerable scale. Furthermore, a case study approach was used to compare our proposal with state-of-practice in the automotive industry. We found our architecture design to be relevant and applicable in the efforts of adopting continuous A/B experiments in automotive embedded software.Comment: To appear in the 45th Annual IEEE Conference on Computers, Software and Applications (COMPSAC'2021

Introducing Continuous Experimentation on Resource-Constrained Cyber-Physical Systems

Software is ubiquitous and shapes our world, but at the same time it can be viewed as a plastic resource offering the possibility to be improved even after its deployment to better serve its purpose. Exploiting this possibility, the Continuous Experimentation practice is gaining momentum on connected software-intensive web-based systems, allowing the product owners to deploy "experiments" on their software systems, i.e., experimental instrumented versions of the software monitoring its performances with respect to a predefined set of target metrics, and to use this data to drive their products\u27 evolution.Unfortunately the software that runs on physical units is not as easily re-deployed: cyber-physical systems, i.e., systems that interact with the physical world to perform their operations, may be in hard-to-reach places or moving in the environment, making the process difficult or energetically disadvantageous. Furthermore, such systems are often designed to have just enough hardware resources to perform their duties, having little computational resources left to perform additional tasks, such as performance monitoring.This thesis explores the possibility to enable the\ua0Continuous Experimentation practice for distributed software running on resource-constrained\ua0cyber-physical systems on the example of self-driving vehicles, with the long-term goal of providing a way to continuously improve the quality of these systems\u27 performances. To achieve this, the included studies analyzed, proposed, and designed their contributions in order to provide suitable first steps for the adoption of this practice to the field which is still an open research question. Firstly, an analysis of the advantages and disadvantages that\ua0Continuous Experimentation could bring to the field was carried out. Then, key architectural characteristics capable to enable\ua0Continuous Experimentation on\ua0cyber-physical systems were identified. Successively, a more in-depth study was conducted to analyze how the\ua0Continuous Experimentation process could cope with the lack of adequate computational resources.Lastly, acknowledging the criticality of the software modules\u27 intercommunication protocol, an analysis of the communication patterns highlighted how bandwidth-efficient alternatives can be developed using contextual knowledge.The main results of this thesis are the key architectural features that allow the adoption of the\ua0Continuous Experimentation practice on resource-constrained cyber-physical systems

Leonardo da Vinci's Contributions from a Design Perspective

[EN] The figure of Leonardo da Vinci has been extensively studied. In fact, the Leonardiana Library brings together tens of thousands of titles on Leonardo and his work. During the second half of the 20th century, various treaties were published focusing on Leonardo¿s activity as an engineer, and more recently, an increasing number of scientific articles that focus on certain aspects of the prolific work of the genius such as construction, mechanics, strength of materials, etc. have been published. This article analyses the main contributions of the Tuscan genius in the field of design focusing on his processes for generating new solutions, his developments regarding graphic representation techniques, his improvements in plotting and measuring instruments, and how some of his devices were implemented and continue to maintain their usefulness.Cerveró-Meliá, E.; Capuz-Rizo, SF.; Ferrer-Gisbert, P. (2020). Leonardo da Vinci's Contributions from a Design Perspective. Designs. 4(3):1-20. https://doi.org/10.3390/designs4030038S12043Braha, D., & Maimon, O. (1997). The design process: properties, paradigms, and structure. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 27(2), 146-166. doi:10.1109/3468.554679Criteria for Accrediting Engineering Programs, 2019–2020 https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2019-2020/#definitionsVeltman, K. H. (2008). Leonardo da Vinci: A Review. Leonardo, 41(4), 381-388. doi:10.1162/leon.2008.41.4.381Innocenzi, P. (2020). Leonardo and the Design of Machines. Advances in Intelligent Systems and Computing, 36-46. doi:10.1007/978-3-030-41018-6_5Oliveira, A. R. E. (2019). The Mechanical Sciences in Leonardo da Vinci’s Work. Advances in Historical Studies, 08(05), 215-238. doi:10.4236/ahs.2019.85016Jaramillo, H. E. (2011). Un Análisis de la Resistencia de Materiales a partir de los Postulados de «Consideraciones y Demostraciones Matemáticas sobre dos Nuevas Ciencias» de Galileo Galilei. Lámpsakos, (5), 53. doi:10.21501/21454086.819Reciprocating machine for weight lifting (Argano), Codex Atlanticus f. 30 v, (1478–1480) https://commons.wikimedia.org/wiki/File:Reproduction_of_page_from_notebook_of_Leonardo_da_Vinci_showing_a_geared_device_assembled_and_disassembled_LCCN2006681098.jpgModel at the Museum of Science and Technology of Milan https://commons.wikimedia.org/wiki/File:Argano_sollevatore_pesi_Leonardo_Museo_scienza_e_tecnologia_Milano.jpgMap of the Val di Chiana, Royal Collection, RLW 12278, (1502–1504) https://commons.wikimedia.org/wiki/File:Val_di_Chiana.jpgReproduction of a compass designed by Leonardo https://commons.wikimedia.org/wiki/File:Compas_Léonard_de_Vinci.JPGProportional or reduction compass. Forster Codex I f. 45 (1485) https://commons.wikimedia.org/wiki/File:Reduction_Compass_Leonardo.jpgParabolic Compass. Codex Atlanticus f. 1093 r https://upload.wikimedia.org/wikipedia/commons/archive/0/03/20171027130237%21Leonardo_parabolic_compass.JPG.Detail of the Codex Atlanticus f. 5 r. Enlarged detail of the prospectograph being used by Leonardo https://commons.wikimedia.org/wiki/File:Codice_Atlantico_-_Perspectograph.jpgStudy ot two odometers. Codex Atlanticus, f. 1b r https://commons.wikimedia.org/wiki/File:Odomètre-Léonard.jpgOdometer model. Museo Nazionale della Scienza e della Tecnologia Leonardo da Vinci. (National Museum of Science and Technology of Milan) https://commons.wikimedia.org/wiki/File:Odometro_a_carriola_-_Museo_scienza_tecnologia_Milano_09908_01.jpgPugno, N. M. (2019). The commemoration of Leonardo da Vinci. Meccanica, 54(15), 2317-2324. doi:10.1007/s11012-019-01099-9Study for the mechanism of a manual lift (1495–1497), Madrid Codex I, f. 9 r https://commons.wikimedia.org/wiki/File:Ascenceur_à_manivelle-Léonard.jpgStudy of a piling machine. Codex Atlanticus, f 785, Ambrosian Library of Milan https://commons.wikimedia.org/wiki/File:Sonnette-Léonard.jpgModel of Leonardo’s pile machine, at the National Museum of Science and Technology of Milan https://commons.wikimedia.org/wiki/File:Battipalo_-_Museo_scienza_tecnologia_Milano_00040_01.jpgDetail of a mechanical jack, Codex Atlanticus, f. 0998 r, Ambrosian Library of Milan https://commons.wikimedia.org/wiki/File:Cric-Léonard.jpgManuscript of the self-propelled vehicle, Codex Atlanticus, f. 812 r (1478-1480), Ambrosiana Library of Milan https://commons.wikimedia.org/wiki/File:Leonardo_da_vinci,_Automobile.jpgModel of the self-propelled vehicle, at the National Museum of Science and Technology of Milan https://commons.wikimedia.org/wiki/File:Carro_semovente_-_Museo_scienza_tecnologia_Milano_09082_02.jp