Systematic and Realistic Testing in Simulation of Control Code for Robots in Collaborative Human-Robot Interactions

© Springer International Publishing Switzerland 2016. Industries such as flexible manufacturing and home care will be transformed by the presence of robotic assistants. Assurance of safety and functional soundness for these robotic systems will require rigorous verification and validation. We propose testing in simulation using Coverage-Driven Verification (CDV) to guide the testing process in an automatic and systematic way. We use a two-tiered test generation approach, where abstract test sequences are computed first and then concretized (e.g., data and variables are instantiated), to reduce the complexity of the test generation problem. To demonstrate the effectiveness of our approach, we developed a testbench for robotic code, running in ROS-Gazebo, that implements an object handover as part of a humanrobot interaction (HRI) task. Tests are generated to stimulate the robot’s code in a realistic manner, through stimulating the human, environment, sensors, and actuators in simulation. We compare the merits of unconstrained, constrained and model-based test generation in achieving thorough exploration of the code under test, and interesting combinations of human-robot interactions. Our results show that CDV combined with systematic test generation achieves a very high degree of automation in simulation-based verification of control code for robots in HRI

Dimensional Analysis of Robot Software without Developer Annotations

Robot software risks the hazard of dimensional inconsistencies. These inconsistencies occur when a program incorrectly manipulates values representing real-world quantities. Incorrect manipulation has real-world consequences that range in severity from benign to catastrophic. Previous approaches detect dimensional inconsistencies in programs but require extra developer effort and technical complications. The extra effort involves developers creating type annotations for every variable representing a real-world quantity that has physical units, and the technical complications include toolchain burdens like specialized compilers or type libraries. To overcome the limitations of previous approaches, this thesis presents novel methods to detect dimensional inconsistencies without developer annotations. We start by empirically assessing the difficulty developers have in making type annotations. In a human study of 83 subjects, we find that developers are only 51% accurate and require more than 2 minutes per annotation. We further find that type suggestions have a significant impact on annotation accuracy. We find that when showing developers annotation suggestions, three suggestions are better than a single suggestion because they are as helpful when correct and less harmful when incorrect. Since developers struggle to make type annotations accurately, we present a novel method to infer physical unit types without developer annotations. This is novel because it is the first method to detect dimensional inconsistencies in ROS C++ without developer annotations, and this is important because robot software and ROS are increasingly used in real-world applications. Our method leverages a property of robotic middleware architecture that reuses standardized data structures, and we implement our method in an open-source tool, Phriky. We evaluate our method empirically on a corpus of 5.9 M lines of code and find that it detects real inconsistencies with an 87% TP rate. However, our method only assigns physical unit types to 25% of variables, leaving much of the annotation space unaddressed. To overcome these limitations, we extend our method to utilize uncertain evidence in identifiers using probabilistic reasoning. We implement our new probabilistic method in a tool Phys and find that it assigns units to 75% of variables while retaining a TP rate of 82%. We present the first open dataset of dimensional inconsistencies in open-source robotics code, to our knowledge. Lastly, we identify extensions to our work and next steps for software tool developers to build more powerful robot software development tools. Advisers: Sebastian Elbaum and Carrick Detweile

Computer Aided Verification

This open access two-volume set LNCS 10980 and 10981 constitutes the refereed proceedings of the 30th International Conference on Computer Aided Verification, CAV 2018, held in Oxford, UK, in July 2018. The 52 full and 13 tool papers presented together with 3 invited papers and 2 tutorials were carefully reviewed and selected from 215 submissions. The papers cover a wide range of topics and techniques, from algorithmic and logical foundations of verification to practical applications in distributed, networked, cyber-physical, and autonomous systems. They are organized in topical sections on model checking, program analysis using polyhedra, synthesis, learning, runtime verification, hybrid and timed systems, tools, probabilistic systems, static analysis, theory and security, SAT, SMT and decisions procedures, concurrency, and CPS, hardware, industrial applications

Methoden und Ansätze für die Entwicklung und den Test prädiktiver Fahrzeugregelungsfunktionen

In dieser Arbeit werden das aktuelle Vorgehen und die Prozesse in der automobilen Produktentwicklung sowie die etablierten Methoden für die Entwicklung, Verifikation und Validierung von Fahrzeugregelungsfunktionen analysiert. Dem wird eine Taxonomie und Analyse aktueller Serienanwendungen und Forschungskonzepte gegenüber gestellt. Ziel ist es, durch eine ganzheitliche Betrachtung die aktuellen Rahmenbedingungen und Herausforderungen bei der Entwicklung innovativer Funktionen für die Automatisierung der Fahraufgabe zu identifizieren. Auf dieser Grundlage wird ein neuartiges Konzept für die Entwicklung und den Test prädiktiver Fahrzeugregelungsfunktionen erarbeitet. Das Kernstück des entwickelten Konzepts stellt die Reactive-Replay Methode dar. Sie ermöglicht eine enge Verzahnung von Erprobungsfahrten in der realen Welt mit der Ausführung der entwickelten Fahrzeugfunktion innerhalb einer Simulationsumgebung. Die adaptive Wiedergabe von während der Erprobung aufgezeichneten Daten des fahrzeuginternen Kommunikationsnetzes ermöglicht einen nahtlosen Übergang von der realen Welt im Fahrzeug in die Simulation im Büro. Auf diese Weise können in der Realität aufgetretene Situationen und Szenarien detailliert und unter Laborbedingungen untersucht und für Tests wiederverwendet werden. Darüber hinaus ermöglicht dieser Ansatz eine effiziente Generierung valider Testszenarien, die durch ihre Vielfältigkeit und Varianz zu einer verbesserten Testabdeckung beitragen. Um die entwickelte Methode systematisch in den produktiven Alltag der Funktionsentwicklung zu integrieren, wird ein schlankes, iteratives Vorgehen zur prozessualen Integration der Reactive-Replay Methode vorgeschlagen. Die Verifikation in der Simulationsumgebung wird so mit der Validierung in der Fahrzeugerprobung gekoppelt. Dies unterstützt die frühzeitige und durchgängige Qualitätsbewertung der entwickelten Fahrzeugfunktion. Weiter wird eine Methode zur kontinuierlichen Überprüfung von Anforderungen während der Simulationsausführung untersucht. Ein Ansatz zur effizienten Auswahl von Testszenarien auf Basis der innerhalb eines Szenarios erreichten Parameterüberdeckung rundet die Arbeit ab

Computer Aided Verification

This open access two-volume set LNCS 10980 and 10981 constitutes the refereed proceedings of the 30th International Conference on Computer Aided Verification, CAV 2018, held in Oxford, UK, in July 2018. The 52 full and 13 tool papers presented together with 3 invited papers and 2 tutorials were carefully reviewed and selected from 215 submissions. The papers cover a wide range of topics and techniques, from algorithmic and logical foundations of verification to practical applications in distributed, networked, cyber-physical, and autonomous systems. They are organized in topical sections on model checking, program analysis using polyhedra, synthesis, learning, runtime verification, hybrid and timed systems, tools, probabilistic systems, static analysis, theory and security, SAT, SMT and decisions procedures, concurrency, and CPS, hardware, industrial applications

GSI Scientific Report 2016

PLEASE GO TO FILES TO SELECT YOUR DOWNLOAD SECTION. Lience: https://creativecommons.org/licenses/by/4.0