Making Bipedal Robot Experiments Reproducible and Comparable: The Eurobench Software Approach

This study describes the software methodology designed for systematic benchmarking of bipedal systems through the computation of performance indicators from data collected during an experimentation stage. Under the umbrella of the European project Eurobench, we collected approximately 30 protocols with related testbeds and scoring algorithms, aiming at characterizing the performances of humanoids, exoskeletons, and/or prosthesis under different conditions. The main challenge addressed in this study concerns the standardization of the scoring process to permit a systematic benchmark of the experiments. The complexity of this process is mainly due to the lack of consistency in how to store and organize experimental data, how to define the input and output of benchmarking algorithms, and how to implement these algorithms. We propose a simple but efficient methodology for preparing scoring algorithms, to ensure reproducibility and replicability of results. This methodology mainly constrains the interface of the software and enables the engineer to develop his/her metric in his/her favorite language. Continuous integration and deployment tools are then used to verify the replicability of the software and to generate an executable instance independent of the language through dockerization. This article presents this methodology and points at all the metrics and documentation repositories designed with this policy in Eurobench. Applying this approach to other protocols and metrics would ease the reproduction, replication, and comparison of experiments.This study is supported by the European Union’s Horizon 2020 research and innovation program under Grant Agreement no 779963, project Eurobench

A method to quantify the reduction of back and hip muscle fatigue of lift-support exoskeletons

Cumulative back muscle fatigue plays a role in the occurrence of low-back injuries in occupations that require repetitive lifting of heavy loads and working in forward leaning postures. Lift-support exoskeletons have the potential to reduce back and hip muscle activity, thereby delaying the onset of fatigue in these muscles. Therefore, exoskeletons are being considered a potentially important tool to further reduce workload-related injuries. However, today no standards have been established on how to benchmark the support level of lift-support exoskeletons. This work proposes an experimental protocol to quantify the support level of a lift-support exoskeletons on instant changes in muscle activity and fatigue development while maintaining a static forward leaning posture. It then applies the protocol to experimentally assess the effect of the support provided by a commercially available lift-support exoskeleton, the LiftSuit 2.0 (Auxivo AG, Schwerzenbach, Switzerland), on the user. In a sample of 14 participants, the amplitude of the muscle activity of the back muscles and hip muscles was significantly reduced. Wearing the exoskeleton significantly reduced the amount of fatigue developed during the task . Changes in muscle fatigue can be objectively recorded and correlated with relevant changes for exoskeleton users: the time a task can be performed and perceived low-back fatigue. Thus, including such measures of fatigue in standardized benchmarking procedures will help quantify the benefits of exoskeletons for occupational use.This project has received funding from the European Union’s Horizon 2020 research and innovation program under the framework of the EUROBENCH project (grant agreement no. 779963)