Modelling, synthesis and model-based motion planning for hyper-redundant, binary actuated manipulators

Die Untersuchung von schwer zu erreichenden Hohlräumen durch schmale Zugänge wird im technischen Umfeld als Boroskopie und in der Medizin als Endoskopie bezeichnet. Wenn neben der reinen Inspektion auch eine Manipulation erfolgen soll, wird ergänzend zu einer guten Anpassbarkeit an gekrümmte Pfade auch eine stabile Arbeitsplattform zur Aufnahme von Manipulationskräften benötigt. Einen Ansatz, die daraus resultierenden Anforderungen an die verwendeten Systeme zu adressieren, stellen schlangenartige Roboter dar. Ihre hyperredundante Struktur aus einzelnen Stellgliedern bietet eine vielseitige Positionierbarkeit. Die Verwendung von binären, kippstabilen Aktoren beschränkt zwar den Arbeitsraum auf wenige diskrete Punkte, bietet aber – in Abhängigkeit vom Antriebsmechanismus – besonders hohe Haltemomente und ermöglicht damit eine gezielte Systemversteifung. Eine Kombination beider Ansätze zur Klasse der binär aktuierten, hyperredundanten Manipulatoren ist in der Lage, diese Anforderungen zu erfüllen, jedoch existiert deutlicher Forschungsbedarf hinsichtlich Methoden zur optimalen Auslegung sowie zur gezielten Verfolgung von Referenzpfaden, sodass Kern der vorliegenden Arbeit die Erforschung der modellbasierten Bewegungsplanung dieser Roboterklasse ist. Voraussetzung für eine hohe Pfadfolgegenauigkeit ist, dass der Manipulator sich grundsätzlich gut an einen vorgegebenen Referenzpfad anschmiegen kann. Der Einschränkungsgrad durch die diskrete Positionierbarkeit des Manipulators ist dabei abhängig von den geometrischen Parametern der einzelnen Segmente. Die Untersuchungen in dieser Arbeit zeigen, dass durch die Analyse kinematischer Leistungsmerkmale, wie Arbeitsraum(-dichte) oder erzielbarer Krümmungsradius, kein allgemeingültiges optimales Design gefunden werden kann. Daher wird eine Maßsynthese unter Berücksichtigung von Randbedingungen entworfen, in der optimale geometrische Parameter eines einzelnen binären Aktors synthetisiert werden. Darauf aufbauend wird eine Pfadverfolgung gemäß dem „Follow-the-Leader“-Prinzip erarbeitet. Grundidee ist, dass das Endeffektorsegment den Referenzpfad exploriert, während alle weiteren Aktoren dem führenden Segment automatisch folgen. Da binäre Aktoren einen nicht-kontinuierlichen Schaltprozess aufweisen, wird ein modellbasierter Ansatz für die Bestimmung optimaler Schaltsequenzen vorgeschlagen, die zu jedem Zeitpunkt eine hohe Pfadtreue garantieren. Die anschließende experimentelle Evaluation erfolgt nach der Modellierung und Identifikation relevanter Parameter für den Prototyp einer elektromagnetischen Kippaktorkette. Grundsätzlich kann die Funktionsfähigkeit der in dieser Arbeit erforschten Methoden zur Bewegungsplanung sowohl in der Simulation als auch experimentell nachgewiesen werden.The investigation of difficult to reach cavities through narrow accesses is called borescopy in the technical environment and endoscopy in medicine. If manipulation is to be performed in addition to pure inspection, a stable working platform is required to withstand manipulation forces in combination with good adaptability to curved paths. One approach to address the resulting requirements for the systems used are snake-like robots. Their hyper-redundant structure of individual actuators allows for versatile positioning. Although the use of binary, tilt-stable actuators limits the working space to a few discrete points, they offer - depending on the drive mechanism - particularly high holding torques and thus enable a targeted system stiffening. A combination of both approaches to the class of binary actuated, hyper-redundant manipulators is able to meet the required requirements, however, there is a clear need for research into methods for optimal design and the targeted pursuit of reference paths, so that the core of the present work consists the investigation of model-based motion planning of this robot class. A prerequisite for a high path following accuracy is that the manipulator is able to adapt well to a given reference path. The degree of limitation due to discrete positionability of the manipulator depends on the geometric parameters of the individual segments. The studies in this thesis show that the analysis of kinematic performance characteristics, such as work space (density) or achievable radius of curvature, does not lead to a generally valid optimal design. Therefore, a dimensional synthesis is developed under consideration of boundary conditions, in which optimal geometric parameters of a single binary actuator are synthesized. Based on this, a path following according to the "Follow-the-Leader"principle is elaborated. The basic idea is that the end effector segment explores the reference path, while all other actuators automatically follow the leading segment. Since binary actuators have a discontinuous switching process, a model-based approach is proposed for determining optimal switching sequences that guarantee high path accuracy at all times. The subsequent experimental evaluation is performed after modelling and identification of relevant parameters for the prototype of an electromagnetic tilting actuator chain. In principle, the functionality of the motion planning methods investigated in this thesis are proven both in simulation and experimentally

A Comprehensive Study of Modern Architectures and Regularization Approaches on CheXpert5000

Computer aided diagnosis (CAD) has gained an increased amount of attention in the general research community over the last years as an example of a typical limited data application - with experiments on labeled 100k-200k datasets. Although these datasets are still small compared to natural image datasets like ImageNet1k, ImageNet21k and JFT, they are large for annotated medical datasets, where 1k-10k labeled samples are much more common. There is no baseline on which methods to build on in the low data regime. In this work we bridge this gap by providing an extensive study on medical image classification with limited annotations (5k). We present a study of modern architectures applied to a fixed low data regime of 5000 images on the CheXpert dataset. Conclusively we find that models pretrained on ImageNet21k achieve a higher AUC and larger models require less training steps. All models are quite well calibrated even though we only fine-tuned on 5000 training samples. All 'modern' architectures have higher AUC than ResNet50. Regularization of Big Transfer Models with MixUp or Mean Teacher improves calibration, MixUp also improves accuracy. Vision Transformer achieve comparable or on par results to Big Transfer Models.Comment: Accepted at MICCAI 202

Sobi: An Interactive Social Service Robot for Long-Term Autonomy in Open Environments

Long-term autonomy in service robotics is a current research topic, especially for dynamic, large-scale environments that change over time. We present Sobi, a mobile service robot developed as an interactive guide for open environments, such as public places with indoor and outdoor areas. The robot will serve as a platform for environmental modeling and human-robot interaction. Its main hardware and software components, which we freely license as a documented open source project, are presented. Another key focus is Sobi’s monitoring system for long-term autonomy, which restores system components in a targeted manner in order to extend the total system lifetime without unplanned intervention. We demonstrate first results of the long-term autonomous capabilities in a 16-day indoor deployment, in which the robot patrols a total of 66.6 km with an average of 5.5 hours of travel time per weekday, charging autonomously in between. In a user study with 12 participants, we evaluate the appearance and usability of the user interface, which allows users to interactively query information about the environment and directions.© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Have I been here before? Learning to Close the Loop with LiDAR Data in Graph-Based SLAM

This work presents an extension of graph-based SLAM methods to exploit the potential of 3D laser scans for loop detection. Every high-dimensional point cloud is replaced by a compact global descriptor, whereby a trained detector decides whether a loop exists. Searching for loops is performed locally in a variable space to consider the odometry drift. Since closing a wrong loop has fatal consequences, an extensive verification is performed before acceptance. The proposed algorithm is implemented as an extension of the widely used state-of-the-art library RTAB-Map, and several experiments show the improvement: During SLAM with a mobile service robot in changing indoor and outdoor campus environments, our approach improves RTABMap regarding total number of closed loops. Especially in the presence of significant environmental changes, which typically lead to failure, localization becomes possible by our extension. Experiments with a car in traffic (KITTI benchmark) show the general applicability of our approach. These results are comparable to the state-of-the-art LiDAR method LOAM. The developed ROS package is freely available.© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Behavior-Tree-Based Person Search for Symbiotic Autonomous Mobile Robot Tasks

We consider the problem of people search by a mobile social robot in case of a situation that cannot be solved by the robot alone. Examples are physically opening a closed door or operating an elevator. Based on the Behavior Tree framework, we create a modular and easily extendable action sequence with the goal of finding a person to assist the robot. By decomposing the Behavior Tree as a Discrete Time Markov Chain, we obtain an estimate of the probability and rate of success of the options for action, especially where the robot should wait or search for people. In a real-world experiment, the presented method is compared with other common approaches in a total of 588 test runs over the course of one week, starting at two different locations in a university building. We show our method to be superior to other approaches in terms of success rate and duration until a finding person and returning to the start location.© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Intuitive Telemanipulation of Hyper-Redundant Snake Robots within Locomotion and Reorientation using Task-Priority Inverse Kinematics

Snake robots offer considerable potential for endoscopic interventions due to their ability to follow curvilinear paths. Telemanipulation is an open problem due to hyper-redundancy, as input devices only allow a specification of six degrees of freedom. Our work addresses this by presenting a unified telemanipulation strategy which enables follow-the-leader locomotion and reorientation keeping the shape change as small as possible. The basis for this is a novel shape-fitting approach for solving the inverse kinematics in only a few milliseconds. Shape fitting is performed by maximizing the similarity of two curves using Fréchet distance while simultaneously specifying the position and orientation of the end effector. Telemanipulation performance is investigated in a study in which 14 participants controlled a simulated snake robot to locomote into the target area. In a final validation, pivot reorientation within the target area is addressed.© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Trajectory Optimization Methods for Robotic Cells Considering Energy Efficiency and Collisions

In production robots are moved at maximum speed whenever possible in order to achieve the shortest overall cycle time. This can lead to individual waiting times, especially in interlinked production processes. These waiting times offer opportunities for optimization. Due to high energy prices and political efforts, energy efficiency has become the focus of trajectory optimization in recent years. Robot cells with a common intermediate circuit offer the possibility of energy exchange across individual axes or robots. By adapting the robot trajectories, the total power consumption of a robotic cell on the grid side can be significantly reduced. This paper focuses on trajectory optimization, whereby a detailed collision detection of individual robots is included within the analysis. It is shown that with collision detection energy optimization for cramped robot cells becomes possible and the losses in efficiency compared to the optimization without it are minute

Deep-learning-based instrument detection for intra-operative robotic assistance

Purpose: Robotic scrub nurses have the potential to become an attractive solution for the operating room. Surgical instrument detection is a fundamental task for these systems, which is the focus of this work. We address the detection of the complete surgery set for wisdom teeth extraction, and propose a data augmentation technique tailored for this task. Methods: Using a robotic scrub nurse system, we create a dataset of 369 unique multi-instrument images with manual annotations. We then propose the Mask-Based Object Insertion method, capable of automatically generating a large amount of synthetic images. By using both real and artificial data, different Mask R-CNN models are trained and evaluated. Results: Our experiments reveal that models trained on the synthetic data created with our method achieve comparable performance to that of models trained on real images. Moreover, we demonstrate that the combination of real and our artificial data can lead to a superior level of generalization. Conclusion: The proposed data augmentation technique is capable of dramatically reducing the labelling work required for training a deep-learning-based detection algorithm. A dataset for the complete instrument set for wisdom teeth extraction is made available for the scientific community, as well as the raw information required for the generation of the synthetic data (https://github.com/Jorebs/Deep-learning-based-instrument-detection-for-intra operative-robotic-assistance)

Stereo Laryngoscopic Impact Site Prediction for Droplet-Based Stimulation of the Laryngeal Adductor Reflex

The laryngeal adductor reflex (LAR) is a vital reflex of the human larynx. LAR malfunctions may cause life-threatening aspiration events. An objective, noninvasive, and reproducible method for LAR assessment is still lacking. Stimulation of the larynx by droplet impact, termed Microdroplet Impulse Testing of the LAR (MIT-LAR), may remedy this situation. However, droplet instability and imprecise stimulus application thus far prevented MIT-LAR from gaining clinical relevance. We present a system comprising two alternative, custom-built stereo laryngoscopes, each offering a distinct set of properties, a droplet applicator module, and image/point cloud processing algorithms to enable a targeted, droplet-based LAR stimulation. Droplet impact site prediction (ISP) is achieved by droplet trajectory identification and spatial target reconstruction. The reconstruction and ISP accuracies were experimentally evaluated. Global spatial reconstruction errors at the glottal area of (0.3±0.3) mm and (0.4±0.3) mm and global ISP errors of (0.9±0.6) mm and (1.3±0.8) mm were found for a rod lens-based and an alternative, fiberoptic laryngoscope, respectively. In the case of the rod lens-based system, 96% of all observed ISP error values are inferior to 2 mm; a value of 80% was found with the fiberoptic assembly. This contribution represents an important step towards introducing a reproducible and objective LAR screening method into the clinical routine

Tendon Actuated Continuous Structures in Planar Parallel Robots: A Kinematic Analysis

This is an accepted manuscript originally published by the American Society of Mechanical Engineers.The use of continuous and flexible structures instead of rigid links and discrete joints is a growing field of robotics research. Recent work focuses on the inclusion of continuous segments in parallel robots to benefit from their structural advantages, such as a high dexterity and compliance. While some applications and designs of these novel parallel continuum robots have been presented, the field remains largely unexplored. Furthermore, an exact quantification of the kinematic advantages and disadvantages when using continuous structures in parallel robots is yet to be performed. In this paper, planar parallel robot designs using tendon actuated continuum robots instead of rigid links and discrete joints are proposed. Using the well-known 3-RRR manipulator as a reference design, two parallel continuum robots are derived. Inverse and differential kinematics of these designs are modeled using constant curvature assumptions, which can be adapted for other actuation mechanisms than tendons. Their kinematic performances are compared to the conventional parallel robot counterpart. On the basis of this comparison, the advantages and disadvantages of using continuous structures in parallel robots are quantified and analyzed. Results show that parallel continuum robots can be kinematic equivalent and exhibit similar kinematic performances in comparison to conventional parallel robots depending on the chosen design.This work was partially supported by the German Research Foundation under award numbers BU 2935/5-1 and OR 196/35-1