Robotics Middleware: A Comprehensive Literature Survey and Attribute-Based Bibliography

Autonomous robots are complex systems that require the interaction between numerous heterogeneous components (software and hardware). Because of the increase in complexity of robotic applications and the diverse range of hardware, robotic middleware is designed to manage the complexity and heterogeneity of the hardware and applications, promote the integration of new technologies, simplify software design, hide the complexity of low-level communication and the sensor heterogeneity of the sensors, improve software quality, reuse robotic software infrastructure across multiple research efforts, and to reduce production costs. This paper presents a literature survey and attribute-based bibliography of the current state of the art in robotic middleware design. The main aim of the survey is to assist robotic middleware researchers in evaluating the strengths and weaknesses of current approaches and their appropriateness for their applications. Furthermore, we provide a comprehensive set of appropriate bibliographic references that are classified based on middleware attributes.http://dx.doi.org/10.1155/2012/95901

Real-Time Scheduling Method for Middleware of Industrial Automation Devices

In this study, a real-time scheduling algorithm, which supports periodic and sporadic executions with event handling, is proposed for the middleware of industrial automation devices or controllers, such as industrial robots and programmable logic controllers. When sensors and embedded controllers are included in control loops having different control periods, they should transmit their data periodically to the controllers and actuators; otherwise, fatal failure of the system including the devices could occur. The proposed scheduling algorithm manages modules, namely, the thread type (or .so type) and process type (or .exe type), for periodic execution, sporadic execution, and non-real-time execution. The program structures for the thread-type and process-type modules that can make the proposed algorithm manage the modules efficiently are suggested; then, they are applied in periodic and sporadic executions. For sporadic executions, the occurrences of events are first examined to invoke the execution modules corresponding to the events. The proposed scheduling algorithm is implemented using the Xenomai real-time operating system (OS) and Linux, and it is validated through several examples

Robotics software frameworks for multi-agent robotic systems development

Robotics is an area of research in which the paradigm of Multi-Agent Systems (MAS) can prove to be highly useful. Multi-Agent Systems come in the form of cooperative robots in a team, sensor networks based on mobile robots, and robots in Intelligent Environments, to name but a few. However, the development of Multi-Agent Robotic Systems (MARS) still presents major challenges. Over the past decade, a high number of Robotics Software Frameworks (RSFs) have appeared which propose some solutions to the most recurrent problems in robotics. Some of these frameworks, such as ROS, YARP, OROCOS, ORCA, Open-RTM, and Open-RDK, possess certain characteristics and provide the basic infrastructure necessary for the development of MARS. The contribution of this work is the identification of such characteristics as well as the analysis of these frameworks in comparison with the general-purpose Multi-Agent System Frameworks (MASFs), such as JADE and Mobile-C.Ministerio de Ciencia e Innovación TEC2009-10639-C04-02Junta de Andalucía P06-TIC-2298Junta de Andalucía P08-TIC-0386

On Software Quality-motivated Design of a Real-time Framework for Complex Robot Control Systems

Frameworks have fundamental impact on software quality of robot control systems. We propose systematic framework design aiming at high levels of support for all quality attributes that are relevant in the robotics domain. Design decisions are taken accordingly. We argue that certain areas of design are especially critical, as changing decisions there would likely require rewriting significant parts of the implementation. For these areas, quality-motivated solutions and benefits for actual applications are discussed. We illustrate and evaluate their implementations in our framework Finroc - after briefly introducing it. This includes a highly modular framework core and a well-performing, lock-free, zero-copying communication mechanism. Finroc is being used in complex and also in commercial robotic projects - which evinces that the approaches are suitable for real-world applications

EUD-MARS: End-User Development of Model-Driven Adaptive Robotics Software Systems

Empowering end-users to program robots is becoming more significant. Introducing software engineering principles into end-user programming could improve the quality of the developed software applications. For example, model-driven development improves technology independence and adaptive systems act upon changes in their context of use. However, end-users need to apply such principles in a non-daunting manner and without incurring a steep learning curve. This paper presents EUD-MARS that aims to provide end-users with a simple approach for developing model-driven adaptive robotics software. End-users include people like hobbyists and students who are not professional programmers but are interested in programming robots. EUD-MARS supports robots like hobby drones and educational humanoids that are available for end-users. It offers a tool for software developers and another one for end-users. We evaluated EUD-MARS from three perspectives. First, we used EUD-MARS to program different types of robots and assessed its visual programming language against existing design principles. Second, we asked software developers to use EUD-MARS to configure robots and obtained their feedback on strengths and points for improvement. Third, we observed how end-users explain and develop EUD-MARS programs, and obtained their feedback mainly on understandability, ease of programming, and desirability. These evaluations yielded positive indications of EUD-MARS

The Cognitive Interaction Toolkit – Improving Reproducibility of Robotic Systems Experiments

Lier F, Wienke J, Nordmann A, Wachsmuth S, Wrede S. The Cognitive Interaction Toolkit – Improving Reproducibility of Robotic Systems Experiments. In: Brugali D, Broenink JF, Kroeger T, MacDonald BA, eds. SIMPAR: International Conference on Simulation, Modeling, and Programming for Autonomous Robots. Lecture Notes in Computer Science . Vol 8810. Cham: Springer; 2014: 400-411.Research on robot systems either integrating a large number of capabilities in a single architecture or displaying outstanding perfor- mance in a single domain achieved considerable progress over the last years. Results are typically validated through experimental evaluation or demonstrated live, e.g., at robotics competitions. While common robot hardware, simulation and programming platforms yield an improved ba- sis, many of the described experiments still cannot be reproduced easily by interested researchers to confirm the reported findings. We consider this a critical challenge for experimental robotics. Hence, we address this problem with a novel process which facilitates the reproduction of robotics experiments. We identify major obstacles to experiment repli- cation and introduce an integrated approach that allows (i) aggregation and discovery of required research artifacts, (ii) automated software build and deployment, as well as (iii) experiment description, repeatable exe- cution and evaluation. We explain the usage of the introduced process along an exemplary robotics experiment and discuss our approach in the context of current ecosystems for robot programming and simulation

Axon: A Middleware for Robotics

The area of multi-robot systems and frameworks has become, in recent years, a hot research area in the field of robotics. This is attributed to the great advances made in robotic hardware, software, and the diversity of robotic systems. The need to integrate different heterogeneous robotic components and systems has led to the birth of robotic middleware. A robotic middleware is an intricate piece of software that masks the heterogeneity of underlying components and provides high-level interfaces that enable developers to make efficient use of the components. A large number of robotic middleware programs exist today. Each one comes with its own design methodologies and complexities. Up to this moment, however, there exists no unified standard for robotic middleware. Moreover, many of the middleware in use today deal with low-level and hardware aspects. This adds unnecessary complexity in research involving robotic behavior, inter-robot collaboration, and other high-level experiments which do not require prior knowledge of low-level details. In addition, the notion of structured lightweight data transfer between robots is not emphasized in existing work. This dissertation tackles the robotic middleware problem from a different perspective. The aim of this work is to develop a robust middleware that is able to handle multiple robots and clients within a laboratory environment. In the proposed middleware, a high-level representation of robots in an environment is introduced. Also, this work introduces the notion of structured and efficient data exchange as an important issue in robotic middleware research. The middleware has been designed and developed using rigorous methodologies and leading edge technologies. Moreover, the middleware’s ability to integrate different types of robots in a seamless manner, as well as its ability to accommodate multiple robots and clients, has been tested and evaluated

Scheduling Problems

Scheduling is defined as the process of assigning operations to resources over time to optimize a criterion. Problems with scheduling comprise both a set of resources and a set of a consumers. As such, managing scheduling problems involves managing the use of resources by several consumers. This book presents some new applications and trends related to task and data scheduling. In particular, chapters focus on data science, big data, high-performance computing, and Cloud computing environments. In addition, this book presents novel algorithms and literature reviews that will guide current and new researchers who work with load balancing, scheduling, and allocation problems

협업 로봇을 위한 서비스 기반과 모델 기반의 소프트웨어 개발 방법론

학위논문(박사)--서울대학교 대학원 :공과대학 전기·컴퓨터공학부,2020. 2. 하순회.가까운 미래에는 다양한 로봇이 다양한 분야에서 하나의 임무를 협력하여 수행하는 모습은 흔히 볼 수 있게 될 것이다. 그러나 실제로 이러한 모습이 실현되기에는 두 가지의 어려움이 있다. 먼저 로봇을 운용하기 위한 소프트웨어를 명세하는 기존 연구들은 대부분 개발자가 로봇의 하드웨어와 소프트웨어에 대한 지식을 알고 있는 것을 가정하고 있다. 그래서 로봇이나 컴퓨터에 대한 지식이 없는 사용자들이 여러 대의 로봇이 협력하는 시나리오를 작성하기는 쉽지 않다. 또한, 로봇의 소프트웨어를 개발할 때 로봇의 하드웨어의 특성과 관련이 깊어서, 다양한 로봇의 소프트웨어를 개발하는 것도 간단하지 않다. 본 논문에서는 상위 수준의 미션 명세와 로봇의 행위 프로그래밍으로 나누어 새로운 소프트웨어 개발 프레임워크를 제안한다. 또한, 본 프레임워크는 크기가 작은 로봇부터 계산 능력이 충분한 로봇들이 서로 군집을 이루어 미션을 수행할 수 있도록 지원한다. 본 연구에서는 로봇의 하드웨어나 소프트웨어에 대한 지식이 부족한 사용자도 로봇의 동작을 상위 수준에서 명세할 수 있는 스크립트 언어를 제안한다. 제안하는 언어는 기존의 스크립트 언어에서는 지원하지 않는 네 가지의 기능인 팀의 구성, 각 팀의 서비스 기반 프로그래밍, 동적으로 모드 변경, 다중 작업(멀티 태스킹)을 지원한다. 우선 로봇은 팀으로 그룹 지을 수 있고, 로봇이 수행할 수 있는 기능을 서비스 단위로 추상화하여 새로운 복합 서비스를 명세할 수 있다. 또한 로봇의 멀티 태스킹을 위해 '플랜' 이라는 개념을 도입하였고, 복합 서비스 내에서 이벤트를 발생시켜서 동적으로 모드가 변환할 수 있도록 하였다. 나아가 여러 로봇의 협력이 더욱 견고하고, 유연하고, 확장성을 높이기 위해, 군집 로봇을 운용할 때 로봇이 임무를 수행하는 도중에 문제가 생길 수 있으며, 상황에 따라 로봇을 동적으로 다른 행위를 수행할 수 있다고 가정한다. 이를 위해 동적으로도 팀을 구성할 수 있고, 여러 대의 로봇이 하나의 서비스를 수행하는 그룹 서비스를 지원하고, 일대 다 통신과 같은 새로운 기능을 스크립트 언어에 반영하였다. 따라서 확장된 상위 수준의 스크립트 언어는 비전문가도 다양한 유형의 협력 임무를 쉽게 명세할 수 있다. 로봇의 행위를 프로그래밍하기 위해 다양한 소프트웨어 개발 프레임워크가 연구되고 있다. 특히 재사용성과 확장성을 중점으로 둔 연구들이 최근 많이 사용되고 있지만, 대부분의 이들 연구는 리눅스 운영체제와 같이 많은 하드웨어 자원을 필요로 하는 운영체제를 가정하고 있다. 또한, 프로그램의 분석 및 성능 예측 등을 고려하지 않기 때문에, 자원 제약이 심한 크기가 작은 로봇의 소프트웨어를 개발하기에는 어렵다. 그래서 본 연구에서는 임베디드 소프트웨어를 설계할 때 쓰이는 정형적인 모델을 이용한다. 이 모델은 정적 분석과 성능 예측이 가능하지만, 로봇의 행위를 표현하기에는 제약이 있다. 그래서 본 논문에서 외부의 이벤트에 의해 수행 중간에 행위를 변경하는 로봇을 위해 유한 상태 머신 모델과 데이터 플로우 모델이 결합하여 동적 행위를 명세할 수 있는 확장된 모델을 적용한다. 그리고 딥러닝과 같이 계산량을 많이 필요로 하는 응용을 분석하기 위해, 루프 구조를 명시적으로 표현할 수 있는 모델을 제안한다. 마지막으로 여러 로봇의 협업 운용을 위해 로봇 사이에 공유되는 정보를 나타내기 위해 두 가지 모델을 사용한다. 먼저 중앙에서 공유 정보를 관리하기 위해 라이브러리 태스크라는 특별한 태스크를 통해 공유 정보를 나타낸다. 또한, 로봇이 자신의 정보를 가까운 로봇들과 공유하기 위해 멀티캐스팅을 위한 새로운 포트를 추가한다. 이렇게 확장된 정형적인 모델은 실제 로봇 코드로 자동 생성되어, 소프트웨어 설계 생산성 및 개발 효율성에 이점을 가진다. 비전문가가 명세한 스크립트 언어는 정형적인 태스크 모델로 변환하기 위해 중간 단계인 전략 단계를 추가하였다. 제안하는 방법론의 타당성을 검증하기 위해, 시뮬레이션과 여러 대의 실제 로봇을 이용한 협업하는 시나리오에 대해 실험을 진행하였다.In the near future, it will be common that a variety of robots are cooperating to perform a mission in various fields. There are two software challenges when deploying collaborative robots: how to specify a cooperative mission and how to program each robot to accomplish its mission. In this paper, we propose a novel software development framework that separates mission specification and robot behavior programming, which is called service-oriented and model-based (SeMo) framework. Also, it can support distributed robot systems, swarm robots, and their hybrid. For mission specification, a novel scripting language is proposed with the expression capability. It involves team composition and service-oriented behavior specification of each team, allowing dynamic mode change of operation and multi-tasking. Robots are grouped into teams, and the behavior of each team is defined with a composite service. The internal behavior of a composite service is defined by a sequence of services that the robots will perform. The notion of plan is applied to express multi-tasking. And the robot may have various operating modes, so mode change is triggered by events generated in a composite service. Moreover, to improve the robustness, scalability, and flexibility of robot collaboration, the high-level mission scripting language is extended with new features such as team hierarchy, group service, one-to-many communication. We assume that any robot fails during the execution of scenarios, and the grouping of robots can be made at run-time dynamically. Therefore, the extended mission specification enables a casual user to specify various types of cooperative missions easily. For robot behavior programming, an extended dataflow model is used for task-level behavior specification that does not depend on the robot hardware platform. To specify the dynamic behavior of the robot, we apply an extended task model that supports a hybrid specification of dataflow and finite state machine models. Furthermore, we propose a novel extension to allow the explicit specification of loop structures. This extension helps the compute-intensive application, which contains a lot of loop structures, to specify explicitly and analyze at compile time. Two types of information sharing, global information sharing and local knowledge sharing, are supported for robot collaboration in the dataflow graph. For global information, we use the library task, which supports shared resource management and server-client interaction. On the other hand, to share information locally with near robots, we add another type of port for multicasting and use the knowledge sharing technique. The actual robot code per robot is automatically generated from the associated task graph, which minimizes the human efforts in low-level robot programming and improves the software design productivity significantly. By abstracting the tasks or algorithms as services and adding the strategy description layer in the design flow, the mission specification is refined into task-graph specification automatically. The viability of the proposed methodology is verified with preliminary experiments with three cooperative mission scenarios with heterogeneous robot platforms and robot simulator.Chapter 1. Introduction 1 1.1 Motivation 1 1.2 Contribution 7 1.3 Dissertation Organization 9 Chapter 2. Background and Existing Research 11 2.1 Terminologies 11 2.2 Robot Software Development Frameworks 25 2.3 Parallel Embedded Software Development Framework 31 Chapter 3. Overview of the SeMo Framework 41 3.1 Motivational Examples 45 Chapter 4. Robot Behavior Programming 47 4.1 Related works 48 4.2 Model-based Task Graph Specification for Individual Robots 56 4.3 Model-based Task Graph Specification for Cooperating Robots 70 4.4 Automatic Code Generation 74 4.5 Experiments 78 Chapter 5. High-level Mission Specification 81 5.1 Service-oriented Mission Specification 82 5.2 Strategy Description 93 5.3 Automatic Task Graph Generation 96 5.4 Related works 99 5.5 Experiments 104 Chapter 6. Conclusion 114 6.1 Future Research 116 Bibliography 118 Appendices 133 요약 158Docto

State-based Safety of Component-based Medical and Surgical Robot Systems

Safety has not received sufficient attention in the medical robotics community despite a consensus of its paramount importance and the pioneering work in the early 90s. Partly because of its emergent and non-functional characteristics, it is challenging to capture and represent the design of safety features in a consistent, structured manner. In addition, significant engineering efforts are required in practice when designing and developing medical robot systems with safety. Still, academic researchers in medical robotics have to deal with safety to perform clinical studies. This dissertation presents the concept, model and architecture to reformulate safety as a visible, reusable, and verifiable property, rather than an embedded, hard-to-reuse, and hard-to-test property that is tightly coupled with the system. The concept enables reuse and structured understanding of the design of safety features, and the model allows the system designers to explicitly define and capture the run-time status of component-based systems with support for error propagation. The architecture leverages the benefits of the concept and the model by decomposing safety features into reusable mechanisms and configurable specifications. We show the concept and feasibility of the proposed methods by building an open source framework that aims to facilitate research and development of safety systems of medical robots. Using the cisst component-based framework, we empirically evaluate the proposed methods by applying the developed framework to two research systems -- one based on a commercial robot system for orthopedic surgery and another robot soon to be clinically applied for manipulation of flexible endoscopes