14 research outputs found
Distance-based kernels for dynamical movement primitives
In the Anchoring Problem actions and objects must be anchored to symbols; and movement primitives as DMPs seems a good option to describe actions. In the bottom-up approach to anchoring, the recognition of an action is done applying learning techniques as clustering. Although most work done about movement recognition with DMPs is focus on weights, we propose to use the shape-attractor function as feature vector. As several DMPs formulations exist, we have analyzed the two most known to check if using the shape-attractor instead of weights is feasible for both formulations. In addition, we propose to use distance-based kernels, as RBF and TrE, to classify DMPs in some predefined actions. Our experiments based on an existing dataset and using 1-NN and SVM techniques confirm that shape-attractor function is a better choice for movement recognition with DMPs.Peer ReviewedPostprint (author's final draft
Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning
Telerobotic systems must adapt to new environmental conditions and deal with
high uncertainty caused by long-time delays. As one of the best alternatives to
human-level intelligence, Reinforcement Learning (RL) may offer a solution to
cope with these issues. This paper proposes to integrate RL with the Model
Mediated Teleoperation (MMT) concept. The teleoperator interacts with a
simulated virtual environment, which provides instant feedback. Whereas
feedback from the real environment is delayed, feedback from the model is
instantaneous, leading to high transparency. The MMT is realized in combination
with an intelligent system with two layers. The first layer utilizes Dynamic
Movement Primitives (DMP) which accounts for certain changes in the avatar
environment. And, the second layer addresses the problems caused by uncertainty
in the model using RL methods. Augmented reality was also provided to fuse the
avatar device and virtual environment models for the teleoperator. Implemented
on DLR's Exodex Adam hand-arm haptic exoskeleton, the results show RL methods
are able to find different solutions when changes are applied to the object
position after the demonstration. The results also show DMPs to be effective at
adapting to new conditions where there is no uncertainty involved
Unifying Skill-Based Programming and Programming by Demonstration through Ontologies
Smart manufacturing requires easily reconfigurable robotic systems to increase the flexibility in presence of market uncertainties by
reducing the set-up times for new tasks. One enabler of fast reconfigurability is given by intuitive robot programming methods. On
the one hand, offline skill-based programming (OSP) allows the definition of new tasks by sequencing pre-defined, parameterizable
building blocks termed as skills in a graphical user interface. On the other hand, programming by demonstration (PbD) is a
well known technique that uses kinesthetic teaching for intuitive robot programming, where this work presents an approach to
automatically recognize skills from the human demonstration and parameterize them using the recorded data. The approach further
unifies both programming modes of OSP and PbD with the help of an ontological knowledge base and empowers the end user to
choose the preferred mode for each phase of the task. In the experiments, we evaluate two scenarios with different sequences of
programming modes being selected by the user to define a task. In each scenario, skills are recognized by a data-driven classifier
and automatically parameterized from the recorded data. The fully defined tasks consist of both manually added and automatically
recognized skills and are executed in the context of a realistic industrial assembly environment
Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning
elerobotic systems must adapt to new environmental conditions and deal with high uncertainty caused by long-time delays. As one of the best alternatives to human-level intelligence, Reinforcement Learning (RL) may offer a solution to cope with these issues. This paper proposes to integrate RL with the Model Mediated Teleoperation (MMT) concept. The teleoperator interacts with a simulated virtual environment, which provides instant feedback. Whereas feedback from the real environment is delayed, feedback from the model is instantaneous, leading to high transparency. The MMT is realized in combination with an intelligent system with two layers. The first layer utilizes Dynamic Movement Primitives (DMP) which accounts for certain changes in the avatar environment. And, the second layer addresses the problems caused by uncertainty in the model using RL methods. Augmented reality was also provided to fuse the avatar device and virtual environment models for the teleoperator. Implemented on DLR's Exodex Adam hand-arm haptic exoskeleton, the results show RL methods are able to find different solutions when changes are applied to the object position after the demonstration. The results also show DMPs to be effective at adapting to new conditions where there is no uncertainty involved
Incremental motor skill learning and generalization from human dynamic reactions based on dynamic movement primitives and fuzzy logic system
Different from previous work on single skill learning from human demonstrations, an incremental motor skill learning, generalization and control method based on dynamic movement primitives (DMP) and broad learning system (BLS) is proposed for extracting both ordinary skills and instant reactive skills from demonstrations, the latter of which is usually generated to avoid a sudden danger (e.g., touching a hot cup). The method is completed in three steps. First, ordinary skills are basically learned from demonstrations in normal cases by using DMP. Then the incremental learning idea of BLS is combined with DMP to achieve multi-stylistic reactive skill learning such that the forcing function of the ordinary skills will be reasonably extended into multiple stylistic functions by adding enhancement terms and updating weights of the radial basis function (RBF) kernels. Finally, electromyography (EMG) signals are collected from human muscles and processed to achieve stiffness factors. By using fuzzy logic system (FLS), the two kinds of skills learned are integrated and generalized in new cases such that not only start, end and scaling factors but also the environmental conditions, robot reactive strategies and impedance control factors will be generalized to lead to various reactions. To verify the effectiveness of the proposed method, an obstacle avoidance experiment that enables robots to approach destinations flexibly in various situations with barriers will be undertaken
๋ชจ์ ํ๋ฆฌ๋จธํฐ๋ธ๋ฅผ ์ด์ฉํ ๋ณต์กํ ๋ก๋ด ์๋ฌด ํ์ต ๋ฐ ์ผ๋ฐํ ๊ธฐ๋ฒ
ํ์๋
ผ๋ฌธ (๋ฐ์ฌ) -- ์์ธ๋ํ๊ต ๋ํ์ : ๊ณต๊ณผ๋ํ ํญ๊ณต์ฐ์ฃผ๊ณตํ๊ณผ, 2020. 8. ๊นํ์ง.Learning from demonstrations (LfD) is a promising approach that enables robots to perform a specific movement. As robotic manipulations are substituting a variety of tasks, LfD algorithms are widely used and studied for specifying the robot configurations for the various types of movements.
This dissertation presents an approach based on parametric dynamic movement primitives (PDMP) as a motion representation algorithm which is one of relevant LfD techniques. Unlike existing motion representation algorithms, this work not only represents a prescribed motion but also computes the new behavior through a generalization of multiple demonstrations in the actual environment. The generalization process uses Gaussian process regression (GPR) by representing the nonlinear relationship between the PDMP parameters that determine motion and the corresponding environmental variables. The proposed algorithm shows that it serves as a powerful optimal and real-time motion planner among the existing planning algorithms when optimal demonstrations are provided as dataset.
In this dissertation, the safety of motion is also considered. Here, safety refers to keeping the system away from certain configurations that are unsafe. The safety criterion of the PDMP internal parameters are computed to check the safety. This safety criterion reflects the new behavior computed through the generalization process, as well as the individual motion safety of the demonstration set. The demonstrations causing unsafe movement are identified and removed. Also, the demolished demonstrations are replaced by proven demonstrations upon this criterion.
This work also presents an extension approach reducing the number of required demonstrations for the PDMP framework. This approach is effective where a single mission consists of multiple sub-tasks and requires numerous demonstrations in generalizing them. The whole trajectories in provided demonstrations are segmented into multiple sub-tasks representing unit motions. Then, multiple PDMPs are formed independently for correlated-segments. The phase-decision process determines which sub-task and associated PDMPs to be executed online, allowing multiple PDMPs to be autonomously configured within an integrated framework. GPR formulations are applied to obtain execution time and regional goal configuration for each sub-task.
Finally, the proposed approach and its extension are validated with the actual experiments of mobile manipulators. The first two scenarios regarding cooperative aerial transportation demonstrate the excellence of the proposed technique in terms of quick computation, generation of efficient movement, and safety assurance. The last scenario deals with two mobile manipulations using ground vehicles and shows the effectiveness of the proposed extension in executing complex missions.์์ฐ ํ์ต ๊ธฐ๋ฒ(Learning from demonstrations, LfD)์ ๋ก๋ด์ด ํน์ ๋์์ ์ํํ ์ ์๋๋ก ํ๋ ์ ๋งํ ๋์ ์์ฑ ๊ธฐ๋ฒ์ด๋ค. ๋ก๋ด ์กฐ์๊ธฐ๊ฐ ์ธ๊ฐ ์ฌํ์์ ๋ค์ํ ์
๋ฌด๋ฅผ ๋์ฒดํด ๊ฐ์ ๋ฐ๋ผ, ๋ค์ํ ์๋ฌด๋ฅผ ์ํํ๋ ๋ก๋ด์ ๋์์ ์์ฑํ๊ธฐ ์ํด LfD ์๊ณ ๋ฆฌ์ฆ๋ค์ ๋๋ฆฌ ์ฐ๊ตฌ๋๊ณ , ์ฌ์ฉ๋๊ณ ์๋ค.
๋ณธ ๋
ผ๋ฌธ์ LfD ๊ธฐ๋ฒ ์ค ๋ชจ์
ํ๋ฆฌ๋จธํฐ๋ธ ๊ธฐ๋ฐ์ ๋์ ์ฌ์์ฑ ์๊ณ ๋ฆฌ์ฆ์ธ Parametric dynamic movement primitives(PDMP)์ ๊ธฐ์ดํ ์๊ณ ๋ฆฌ์ฆ์ ์ ์ํ๋ฉฐ, ์ด๋ฅผ ํตํด ๋ค์ํ ์๋ฌด๋ฅผ ์ํํ๋ ๋ชจ๋ฐ์ผ ์กฐ์๊ธฐ์ ๊ถค์ ์ ์์ฑํ๋ค. ๊ธฐ์กด์ ๋์ ์ฌ์์ฑ ์๊ณ ๋ฆฌ์ฆ๊ณผ ๋ฌ๋ฆฌ, ์ด ์ฐ๊ตฌ๋ ์ ๊ณต๋ ์์ฐ์์ ํํ๋ ๋์์ ๋จ์ํ ์ฌ์์ฑํ๋ ๊ฒ์ ๊ทธ์น์ง ์๊ณ , ์๋ก์ด ํ๊ฒฝ์ ๋ง๊ฒ ์ผ๋ฐํ ํ๋ ๊ณผ์ ์ ํฌํจํ๋ค. ์ด ๋
ผ๋ฌธ์์ ์ ์ํ๋ ์ผ๋ฐํ ๊ณผ์ ์ PDMPs์ ๋ด๋ถ ํ๋ผ๋ฏธํฐ ๊ฐ์ธ ์คํ์ผ ํ๋ผ๋ฏธํฐ์ ํ๊ฒฝ ๋ณ์ ์ฌ์ด์ ๋น์ ํ ๊ด๊ณ๋ฅผ ๊ฐ์ฐ์ค ํ๊ท ๊ธฐ๋ฒ (Gaussian process regression, GPR)์ ์ด์ฉํ์ฌ ์์์ ์ผ๋ก ํํํ๋ค. ์ ์๋ ๊ธฐ๋ฒ์ ๋ํ ์ต์ ์์ฐ๋ฅผ ํ์ตํ๋ ๋ฐฉ์์ ํตํด ๊ฐ๋ ฅํ ์ต์ ์ค์๊ฐ ๊ฒฝ๋ก ๊ณํ ๊ธฐ๋ฒ์ผ๋ก๋ ์์ฉ๋ ์ ์๋ค.
๋ณธ ๋
ผ๋ฌธ์์๋ ๋ํ ๋ก๋ด์ ๊ตฌ๋ ์์ ์ฑ๋ ๊ณ ๋ คํ๋ค. ๊ธฐ์กด ์ฐ๊ตฌ๋ค์์ ๋ค๋ฃจ์ด์ง ์์ฐ ๊ด๋ฆฌ ๊ธฐ์ ์ด ๋ก๋ด์ ๊ตฌ๋ ํจ์จ์ฑ์ ๊ฐ์ ํ๋ ๋ฐฉํฅ์ผ๋ก ์ ์๋ ๊ฒ๊ณผ ๋ฌ๋ฆฌ, ์ด ์ฐ๊ตฌ๋ ๊ฐํ ๊ตฌ์์กฐ๊ฑด์ผ๋ก ๋ก๋ด์ ๊ตฌ๋ ์์ ์ฑ์ ํ๋ณดํ๋ ์์ฐ ๊ด๋ฆฌ ๊ธฐ์ ์ ํตํด ์์ ์ฑ์ ๊ณ ๋ คํ๋ ์๋ก์ด ๋ฐฉ์์ ์ ์ํ๋ค. ์ ์๋ ๋ฐฉ์์ ์คํ์ผ ํ๋ผ๋ฏธํฐ ๊ฐ ์์์ ์์ ์ฑ ๊ธฐ์ค์ ๊ณ์ฐํ๋ฉฐ, ์ด ์์ ๊ธฐ์ค์ ํตํด ์์ฐ์ ์ ๊ฑฐํ๋ ์ผ๋ จ์ ์์
์ ์ํํ๋ค. ๋ํ, ์ ๊ฑฐ๋ ์์๋ฅผ ์์ ๊ธฐ์ค์ ๋ฐ๋ผ ์
์ฆ๋ ์์๋ก ๋์ฒดํ์ฌ ์ผ๋ฐํ ์ฑ๋ฅ์ ์ ํ์ํค์ง ์๋๋ก ์์๋ฅผ ๊ด๋ฆฌํ๋ค. ์ด๋ฅผ ํตํด ๋ค์์ ์์ฐ ๊ฐ๊ฐ ๊ฐ๋ณ ๋์ ์์ ์ฑ ๋ฟ ์๋๋ผ ์จ๋ผ์ธ ๋์์ ์์ ์ฑ๊น์ง ๊ณ ๋ คํ ์ ์์ผ๋ฉฐ, ์ค์๊ฐ ๋ก๋ด ์กฐ์๊ธฐ ์ด์ฉ์ ์์ ์ฑ์ด ํ๋ณด๋ ์ ์๋ค. ์ ์๋ ์์ ์ฑ์ ๊ณ ๋ คํ ์์ฐ ๊ด๋ฆฌ ๊ธฐ์ ์ ๋ํ ํ๊ฒฝ์ ์ ์ ์ค์ ์ด ๋ณ๊ฒฝ๋์ด ๋ชจ๋ ์์ฐ์ ๊ต์ฒดํด์ผ ํ ์ ์๋ ์ํฉ์์ ์ฌ์ฉํ ์ ์๋ ์์ฐ๋ค์ ํ๋ณํ๊ณ , ํจ์จ์ ์ผ๋ก ์ฌ์ฌ์ฉํ๋ ๋ฐ ์์ฉํ ์ ์๋ค.
๋ํ ๋ณธ ๋
ผ๋ฌธ์ ๋ณต์กํ ์๋ฌด์์ ์ ์ฉ๋ ์ ์๋ PDMPs์ ํ์ฅ ๊ธฐ๋ฒ์ธ seg-PDMPs๋ฅผ ์ ์ํ๋ค. ์ด ์ ๊ทผ๋ฐฉ์์ ๋ณต์กํ ์๋ฌด๊ฐ ์ผ๋ฐ์ ์ผ๋ก ๋ณต์๊ฐ์ ๊ฐ๋จํ ํ์ ์์
์ผ๋ก ๊ตฌ์ฑ๋๋ค๊ณ ๊ฐ์ ํ๋ค. ๊ธฐ์กด PDMPs์ ๋ฌ๋ฆฌ seg-PDMPs๋ ์ ์ฒด ๊ถค์ ์ ํ์ ์์
์ ๋ํ๋ด๋ ์ฌ๋ฌ ๊ฐ์ ๋จ์ ๋์์ผ๋ก ๋ถํ ํ๊ณ , ๊ฐ ๋จ์๋์์ ๋ํด ์ฌ๋ฌ๊ฐ์ PDMPs๋ฅผ ๊ตฌ์ฑํ๋ค. ๊ฐ ๋จ์ ๋์ ๋ณ๋ก ์์ฑ๋ PDMPs๋ ํตํฉ๋ ํ๋ ์์ํฌ๋ด์์ ๋จ๊ณ ๊ฒฐ์ ํ๋ก์ธ์ค๋ฅผ ํตํด ์๋์ ์ผ๋ก ํธ์ถ๋๋ค. ๊ฐ ๋จ๊ณ ๋ณ๋ก ๋จ์ ๋์์ ์ํํ๊ธฐ ์ํ ์๊ฐ ๋ฐ ํ์ ๋ชฉํ์ ์ ๊ฐ์ฐ์ค ๊ณต์ ํ๊ท(GPR)๋ฅผ ์ด์ฉํ ํ๊ฒฝ๋ณ์์์์ ๊ด๊ณ์์ ํตํด ์ป๋๋ค. ๊ฒฐ๊ณผ์ ์ผ๋ก, ์ด ์ฐ๊ตฌ๋ ์ ์ฒด์ ์ผ๋ก ์๊ตฌ๋๋ ์์ฐ์ ์๋ฅผ ํจ๊ณผ์ ์ผ๋ก ์ค์ผ ๋ฟ ์๋๋ผ, ๊ฐ ๋จ์๋์์ ํํ ์ฑ๋ฅ์ ๊ฐ์ ํ๋ค.
์ ์๋ ์๊ณ ๋ฆฌ์ฆ์ ํ๋ ๋ชจ๋ฐ์ผ ๋ก๋ด ์กฐ์๊ธฐ ์คํ์ ํตํ์ฌ ๊ฒ์ฆ๋๋ค. ์ธ ๊ฐ์ง์ ์๋๋ฆฌ์ค๊ฐ ๋ณธ ๋
ผ๋ฌธ์์ ๋ค๋ฃจ์ด์ง๋ฉฐ, ํญ๊ณต ์ด์ก๊ณผ ๊ด๋ จ๋ ์ฒซ ๋ ๊ฐ์ง ์๋๋ฆฌ์ค๋ PDMPs ๊ธฐ๋ฒ์ด ๋ก๋ด ์กฐ์๊ธฐ์์ ๋น ๋ฅธ ์ ์์ฑ, ์๋ฌด ํจ์จ์ฑ๊ณผ ์์ ์ฑ ๋ชจ๋ ๋ง์กฑํ๋ ๊ฒ์ ์
์ฆํ๋ค. ๋ง์ง๋ง ์๋๋ฆฌ์ค๋ ์ง์ ์ฐจ๋์ ์ด์ฉํ ๋ ๊ฐ์ ๋ก๋ด ์กฐ์๊ธฐ์ ๋ํ ์คํ์ผ๋ก ๋ณต์กํ ์๋ฌด ์ํ์ ํ๊ธฐ ์ํด ํ์ฅ๋ ๊ธฐ๋ฒ์ธ seg-PDMPs๊ฐ ํจ๊ณผ์ ์ผ๋ก ๋ณํํ๋ ํ๊ฒฝ์์ ์ผ๋ฐํ๋ ๋์์ ์์ฑํจ์ ๊ฒ์ฆํ๋ค.1 Introduction 1
1.1 Motivations 1
1.2 Literature Survey 3
1.2.1 Conventional Motion Planning in Mobile Manipulations 3
1.2.2 Motion Representation Algorithms 5
1.2.3 Safety-guaranteed Motion Representation Algorithms 7
1.3 Research Objectives and Contributions 7
1.3.1 Motion Generalization in Motion Representation Algorithm 9
1.3.2 Motion Generalization with Safety Guarantee 9
1.3.3 Motion Generalization for Complex Missions 10
1.4 Thesis Organization 11
2 Background 12
2.1 DMPs 12
2.2 Mobile Manipulation Systems 13
2.2.1 Single Mobile Manipulation 14
2.2.2 Cooperative Mobile Manipulations 14
2.3 Experimental Setup 17
2.3.1 Test-beds for Aerial Manipulators 17
2.3.2 Test-beds for Robot Manipulators with Ground Vehicles 17
3 Motion Generalization in Motion Representation Algorithm 22
3.1 Parametric Dynamic Movement Primitives 22
3.2 Generalization Process in PDMPs 26
3.2.1 Environmental Parameters 26
3.2.2 Mapping Function 26
3.3 Simulation Results 29
3.3.1 Two-dimensional Hurdling Motion 29
3.3.2 Cooperative Aerial Transportation 30
4 Motion Generalization with Safety Guarantee 36
4.1 Safety Criterion in Style Parameter 36
4.2 Demonstration Management 39
4.3 Simulation Validation 42
4.3.1 Two-dimensional Hurdling Motion 46
4.3.2 Cooperative Aerial Transportation 47
5 Motion Generalization for Complex Missions 51
5.1 Overall Structure of Seg-PDMPs 51
5.2 Motion Segments 53
5.3 Phase-decision Process 54
5.4 Seg-PDMPs for Single Phase 54
5.5 Simulation Results 55
5.5.1 Initial/terminal Offsets 56
5.5.2 Style Generalization 59
5.5.3 Recombination 61
6 Experimental Validation and Results 63
6.1 Cooperative Aerial Transportation 63
6.2 Cooperative Mobile Hang-dry Mission 70
6.2.1 Demonstrations 70
6.2.2 Simulation Validation 72
6.2.3 Experimental Results 78
7 Conclusions 82
Abstract (in Korean) 93Docto
Bootstrapping of parameterized skills through hybrid optimization in task and policy spaces
Queiรer J, Steil JJ. Bootstrapping of parameterized skills through hybrid optimization in task and policy spaces. Frontiers in Robotics and AI. 2018;5:49.Modern robotic applications create high demands on adaptation of actions with respect to
variance in a given task. Reinforcement learning is able to optimize for these changing conditions,
but relearning from scratch is hardly feasible due to the high number of required rollouts. We
propose a parameterized skill that generalizes to new actions for changing task parameters,
which is encoded as a meta-learner that provides parameters for task-specific dynamic motion
primitives. Our work shows that utilizing parameterized skills for initialization of the optimization
process leads to a more effective incremental task learning. In addition, we introduce a hybrid
optimization method that combines a fast coarse optimization on a manifold of policy parameters
with a fine grained parameter search in the unrestricted space of actions. The proposed algorithm
reduces the number of required rollouts for adaptation to new task conditions. Application in
illustrative toy scenarios, for a 10-DOF planar arm, and a humanoid robot point reaching task
validate the approach