1 research outputs found
ํด์ ์์ ์ง์์ ์ ์์จ ์ดํญ ๋ฐ ์ค์น ์์ ์ง์์ ์ํ ์๋ฎฌ๋ ์ด์ ๋ฐฉ๋ฒ
ํ์๋
ผ๋ฌธ (๋ฐ์ฌ)-- ์์ธ๋ํ๊ต ๋ํ์ : ๊ณต๊ณผ๋ํ ์กฐ์ ํด์๊ณตํ๊ณผ, 2019. 2. ๋
ธ๋ช
์ผ.Autonomous ships have gained a huge amount of interest in recent years, like their counterparts on land{autonomous cars, because of their potential to significantly lower the cost of operation, attract seagoing professionals and increase transportation safety. Technologies developed for the autonomous ships have potential to notably reduce maritime accidents where 75% cases can be attributed to human error and a significant proportion of these are caused by fatigue and attention deficit. However, developing a high-level autonomous system which can operate in an unstructured and unpredictable environment is still a challenging task. When the autonomous ships are operating in the congested waterway with other manned or unmanned vessels, the collision avoidance algorithm is the crucial point in keeping the safety of both the own ship and any encountered ships. Instead of developing new traffic rules for the autonomous ships to avoid collisions with each other, autonomous ships are expected to follow the existing guidelines based on the International Regulations for Preventing Collisions at Sea (COLREGs). Furthermore, when using the crane on the autonomous ship to transfer and install subsea equipment to the seabed, the heave and swaying phenomenon of the subsea equipment at the end of flexible wire ropes makes its positioning at an exact position is very difficult. As a result, an Anti-Motion Control (AMC) system for the crane is necessary to ensure the successful installation operation.
The autonomous ship is highly relying on the effectiveness of autonomous systems such as autonomous path following system, collision avoidance system, crane control system and so on. During the previous two decades, considerable attention has been paid to develop robust autonomous systems. However, several are facing challenges and it is worthwhile devoting much effort to this. First of all, the development and testing of the proposed control algorithms should be adapted across a variety of environmental conditions including wave, wind, and current. This is one of the challenges of this work aimed at creating an autonomous path following and collision avoidance system in the ship. Secondly, the collision avoidance system has to comply with the regulations and rules in developing an autonomous ship. Thirdly, AMC system with anti-sway abilities for a knuckle boom crane remains problems regarding its under-actuated mechanism. At last, the performance of the control system should be evaluated in advance of the operation to perform its function successfully. In particular, such performance analysis is often very costly and time-consuming, and realistic
conditions are typically impossible to establish in a testing environment.
Consequently, to address these issues, we proposed a simulation framework with the following scenarios, which including the autonomous navigation scenario and crane operation scenario. The research object of this study is an autonomous offshore support vessel (OSV), which provides support services to offshore oil and gas field development such as offshore drilling, pipe laying, and oil producing assets (production platforms and FPSOs) utilized in EP (Exploration Production) activities.
Assume that the autonomous OSV confronts an urgent mission under the
harsh environmental conditions: on the way to an imperative offshore construction site, the autonomous OSV has to avoid target ships while following a predefined path. When arriving at the construction site, it starts to install a piece of subsea equipment on the seabed. So what technologies are needed, what should be invested for ensuring the autonomous OSV could robustly kilometers from shore, and how can an autonomous OSV be made at least as safe as the conventional ship. In this dissertation, we focus on the above critical activities
for answering the above questions.
In the general context of the autonomous navigation and crane control problem, the objective of this dissertation is thus fivefold:
โข Developing a COLREGs-compliant collision avoidance system.
โข Building a robust path following and collision avoidance system which can handle the unknown and complicated environment.
โข Investigating an efficient multi-ship collision avoidance method enable it easy to extend.
โข Proposing a hardware-in-the-loop simulation environment for the AHC system.
โข Solving the anti-sway problem of the knuckle boom crane on an autonomous OSV.
First of all, we propose a novel deep reinforcement learning (RL) algorithm to achieve effective and efficient capabilities of the path following and collision avoidance system. To perform and verify the proposed algorithm, we conducted simulations for an autonomous ship under unknown environmental disturbance
iiito adjust its heading in real-time. A three-degree-of-freedom dynamic model of the autonomous ship was developed, and the Line-of-sight (LOS) guidance system was used to converge the autonomous ship to follow the predefined path. Then, a proximal policy optimization (PPO) algorithm was implemented on the problem. By applying the advanced deep RL method, in which the autonomous OSV learns the best behavior through repeated trials to determine a safe and economical avoidance behavior in various circumstances. The simulation results showed that the proposed algorithm has the capabilities to guarantee collision avoidance of moving encountered ships while ensuring following a predefined path. Also, the algorithm demonstrated that it could manage complex scenarios with various encountered ships in compliance with COLREGs and have the
excellent adaptability to the unknown, sophisticated environment.
Next, the AMC system includes Anti-Heave Control (AHC) and Anti-Sway Control (ASC), which is applied to install subsea equipment in regular and irregular for performance analysis. We used the proportional-integral-derivative (PID) control method and the sliding mode control method respectively to
achieve the control objective. The simulation results show that heave and sway motion could be significantly reduced by the proposed control methods during the construction. Moreover, to evaluate the proposed control system, we have constructed the HILS environment for the AHC system, then conducted a performance analysis of it. The simulation results show the AHC system could be evaluated effectively within the HILS environment. We can conclude that the proposed or adopted methods solve the problems issued in autonomous system design.ํด์ ์์
์ง์์ (Offshore Support Vessel: OSV)์ ๊ฒฝ์ฐ ๊ทนํ์ ํ๊ฒฝ์๋ ๋ถ๊ตฌํ๊ณ ์ถํญํ์ฌ ํด์์์ ์์
์ ์ํํด์ผ ํ๋ ๊ฒฝ์ฐ๊ฐ ์๋ค. ์ด๋ฌํ ์ํ์์ ๋
ธ์ถ์ ์ต์ํํ๊ธฐ ์ํด ์์จ ์ดํญ์ ๋ํ ์๊ตฌ๊ฐ ์ฆ๊ฐํ๊ณ ์๋ค. ์ฌ๊ธฐ์์ ์์จ ์ดํญ์ ์ ๋ฐ์ด ์ถ๋ฐ์ง์์ ๋ชฉ์ ์ง๊น์ง ์ฌ๋์ ๋์ ์์ด ์ด๋ํจ์ ์๋ฏธํ๋ค. ์์จ ์ดํญ ๋ฐฉ๋ฒ์ ๊ฒฝ๋ก ์ถ์ข
๋ฐฉ๋ฒ๊ณผ ์ถฉ๋ ํํผ ๋ฐฉ๋ฒ์ ํฌํจํ๋ค. ์ฐ์ , ์ดํญ ๋ฐ ์์
์ค ํ๊ฒฝ ํ์ค (๋ฐ๋, ํ๋, ์กฐ๋ฅ ๋ฑ)์ ๋ํ ๊ณ ๋ ค๋ฅผ ํด์ผ ํ๊ณ , ๊ตญ์ ํด์ ์ถฉ๋ ์๋ฐฉ ๊ท์น (Convention of the International Regulations for Preventing Collisions at Sea, COLREGs)์ ์ํ ์ ๋ฐ๊ฐ์ ํญ๋ฒ ๊ท์ ์ ๊ณ ๋ คํ์ฌ ์ถฉ๋ ํํผ ๊ท์น์ ์ค์ํด์ผ ํ๋ค. ํนํ ์ฐ๊ทผํด์ ๋ณต์กํ ํด์ญ์์๋ ๋ง์ ์ ๋ฐ์ ์๋์ผ๋ก ํํผํ ํ์๊ฐ ์๋ค. ๊ธฐ์กด์ ํด์์ ์ธ ๋ฐฉ๋ฒ์ ์ฌ์ฉํ๊ธฐ ์ํด์๋ ์ ๋ฐ๋ค์ ๋ํ ์ ํํ ์์คํ
๋ชจ๋ธ๋ง์ด ๋์ด์ผ ํ๋ฉฐ, ๊ทธ ๊ณผ์ ์์ ๊ฒฝํ (experience)์ ์์กดํ๋ ํ๋ผ๋ฏธํฐ ํ๋์ด ํ์์ ์ด๋ค. ๋ํ, ํํผํด์ผ ํ ์ ๋ฐ ์๊ฐ ๋ง์์ง ๊ฒฝ์ฐ ์์คํ
๋ชจ๋ธ์ด ์ปค์ง๊ฒ ๋๊ณ ๊ณ์ฐ ์๊ณผ ๊ณ์ฐ ์๊ฐ์ด ๋์ด๋ ์ค์๊ฐ ์ ์ฉ์ด ์ด๋ ต๋ค๋ ๋จ์ ์ด ์๋ค. ๋ํ, ๊ฒฝ๋ก ์ถ์ข
๋ฐ ์ถฉ๋ ํํผ๋ฅผ ํฌํจํ์ฌ ์์จ ์ดํญ ๋ฐฉ๋ฒ์ ์ ์ฉํ๊ธฐ๊ฐ ์ด๋ ต๋ค. ๋ฐ๋ผ์ ๋ณธ ์ฐ๊ตฌ์์๋ ๊ฐํ ํ์ต (Reinforcement Learning: RL) ๊ธฐ๋ฒ์ ์ด์ฉํ์ฌ ๊ธฐ์กด ํด์์ ์ธ ๋ฐฉ๋ฒ์ ๋ฌธ์ ์ ์ ๊ทน๋ณตํ ์ ์๋ ๋ฐฉ๋ฒ์ ์ ์ํ์๋ค. ๊ฒฝ๋ก๋ฅผ ์ถ์ข
ํ๋ ์ ๋ฐ (agent)์ ์ธ๋ถ ํ๊ฒฝ (environment)๊ณผ ์ํธ์์ฉํ๋ฉด์ ํ์ต์ ์งํํ๋ค. State S_0 (์ ๋ฐ์ ์์ง์๊ณผ ๊ด๋ จ๋ ๊ฐ์ข
์ํ) ๊ฐ์ง๋ agent๋ policy (ํ์ฌ ์์น์์ ์ด๋ค ์์ง์์ ์ ํํ ๊ฒ์ธ๊ฐ)์ ๋ฐ๋ผ action A_0 (์์ง์ผ ๋ฐฉํฅ) ์ทจํ๋ค. ์ด์ environment๋ agent์ ๋ค์ state S_1 ์ ๊ณ์ฐํ๊ณ , ๊ทธ์ ๋ฐ๋ฅธ ๋ณด์ R_0 (ํด๋น ์์ง์์ ์ ํฉ์ฑ)์ ๊ฒฐ์ ํ์ฌ agent์๊ฒ ์ ๋ฌํ๋ค. ์ด๋ฌํ ์์
์ ๋ฐ๋ณตํ๋ฉด์ ๋ณด์์ด ์ต๋๊ฐ ๋๋๋ก policy๋ฅผ ํ์ตํ๊ฒ ๋๋ค.
ํํธ, ํด์์์ ํฌ๋ ์ธ์ ์ด์ฉํ ์ฅ๋น์ ์ด๋์ด๋ ์ค์น ์์
์ ์ํ์ ์ค์ด๊ธฐ ์ํด ํฌ๋ ์ธ์ ๊ฑฐ๋ ์ ์ด์ ๋ํ ์๊ตฌ๊ฐ ์ฆ๊ฐํ๊ณ ์๋ค. ํนํ ํด์์์๋ ์ ๋ฐ์ ์ด๋์ ์ํด ํฌ๋ ์ธ์ ๋งค๋ฌ๋ฆฐ ๋ฌผ์ฒด๊ฐ ์ํ ๋์ (heave)์ ํฌ๋ ์ธ์ ๊ธฐ์ค์ผ๋ก ์ข์ฐ ๋์ (sway)๊ฐ ๋ฐ์ํ๋๋ฐ, ์ด๋ฌํ ์ด๋์ ์์
์ ์ง์ฐ์ํค๊ณ , ์ ํํ ์์น์ ๋ฌผ์ฒด๋ฅผ ๋์ง ๋ชปํ๊ฒ ํ๋ฉฐ, ์์นซ ์ฃผ๋ณ ๊ตฌ์กฐ๋ฌผ๊ณผ์ ์ถฉ๋์ ์ผ๊ธฐํ ์ ์๋ค. ์ด์ ๊ฐ์ ๋์๋ฅผ ์ต์ํํ๋ Anti-Motion Control (AMC) ์์คํ
์ Anti-Heave Control (AHC)๊ณผ Anti-Sway Control (ASC)์ ํฌํจํ๋ค. ๋ณธ ์ฐ๊ตฌ์์๋ ํด์ ์์
์ง์์ ์ ์ ํฉํ AMC ์์คํ
์ ์ค๊ณ ๋ฐ ๊ฒ์ฆ ๋ฐฉ๋ฒ์ ์ฐ๊ตฌํ์๋ค. ๋จผ์ ์ํ ๋์๋ฅผ ์ต์ํํ๊ธฐ ์ํด ํฌ๋ ์ธ์ ์์ด์ด ๊ธธ์ด๋ฅผ ๋ฅ๋์ ์ผ๋ก ์กฐ์ ํ๋ AHC ์์คํ
์ ์ค๊ณํ์๋ค. ๋ํ, ๊ธฐ์กด์ ์ ์ด ์์คํ
์ ๊ฒ์ฆ ๋ฐฉ๋ฒ์ ์ค์ ์ ๋ฐ์ด๋ ํด์ ๊ตฌ์กฐ๋ฌผ์ ํด๋น ์ ์ด ์์คํ
์ ์ง์ ์ค์นํ๊ธฐ ์ ์๋ ๊ทธ ์ฑ๋ฅ์ ํ
์คํธํ๊ธฐ๊ฐ ํ๋ค์๋ค. ์ด๋ฅผ ํด๊ฒฐํ๊ธฐ ์ํด ๋ณธ ์ฐ๊ตฌ์์๋ Hardware-In-the-Loop Simulation (HILS) ๊ธฐ๋ฒ์ ํ์ฉํ์ฌ AHC ์์คํ
์ ๊ฒ์ฆ ๋ฐฉ๋ฒ์ ์ฐ๊ตฌํ์๋ค. ๋ํ, ASC ์์คํ
์ ์ค๊ณํ ๋ ์ ์ด ๋์์ด under-actuated ์์คํ
์ด๊ธฐ ๋๋ฌธ์ ์ ์ดํ๊ธฐ๊ฐ ๋งค์ฐ ์ด๋ ต๋ค. ๋ฐ๋ผ์ ๋ณธ ์ฐ๊ตฌ์์๋ sliding mode control ์๊ณ ๋ฆฌ์ฆ์ ์ด์ฉํ๋ฉฐ ๋ค๊ด์ ํฌ๋ ์ธ (knuckle boom crane)์ ๊ด์ (joint) ๊ฐ๋๋ฅผ ์ ์ดํ์ฌ ์ข์ฐ ๋์๋ฅผ ์ค์ผ ์ ์๋ ASC ์์คํ
์ ์ค๊ณํ์๋ค.Chapter 1 Introduction 1
1.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . 1
1.2 Requirements for Autonomous Operation . . . . . . . . . . . . . 5
1.2.1 Path Following for Autonomous Ship . . . . . . . . . . . . 5
1.2.2 Collision Avoidance for Autonomous Ship . . . . . . . . . 5
1.2.3 Anti-Motion Control System for Autonomous Ship . . . . 6
1.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.1 Related Work for Path Following System . . . . . . . . . 9
1.3.2 Related Work for Collision Avoidance System . . . . . . . 9
1.3.3 Related Work for Anti-Heave Control System . . . . . . . 13
1.3.4 Related Work for Anti-Sway Control System . . . . . . . 14
1.4 Configuration of Simulation Framework . . . . . . . . . . . . . . 16
1.4.1 Application Layer . . . . . . . . . . . . . . . . . . . . . . 16
1.4.2 Autonomous Ship Design Layer . . . . . . . . . . . . . . . 17
1.4.3 General Technique Layer . . . . . . . . . . . . . . . . . . 17
1.5 Contributions (Originality) . . . . . . . . . . . . . . . . . . . . . 19
Chapter 2 Theoretical Backgrounds 20
2.1 Maneuvering Model for Autonomous Ship . . . . . . . . . . . . . 20
2.1.1 Kinematic Equation for Autonomous Ship . . . . . . . . . 20
2.1.2 Kinetic Equation for Autonomous Ship . . . . . . . . . . 21
2.2 Multibody Dynamics Model for Knuckle Boom Crane of Autonomous Ship. . . 25
2.2.1 Embedding Techniques . . . . . . . . . . . . . . . . . . . . 25
2.3 Control System Design . . . . . . . . . . . . . . . . . . . . . . . . 31
2.3.1 Proportional-Integral-Derivative (PID) Control . . . . . . 31
2.3.2 Sliding Mode Control . . . . . . . . . . . . . . . . . . . . 31
2.4 Deep Reinforcement Learning Algorithm . . . . . . . . . . . . . . 34
2.4.1 Value Based Learning Method . . . . . . . . . . . . . . . 36
2.4.2 Policy Based Learning Method . . . . . . . . . . . . . . . 37
2.4.3 Actor-Critic Method . . . . . . . . . . . . . . . . . . . . . 41
2.5 Hardware-in-the-Loop Simulation . . . . . . . . . . . . . . . . . . 43
2.5.1 Integrated Simulation Method . . . . . . . . . . . . . . . 43
Chapter 3 Path Following Method for Autonomous OSV 46
3.1 Guidance System . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.1.1 Line-of-sight Guidance System . . . . . . . . . . . . . . . 46
3.2 Deep Reinforcement Learning for Path Following System . . . . . 50
3.2.1 Deep Reinforcement Learning Setup . . . . . . . . . . . . 50
3.2.2 Neural Network Architecture . . . . . . . . . . . . . . . . 56
3.2.3 Training Process . . . . . . . . . . . . . . . . . . . . . . . 58
3.3 Implementation and Simulation Result . . . . . . . . . . . . . . . 62
3.3.1 Implementation for Path Following System . . . . . . . . 62
3.3.2 Simulation Result . . . . . . . . . . . . . . . . . . . . . . 65
3.4 Comparison Results . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.4.1 Comparison Result of PPO with PID . . . . . . . . . . . 83
3.4.2 Comparison Result of PPO with Deep Q-Network (DQN) 87
Chapter 4 Collision Avoidance Method for Autonomous OSV 89
4.1 Deep Reinforcement Learning for Collision Avoidance System . . 89
4.1.1 Deep Reinforcement Learning Setup . . . . . . . . . . . . 89
4.1.2 Neural Network Architecture . . . . . . . . . . . . . . . . 93
4.1.3 Training Process . . . . . . . . . . . . . . . . . . . . . . . 94
4.2 Implementation and Simulation Result . . . . . . . . . . . . . . . 95
4.2.1 Implementation for Collision Avoidance System . . . . . . 95
4.2.2 Simulation Result . . . . . . . . . . . . . . . . . . . . . . 100
4.3 Implementation and Simulation Result for Multi-ship Collision Avoidance Method . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.3.1 Limitations of Multi-ship Collision Avoidance Method - 1 107
4.3.2 Limitations of Multi-ship Collision Avoidance Method - 2 108
4.3.3 Implementation of Multi-ship Collision Avoidance Method 110
4.3.4 Simulation Result of Multi-ship Collision Avoidance Method 118
Chapter 5 Anti-Motion Control Method for Knuckle Boom Crane 129
5.1 Configuration of HILS for Anti-Heave Control System . . . . . . 129
5.1.1 Virtual Mechanical System . . . . . . . . . . . . . . . . . 132
5.1.2 Virtual Sensor and Actuator . . . . . . . . . . . . . . . . 138
5.1.3 Control System Design . . . . . . . . . . . . . . . . . . . . 141
5.1.4 Integrated Simulation Interface . . . . . . . . . . . . . . . 142
5.2 Implementation and Simulation Result of HILS for Anti-Heave Control System . . . . . . . . 145
5.2.1 Implementation of HILS for Anti-Heave Control System . 145
5.2.2 Simulation Result of HILS for Anti-Heave Control System 146
5.3 Validation of HILS for Anti-Heave Control System . . . . . . . . 159
5.3.1 Hardware Setup . . . . . . . . . . . . . . . . . . . . . . . 159
5.3.2 Comparison Result . . . . . . . . . . . . . . . . . . . . . . 161
5.4 Configuration of Anti-Sway Control System . . . . . . . . . . . . 162
5.4.1 Mechanical System for Knuckle Boom Crane . . . . . . . 162
5.4.2 Anti-Sway Control System Design . . . . . . . . . . . . . 165
5.4.3 Implementation and Simulation Result of Anti-Sway Control . . . . . . . . . . . . . . 168
Chapter 6 Conclusions and Future Works 176
Bibliography 178
Chapter A Appendix 186
๊ตญ๋ฌธ์ด๋ก 188Docto