Earthmoving construction automation with military applications: Past, present and future

© ISARC 2018 - 35th International Symposium on Automation and Robotics in Construction and International AEC/FM Hackathon: The Future of Building Things. All rights reserved. Amongst increasing innovations in frontier engineering sciences, the advancements in Robotic and Autonomous Systems (RAS) has brought about a new horizon in construction applications. There is evidence of the increasing interest in RAS technologies in the civil construction sector being reflected in construction efforts of many military forces. In particular, Army or ground-based forces are frequently called upon to conduct construction tasks as part of military operations, tasks which could be partially or fully aided by the employment of RAS technologies. Along with recent advances in the Internet of Things (IoT) and cyber-physical system infrastructure, it is essential to examine the current maturity, technical feasibility, and affordability, as well as the challenges and future directions of the adoption and application of RAS to military construction. This paper presents a comprehensive survey and provides a contemporary and industry-independent overview on the state-of-the-art of earthmoving construction automation used in defence, spanning current world’s best practice through to that which is predicted over the coming years

자동화 굴착기를 위한 숙련자 굴착력 패턴 기반 굴착 작업 궤적 생성

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 기계공학부, 2020. 8. 이동준.In this thesis, we propose an excavation trajectory generation framework for autonomous excavators based on expert operator forcing pattern. The primary focus is to develop autonomous excavator system which is stable and guarantees a certain quantity of excavation in various surroundings. We nd the excavation trajectories based on the terrain features and the excavation forcing patterns from the excavation data of expert operators. The expert excavation trajectories are encoded with dynamic movement primitives (DMP) and learn through multilayer perceptron (MLP). The excavation trajectory is generated according to the terrain feature using the trained model. The excavator is modeled with 3-DoF rigid body system, and the excavation force on the bucket tip is estimated online by using the momentum-based disturbance observer(DOB). The estimated force is added to the DMP as a coupling term to modulate the excavation trajectory in real-time so that the estimated force can follow the expert excavation force pattern. Lastly, we verify the performance of the suggested framework through simulation and actual excavator test.본 논문에서는 자동화 굴착기를 위한 숙련자 굴착력 패턴 기반 굴착 작업 궤적 계획 프레임워크를 제시한다. 본 프레임워크는 자동화 굴착기의 다양한 작업 환경에서 숙련자와 유사하게 안정된 굴착 작업을 수행하며, 굴착량이 보장되는 작업을 하는 것이 목표이다. 우선 숙련된 굴착기 작업자들의 굴착 작업 데이터로부터 지형 특징에 기반한 작업 궤적과 굴착력 패턴을 찾아내었다. 숙련자의 굴착 궤적은 dynamic movement primitives(DMP)으로 encoding하여 neural network의 한 기법인 multi-layer perceptron(MLP)을 통해 학습하고, 학습된 모델을 기반으로 지형에 따른 굴착 궤적을 생성하였다. 굴착기를 다자유도 강체 시스템으로 모델링 하고, 실시간으로 버켓 끝단에 걸리는 굴착력을 momentum-based disturbance observer를 이용하여 추정하였다. 추정된 굴착력은 실시간으로 굴착 궤적을 재생성 하기위해 DMP에 coupling term으로 추가하였고, 이를 통해 추정되는 굴착력이 숙련자의 굴착 패턴을 따라갈 수 있도록 제어하였다. 마지막으로 제안한 프레임워크에 대해서는 시뮬레이션 실험과 실제 굴착기를 이용한 실험을 통해 정합성 검증을 진행하였다.1 Introduction 1 1.1 Motivation and Objectives 1 1.2 Related Work 2 1.3 Contribution 4 2 Preliminary 6 2.1 System Description 6 2.2 Excavator Dynamic Modeling 7 2.3 Force Estimation via Momentum Based Disturbance Observer 9 2.4 Dynamic Movement Primitives 10 3 Excavation Trajectory Generation 13 3.1 Analysis of Expert's Excavation Trajectory 13 3.2 Generate Nominal Excavation Trajectory by Imitating Expert Operator 19 3.3 Modulate Excavation Trajectory by Force Pattern of Expert Operator 22 4 Experiments 26 4.1 Excavation Simulation 26 4.1.1 Excavation on Flat and Slope Terrain 26 4.1.2 Excavation using Trajectory Generated by Incorrect Terrain Recognition 31 4.1.3 Excavation with Obstacle in the Ground 33 4.2 Excavation Test Result using Excavator 35 5 Conclusion and Future Work 40 5.1 Conclusion 40 5.2 Future Work 41Maste

Learning and Reacting with Inaccurate Prediction: Applications to Autonomous Excavation

Motivated by autonomous excavation, this work investigates solutions to a class of problem where disturbance prediction is critical to overcoming poor performance of a feedback controller, but where the disturbance prediction is intrinsically inaccurate. Poor feedback controller performance is related to a fundamental control problem: there is only a limited amount of disturbance rejection that feedback compensation can provide. It is known, however, that predictive action can improve the disturbance rejection of a control system beyond the limitations of feedback. While prediction is desirable, the problem in excavation is that disturbance predictions are prone to error due to the variability and complexity of soil-tool interaction forces. This work proposes the use of iterative learning control to map the repetitive components of excavation forces into feedforward commands. Although feedforward action shows useful to improve excavation performance, the non-repetitive nature of soil-tool interaction forces is a source of inaccurate predictions. To explicitly address the use of imperfect predictive compensation, a disturbance observer is used to estimate the prediction error. To quantify inaccuracy in prediction, a feedforward model of excavation disturbances is interpreted as a communication channel that transmits corrupted disturbance previews, for which metrics based on the sensitivity function exist. During field trials the proposed method demonstrated the ability to iteratively achieve a desired dig geometry, independent of the initial feasibility of the excavation passes in relation to actuator saturation. Predictive commands adapted to different soil conditions and passes were repeated autonomously until a pre-specified finish quality of the trench was achieved. Evidence of improvement in disturbance rejection is presented as a comparison of sensitivity functions of systems with and without the use of predictive disturbance compensation

On-Board Electronic Control Systems of Future Automated Heavy Machinery

The level of automation and wireless communication has increased in heavy machinery recently. This requires utilizing new devices and communication solutions in heavy machinery applications which involve demanding operating conditions and challenging life-cycle management. Therefore, the applied devices have to be robust and hardware architectures flexible, consisting of generic modules. In research and development projects devices that have various communication interfaces and insufficient mechanical and electrical robustness need to be applied. Although this thesis has its main focus on machines utilized as research platforms, many of the challenges are similar with commercial machines.The applicability of typical solutions for data transfer is discussed. Controller area network with a standardized higher level protocol is proposed to be applied where data signalling rates above 1 Mb/s are not required. The main benefits are the availability of robust, generic devices and well-established software tools for configuration management. Ethernet can be utilized to network equipment with high data rates, typically used for perception. Although deterministic industrial Ethernet protocols would fulfil most requirements, the conventional internet protocol suite is likely to be applied due to device availability.Sometimes sensors and other devices without a suitable communication interface need to be applied. In addition, device-related real-time processing or accurate synchronization of hardware signals may be required. A small circuit board with a microcontroller can be utilized as a generic embedded module for building robust, small and cost-efficient prototype devices that have a controller area network interface. Although various microcontroller boards are commercially available, designing one for heavy machinery applications, in particular, has benefits in robustness, size, interfaces, and flexible software development. The design of such a generic embedded module is presented.The device-specific challenges of building an automated machine are discussed. Unexpected switch-off of embedded computers has to be prevented by the control system to avoid file system errors. Moreover, the control system has to protect the batteries against deep discharge when the engine is not running. With many devices, protective enclosures with heating or cooling are required.The electronic control systems of two automated machines utilized as research platforms are presented and discussed as examples. The hardware architectures of the control systems are presented, following the proposed communication solutions as far as is feasible. Several applications of the generic embedded module within the control systems are described. Several research topics have been covered utilizing the automated machines. In this thesis, a cost-efficient operator-assisting functionality of an excavator is presented and discussed in detail.The results of this thesis give not only research institutes but also machine manufacturers and their subcontractors an opportunity to streamline the prototyping of automated heavy machinery

An Augmented Interaction Strategy For Designing Human-Machine Interfaces For Hydraulic Excavators

Lack of adequate information feedback and work visibility, and fatigue due to repetition have been identified as the major usability gaps in the human-machine interface (HMI) design of modern hydraulic excavators that subject operators to undue mental and physical workload, resulting in poor performance. To address these gaps, this work proposed an innovative interaction strategy, termed “augmented interaction”, for enhancing the usability of the hydraulic excavator. Augmented interaction involves the embodiment of heads-up display and coordinated control schemes into an efficient, effective and safe HMI. Augmented interaction was demonstrated using a framework consisting of three phases: Design, Implementation/Visualization, and Evaluation (D.IV.E). Guided by this framework, two alternative HMI design concepts (Design A: featuring heads-up display and coordinated control; and Design B: featuring heads-up display and joystick controls) in addition to the existing HMI design (Design C: featuring monitor display and joystick controls) were prototyped. A mixed reality seating buck simulator, named the Hydraulic Excavator Augmented Reality Simulator (H.E.A.R.S), was used to implement the designs and simulate a work environment along with a rock excavation task scenario. A usability evaluation was conducted with twenty participants to characterize the impact of the new HMI types using quantitative (task completion time, TCT; and operating error, OER) and qualitative (subjective workload and user preference) metrics. The results indicated that participants had a shorter TCT with Design A. For OER, there was a lower error probability due to collisions (PER1) with Design A, and lower error probability due to misses (PER2)with Design B. The subjective measures showed a lower overall workload and a high preference for Design B. It was concluded that augmented interaction provides a viable solution for enhancing the usability of the HMI of a hydraulic excavator

Design, Development, and Evaluation of a Teleoperated Master-Slave Surgical System for Breast Biopsy under Continuous MRI Guidance

The goal of this project is to design and develop a teleoperated master-slave surgical system that can potentially assist the physician in performing breast biopsy with a magnetic resonance imaging (MRI) compatible robotic system. MRI provides superior soft-tissue contrast compared to other imaging modalities such as computed tomography or ultrasound and is used for both diagnostic and therapeutic procedures. The strong magnetic field and the limited space inside the MRI bore, however, restrict direct means of breast biopsy while performing real-time imaging. Therefore, current breast biopsy procedures employ a blind targeting approach based on magnetic resonance (MR) images obtained a priori. Due to possible patient involuntary motion or inaccurate insertion through the registration grid, such approach could lead to tool tip positioning errors thereby affecting diagnostic accuracy and leading to a long and painful process, if repeated procedures are required. Hence, it is desired to develop the aforementioned teleoperation system to take advantages of real-time MR imaging and avoid multiple biopsy needle insertions, improving the procedure accuracy as well as reducing the sampling errors. The design, implementation, and evaluation of the teleoperation system is presented in this dissertation. A MRI-compatible slave robot is implemented, which consists of a 1 degree of freedom (DOF) needle driver, a 3-DOF parallel mechanism, and a 2-DOF X-Y stage. This slave robot is actuated with pneumatic cylinders through long transmission lines except the 1-DOF needle driver is actuated with a piezo motor. Pneumatic actuation through long transmission lines is then investigated using proportional pressure valves and controllers based on sliding mode control are presented. A dedicated master robot is also developed, and the kinematic map between the master and the slave robot is established. The two robots are integrated into a teleoperation system and a graphical user interface is developed to provide visual feedback to the physician. MRI experiment shows that the slave robot is MRI-compatible, and the ex vivo test shows over 85%success rate in targeting with the MRI-compatible robotic system. The success in performing in vivo animal experiments further confirm the potential of further developing the proposed robotic system for clinical applications

Haptic communication for remote mobile and manipulator robot operations in hazardous environments

Nuclear decommissioning involves the use of remotely deployed mobile vehicles and manipulators controlled via teleoperation systems. Manipulators are used for tooling and sorting tasks, and mobile vehicles are used to locate a manipulator near to the area that it is to be operated upon and also to carry a camera into a remote area for monitoring and assessment purposes. Teleoperations in hazardous environments are often hampered by a lack of visual information. Direct line of sight is often only available through small, thick windows, which often become discoloured and less transparent over time. Ideal camera locations are generally not possible, which can lead to areas of the cell not being visible, or at least difficult to see. Damage to the mobile, manipulator, tool or environment can be very expensive and dangerous. Despite the advances in the recent years of autonomous systems, the nuclear industry prefers generally to ensure that there is a human in the loop. This is due to the safety critical nature of the industry. Haptic interfaces provide a means of allowing an operator to control aspects of a task that would be difficult or impossible to control with impoverished visual feedback alone. Manipulator endeffector force control and mobile vehicle collision avoidance are examples of such tasks. Haptic communication has been integrated with both a Schilling Titan II manipulator teleoperation system and Cybermotion K2A mobile vehicle teleoperation system. The manipulator research was carried out using a real manipulator whereas the mobile research was carried out in simulation. Novel haptic communication generation algorithms have been developed. Experiments have been conducted using both the mobile and the manipulator to assess the performance gains offered by haptic communication. The results of the mobile vehicle experiments show that haptic feedback offered performance improvements in systems where the operator is solely responsible for control of the vehicle. However in systems where the operator is assisted by semi autonomous behaviour that can perform obstacle avoidance, the advantages of haptic feedback were more subtle. The results from the manipulator experiments served to support the results from the mobile vehicle experiments since they also show that haptic feedback does not always improve operator performance. Instead, performance gains rely heavily on the nature of the task, other system feedback channels and operator assistance features. The tasks performed with the manipulator were peg insertion, grinding and drilling.EThOS - Electronic Theses Online ServiceGBUnited Kingdo