Human-Robot Collaboration in Automotive Assembly

In the past decades, automation in the automobile production line has significantly increased the efficiency and quality of automotive manufacturing. However, in the automotive assembly stage, most tasks are still accomplished manually by human workers because of the complexity and flexibility of the tasks and the high dynamic unconstructed workspace. This dissertation is proposed to improve the level of automation in automotive assembly by human-robot collaboration (HRC). The challenges that eluded the automation in automotive assembly including lack of suitable collaborative robotic systems for the HRC, especially the compact-size high-payload mobile manipulators; teaching and learning frameworks to enable robots to learn the assembly tasks, and how to assist humans to accomplish assembly tasks from human demonstration; task-driving high-level robot motion planning framework to make the trained robot intelligently and adaptively assist human in automotive assembly tasks. The technical research toward this goal has resulted in several peer-reviewed publications. Achievements include: 1) A novel collaborative lift-assist robot for automotive assembly; 2) Approaches of vision-based robot learning of placing tasks from human demonstrations in assembly; 3) Robot learning of assembly tasks and assistance from human demonstrations using Convolutional Neural Network (CNN); 4) Robot learning of assembly tasks and assistance from human demonstrations using Task Constraint-Guided Inverse Reinforcement Learning (TC-IRL); 5) Robot learning of assembly tasks from non-expert demonstrations via Functional Objective-Oriented Network (FOON); 6) Multi-model sampling-based motion planning for trajectory optimization with execution consistency in manufacturing contexts. The research demonstrates the feasibility of a parallel mobile manipulator, which introduces novel conceptions to industrial mobile manipulators for smart manufacturing. By exploring the Robot Learning from Demonstration (RLfD) with both AI-based and model-based approaches, the research also improves robots’ learning capabilities on collaborative assembly tasks for both expert and non-expert users. The research on robot motion planning and control in the dissertation facilitates the safety and human trust in industrial robots in HRC

A Survey of Multi-Robot Motion Planning

Multi-robot Motion Planning (MRMP) is an active research field which has gained attention over the years. MRMP has significant roles to improve the efficiency and reliability of multi-robot system in a wide range of applications from delivery robots to collaborative assembly lines. This survey provides an overview of MRMP taxonomy, state-of-the-art algorithms, and approaches which have been developed for multi-robot systems. This study also discusses the strengths and limitations of each algorithm and their applications in various scenarios. Moreover, based on this, we can draw out open problems for future research.Comment: This is my Ph.D. comprehensive exam repor

A generalised multi-attribute task sequencing approach for robotics optical inspection systems

Robot programming usually consists of four steps: (1) task planning; (2) task sequencing; (3) path planning and (4) motion planning. Task (2) and (3–4) are strongly coupled. For example, the optimal robot path, which is function of the robot kinematics, relies on the pre-defined schedule of tasks, whose sequencing is computed based on the assumption that the travelling “cost” from one task to the next is only driven by the Euclidean distance in Cartesian space. Current methods tends to decouple the problem and sequentially compute the task sequencing in the T-space, and then compute the robot path by solving the inverse kinematics in the C-space. However, those approaches suffer the capability to reach a global optimum. This paper aims at developing a novel approach which integrates some of the key computational requirements of the path planning in the early stage of the task sequencing. Multi-attribute objectives are introduced to take into account: robot pose and reachability, data quality, obstacles avoidance, overall cycle time. The paper introduces a novel multi-attribute approach to find the optimized task sequencing via candidate poses solving inverse kinematics in the T-space. This is based on the core idea to combine T-space and C-space. The proposed solution has been tested on a vision-based inspection robot system with application to automotive body assembly systems. Results could however impact a wider area, from navigation systems, game and graph theory, to autonomous driving systems

Efficient Mission Planning for Robot Networks in Communication Constrained Environments

Many robotic systems are remotely operated nowadays that require uninterrupted connection and safe mission planning. Such systems are commonly found in military drones, search and rescue operations, mining robotics, agriculture, and environmental monitoring. Different robotic systems may employ disparate communication modalities such as radio network, visible light communication, satellite, infrared, Wi-Fi. However, in an autonomous mission where the robots are expected to be interconnected, communication constrained environment frequently arises due to the out of range problem or unavailability of the signal. Furthermore, several automated projects (building construction, assembly line) do not guarantee uninterrupted communication, and a safe project plan is required that optimizes collision risks, cost, and duration. In this thesis, we propose four pronged approaches to alleviate some of these issues: 1) Communication aware world mapping; 2) Communication preserving using the Line-of-Sight (LoS); 3) Communication aware safe planning; and 4) Multi-Objective motion planning for navigation. First, we focus on developing a communication aware world map that integrates traditional world models with the planning of multi-robot placement. Our proposed communication map selects the optimal placement of a chain of intermediate relay vehicles in order to maximize communication quality to a remote unit. We also vi propose an algorithm to build a min-Arborescence tree when there are multiple remote units to be served. Second, in communication denied environments, we use Line-of-Sight (LoS) to establish communication between mobile robots, control their movements and relay information to other autonomous units. We formulate and study the complexity of a multi-robot relay network positioning problem and propose approximation algorithms that restore visibility based connectivity through the relocation of one or more robots. Third, we develop a framework to quantify the safety score of a fully automated robotic mission where the coexistence of human and robot may pose a collision risk. A number of alternate mission plans are analyzed using motion planning algorithms to select the safest one. Finally, an efficient multi-objective optimization based path planning for the robots is developed to deal with several Pareto optimal cost attributes

Design and motion planning of body-in-white assembly cells

This paper proposes a method for the automatic and simultaneous identification of the body-in-white assembly cell design and motion plan. The method solution is based on an iterative algorithm that looks for a global optimum by iteratively identifying the optimum of three sub-problems. These sub-problems concern system layout design and motion planning for single and multi-robot systems, while collision detection is addressed. The sub-problems are handled through ad-hoc developed Mixed Integer Programming (MIP) models. The proposed solution overcomes the limitations of the current design and motion plan approaches. In fact, the design of body-in-white assembly cell and the robot motion planning are two time-expensive and interconnected activities, up to now generally managed from different human operators. The resolution of these two activities as non-interrelated could lead to an increase of the engineer-to-order time and a reduction of the solution quality. Thus, a test bed is described in order to prove the applicability of the approach

Modelling an Industrial Robot and Its Impact on Productivity

[EN] This research aims to design an efficient algorithm leading to an improvement of productivity by posing a multi-objective optimization, in which both the time consumed to carry out scheduled tasks and the associated costs of the autonomous industrial system are minimized. The algorithm proposed models the kinematics and dynamics of the industrial robot, provides collision-free trajectories, allows to constrain the energy consumed and meets the physical characteristics of the robot (i.e., restriction on torque, jerks and power in all driving motors). Additionally, the trajectory tracking accuracy is improved using an adaptive fuzzy sliding mode control (AFSMC), which allows compensating for parametric uncertainties, bounded external disturbances and constraint uncertainties. Therefore, the system stability and robustness are enhanced; thus, overcoming some of the limitations of the traditional proportional-integral-derivative (PID) controllers. The trade-offs among the economic issues related to the assembly line and the optimal time trajectory of the desired motion are analyzed using Pareto fronts. The technique is tested in different examples for a six-degrees-of-freedom (DOF) robot system. Results have proved how the use of this methodology enhances the performance and reliability of assembly lines.Llopis-Albert, C.; Rubio Montoya, FJ.; Valero Chuliá, FJ. (2021). Modelling an Industrial Robot and Its Impact on Productivity. Mathematics. 9(7):1-13. https://doi.org/10.3390/math907076911397AOYAMA, T., NISHI, T., & ZHANG, G. (2017). Production planning problem with market impact under demand uncertainty. Journal of Advanced Mechanical Design, Systems, and Manufacturing, 11(2), JAMDSM0019-JAMDSM0019. doi:10.1299/jamdsm.2017jamdsm0019Llopis-Albert, C., Rubio, F., & Valero, F. (2015). Improving productivity using a multi-objective optimization of robotic trajectory planning. Journal of Business Research, 68(7), 1429-1431. doi:10.1016/j.jbusres.2015.01.027Rubio, F., Valero, F., Sunyer, J., & Cuadrado, J. (2012). Optimal time trajectories for industrial robots with torque, power, jerk and energy consumed constraints. Industrial Robot: An International Journal, 39(1), 92-100. doi:10.1108/01439911211192538Llopis-Albert, C., Rubio, F., & Valero, F. (2018). Optimization approaches for robot trajectory planning. Multidisciplinary Journal for Education, Social and Technological Sciences, 5(1), 1. doi:10.4995/muse.2018.9867Yang, Y., Pan, J., & Wan, W. (2019). Survey of optimal motion planning. IET Cyber-Systems and Robotics, 1(1), 13-19. doi:10.1049/iet-csr.2018.0003Gasparetto, A., & Zanotto, V. (2008). A technique for time-jerk optimal planning of robot trajectories. Robotics and Computer-Integrated Manufacturing, 24(3), 415-426. doi:10.1016/j.rcim.2007.04.001Mohammed, A., Schmidt, B., Wang, L., & Gao, L. (2014). Minimizing Energy Consumption for Robot Arm Movement. Procedia CIRP, 25, 400-405. doi:10.1016/j.procir.2014.10.055Van den Berg, J., Abbeel, P., & Goldberg, K. (2011). LQG-MP: Optimized path planning for robots with motion uncertainty and imperfect state information. The International Journal of Robotics Research, 30(7), 895-913. doi:10.1177/0278364911406562Liu, S., Sun, D., & Zhu, C. (2011). Coordinated Motion Planning for Multiple Mobile Robots Along Designed Paths With Formation Requirement. IEEE/ASME Transactions on Mechatronics, 16(6), 1021-1031. doi:10.1109/tmech.2010.2070843Plaku, E., Kavraki, L. E., & Vardi, M. Y. (2010). Motion Planning With Dynamics by a Synergistic Combination of Layers of Planning. IEEE Transactions on Robotics, 26(3), 469-482. doi:10.1109/tro.2010.2047820Rubio, F., Llopis-Albert, C., Valero, F., & Suñer, J. L. (2015). Assembly Line Productivity Assessment by Comparing Optimization-Simulation Algorithms of Trajectory Planning for Industrial Robots. Mathematical Problems in Engineering, 2015, 1-10. doi:10.1155/2015/931048Rubio, F., Llopis-Albert, C., Valero, F., & Suñer, J. L. (2016). Industrial robot efficient trajectory generation without collision through the evolution of the optimal trajectory. Robotics and Autonomous Systems, 86, 106-112. doi:10.1016/j.robot.2016.09.008Llopis-Albert, C., Valero, F., Mata, V., Pulloquinga, J. L., Zamora-Ortiz, P., & Escarabajal, R. J. (2020). Optimal Reconfiguration of a Parallel Robot for Forward Singularities Avoidance in Rehabilitation Therapies. A Comparison via Different Optimization Methods. Sustainability, 12(14), 5803. doi:10.3390/su12145803Llopis-Albert, C., Valero, F., Mata, V., Escarabajal, R. J., Zamora-Ortiz, P., & Pulloquinga, J. L. (2020). Optimal Reconfiguration of a Limited Parallel Robot for Forward Singularities Avoidance. Multidisciplinary Journal for Education, Social and Technological Sciences, 7(1), 113. doi:10.4995/muse.2020.13352Yang, J., Su, H., Li, Z., Ao, D., & Song, R. (2016). Adaptive control with a fuzzy tuner for cable-based rehabilitation robot. International Journal of Control, Automation and Systems, 14(3), 865-875. doi:10.1007/s12555-015-0049-4Zhang, G., & Zhang, X. (2016). Concise adaptive fuzzy control of nonlinearly parameterized and periodically time-varying systems via small gain theory. International Journal of Control, Automation and Systems, 14(4), 893-905. doi:10.1007/s12555-015-0054-7SUTONO, S. B., ABDUL-RASHID, S. H., AOYAMA, H., & TAHA, Z. (2016). Fuzzy-based Taguchi method for multi-response optimization of product form design in Kansei engineering: a case study on car form design. Journal of Advanced Mechanical Design, Systems, and Manufacturing, 10(9), JAMDSM0108-JAMDSM0108. doi:10.1299/jamdsm.2016jamdsm0108DUBEY, A. K. (2009). Performance Optimization Control of ECH using Fuzzy Inference Application. Journal of Advanced Mechanical Design, Systems, and Manufacturing, 3(1), 22-34. doi:10.1299/jamdsm.3.22Zhang, H., Fang, H., Zhang, D., Luo, X., & Zou, Q. (2020). Adaptive Fuzzy Sliding Mode Control for a 3-DOF Parallel Manipulator with Parameters Uncertainties. Complexity, 2020, 1-16. doi:10.1155/2020/2565316Markazi, A. H. D., Maadani, M., Zabihifar, S. H., & Doost-Mohammadi, N. (2018). Adaptive Fuzzy Sliding Mode Control of Under-actuated Nonlinear Systems. International Journal of Automation and Computing, 15(3), 364-376. doi:10.1007/s11633-017-1108-5Truong, H. V. A., Tran, D. T., To, X. D., Ahn, K. K., & Jin, M. (2019). Adaptive Fuzzy Backstepping Sliding Mode Control for a 3-DOF Hydraulic Manipulator with Nonlinear Disturbance Observer for Large Payload Variation. Applied Sciences, 9(16), 3290. doi:10.3390/app9163290Li, T.-H. S., & Huang, Y.-C. (2010). MIMO adaptive fuzzy terminal sliding-mode controller for robotic manipulators. Information Sciences, 180(23), 4641-4660. doi:10.1016/j.ins.2010.08.00

Automated sequence and motion planning for robotic spatial extrusion of 3D trusses

While robotic spatial extrusion has demonstrated a new and efficient means to fabricate 3D truss structures in architectural scale, a major challenge remains in automatically planning extrusion sequence and robotic motion for trusses with unconstrained topologies. This paper presents the first attempt in the field to rigorously formulate the extrusion sequence and motion planning (SAMP) problem, using a CSP encoding. Furthermore, this research proposes a new hierarchical planning framework to solve the extrusion SAMP problems that usually have a long planning horizon and 3D configuration complexity. By decoupling sequence and motion planning, the planning framework is able to efficiently solve the extrusion sequence, end-effector poses, joint configurations, and transition trajectories for spatial trusses with nonstandard topologies. This paper also presents the first detailed computation data to reveal the runtime bottleneck on solving SAMP problems, which provides insight and comparing baseline for future algorithmic development. Together with the algorithmic results, this paper also presents an open-source and modularized software implementation called Choreo that is machine-agnostic. To demonstrate the power of this algorithmic framework, three case studies, including real fabrication and simulation results, are presented.Comment: 24 pages, 16 figure

Combining a hierarchical task network planner with a constraint satisfaction solver for assembly operations involving routing problems in a multi-robot context

This work addresses the combination of a symbolic hierarchical task network planner and a constraint satisfaction solver for the vehicle routing problem in a multi-robot context for structure assembly operations. Each planner has its own problem domain and search space, and the article describes how both planners interact in a loop sharing information in order to improve the cost of the solutions. The vehicle routing problem solver gives an initial assignment of parts to robots, making the distribution based on the distance among parts and robots, trying also to maximize the parallelism of the future assembly operations evaluating during the process the dependencies among the parts assigned to each robot. Then, the hierarchical task network planner computes a scheduling for the given assignment and estimates the cost in terms of time spent on the structure assembly. This cost value is then given back to the vehicle routing problem solver as feedback to compute a better assignment, closing the loop and repeating again the whole process. This interaction scheme has been tested with different constraint satisfaction solvers for the vehicle routing problem. The article presents simulation results in a scenario with a team of aerial robots assembling a structure, comparing the results obtained with different configurations of the vehicle routing problem solver and showing the suitability of using this approach.Unión Europea ARCAS FP7-ICT-287617Unión Europea H2020-ICT-644271Unión europea H2020-ICT-73166