Control of an autonomous robot using machine learning techniques implemented on a GPU device

In a world where artificial intelligence is so consolidated in many fields and in different applications, there is a need to understand how it works and how we can improve these algorithms in order to optimize them. In this project we will focus on the autonomous control of a robot by face detection. To implement this functionality, we have compared different Convolutional Neural Networks. After a detailed study of each of these models and comparing their results, it has been decided to use the FaceNet system for face detection algorithm. To improve the performance and the processing of the video frames, a GPU, Nvidia Jetson Nano, will be used. Finally, by means of the Jetson Nano board we will control the iRobot Roomba 600 robot through commands that will be sent through a serial port, which the robot will receive to manage its actuators and to be able to move following our face

Utilizing Reinforcement Learning and Computer Vision in a Pick-And-Place Operation for Sorting Objects in Motion

This master's thesis studies the implementation of advanced machine learning (ML) techniques in industrial automation systems, focusing on applying machine learning to enable and evolve autonomous sorting capabilities in robotic manipulators. In particular, Inverse Kinematics (IK) and Reinforcement Learning (RL) are investigated as methods for controlling a UR10e robotic arm for pick-and-place of moving objects on a conveyor belt within a small-scale sorting facility. A camera-based computer vision system applying YOLOv8 is used for real-time object detection and instance segmentation. Perception data is utilized to ascertain optimal grip points, specifically through an implemented algorithm that outputs optimal grip position, angle, and width. As the implemented system includes testing and evaluation on a physical system, the intricacies of hardware control, specifically the reverse engineering of an OnRobot RG6 gripper is elaborated as part of this study. The system is implemented on the Robotic Operating System (ROS), and its design is in particular driven by high modularity and scalability in mind. The camera-based vision system serves as the primary input, while the robot control is the output. The implemented system design allows for the evaluation of motion control employing both IK and RL. Computation of IK is conducted via MoveIt2, while the RL model is trained and computed in NVIDIA Isaac Sim. The high-level control of the robotic manipulator was accomplished with use of Proximal Policy Optimization (PPO). The main result of the research is a novel reward function for the pick-and-place operation that takes into account distance and orientation from the target object. In addition, the provided system administers task control by independently initializing pick-and-place operation phases for each environment. The findings demonstrate that PPO was able to significantly enhance the velocity, accuracy, and adaptability of industrial automation. Our research shows that accurate control of the robot arm can be reached by training the PPO Model purely by applying a digital twin simulation

Mechatronic design solution for planar overconstrained cable-driven parallel robot

Cable-driven parallel robots (CDPRs) combine the successful features of parallel manipulators with the benefits of cable transmissions. The payload is divided among light extendable cables, resulting in an energy-efficient system that can achieve high end-effector acceleration over a huge workspace. A CDPR is formed by a set of actuation units, and a mobile platform, working as an end-effector (EE). The cables, driven by the actuation units, are guided inside the robot workspace using a guidance system and then connected to the mobile platform. The variation of cable lengths is responsible for the EE displacement throughout the robot workspace. These features result in easily reconfigurable systems where the workspace can be modified by relocating the actuation and guidance units. Nevertheless, the use of CDPRs in industrial environments is still limited, due to the drawbacks of employing flexible cables. Indeed, cables impose unilateral constraints that can only exert tensile forces and, consequently, the EE cannot withstand any arbitrary external action. To enhance the robot’s controllability, CDPRs can be overconstrained by employing a number of cables higher than the degrees of freedom of the EE. This allows cables to pull one against the other and to keep the overall system controllable over a wide range of externally applied loads. In this thesis, an eight-cable, planar, overconstrained CDPR is designed: the robot has the deployable and reconfigurable features required by the task. In particular, this CDPR has its actuation units installed into the EE mobile platform, and the frame anchor points can be rearranged to obtain a discrete reconfiguration. The cable arrangement, location of anchor points and mechanical design have been studied, by implementing a hybrid optimisation procedure. The genetic algorithm is combined with a local minimum optimiser, maximizing the CDPR volume index and deriving a mechanical design for the prototype

sCAM: An Untethered Insertable Laparoscopic Surgical Camera Robot

Fully insertable robotic imaging devices represent a promising future of minimally invasive laparoscopic vision. Emerging research efforts in this field have resulted in several proof-of-concept prototypes. One common drawback of these designs derives from their clumsy tethering wires which not only cause operational interference but also reduce camera mobility. Meanwhile, these insertable laparoscopic cameras are manipulated without any pose information or haptic feedback, which results in open loop motion control and raises concerns about surgical safety caused by inappropriate use of force.This dissertation proposes, implements, and validates an untethered insertable laparoscopic surgical camera (sCAM) robot. Contributions presented in this work include: (1) feasibility of an untethered fully insertable laparoscopic surgical camera, (2) camera-tissue interaction characterization and force sensing, (3) pose estimation, visualization, and feedback with sCAM, and (4) robotic-assisted closed-loop laparoscopic camera control. Borrowing the principle of spherical motors, camera anchoring and actuation are achieved through transabdominal magnetic coupling in a stator-rotor manner. To avoid the tethering wires, laparoscopic vision and control communication are realized with dedicated wireless links based on onboard power. A non-invasive indirect approach is proposed to provide real-time camera-tissue interaction force measurement, which, assisted by camera-tissue interaction modeling, predicts stress distribution over the tissue surface. Meanwhile, the camera pose is remotely estimated and visualized using complementary filtering based on onboard motion sensing. Facilitated by the force measurement and pose estimation, robotic-assisted closed-loop control has been realized in a double-loop control scheme with shared autonomy between surgeons and the robotic controller.The sCAM has brought robotic laparoscopic imaging one step further toward less invasiveness and more dexterity. Initial ex vivo test results have verified functions of the implemented sCAM design and the proposed force measurement and pose estimation approaches, demonstrating the technical feasibility of a tetherless insertable laparoscopic camera. Robotic-assisted control has shown its potential to free surgeons from low-level intricate camera manipulation workload and improve precision and intuitiveness in laparoscopic imaging

Development of n-DoF Preloaded structures for impact mitigation in cobots

A core issue in collaborative robotics is that of impact mitigation, especially when collisions happen with operators. Passively compliant structures can be used as the frame of the cobot, although, usually, they are implemented by means of a single-degree-offreedom (DoF). However, n-DoF preloaded structures offer a number of advantages in terms of flexibility in designing their behavior. In this work, we propose a comprehensive framework for classifying n-DoF preloaded structures, including one-, two-, and threedimensional arrays. Furthermore, we investigate the implications of the peculiar behavior of these structures-which present sharp stiff-to-compliant transitions at designdetermined load thresholds-on impact mitigation. To this regard, an analytical n-DoF dynamic model was developed and numerically implemented. A prototype of a 10DoF structure was tested under static and impact loads, showing a very good agreement with the model. Future developments will see the application of n-DoF preloaded structures to impact-mitigation on cobots and in the field of mobile robots, as well as to the field of novel architected materials

UNMANNED GROUND VEHICLE (UGV) DOCKING, CONNECTION, AND CABLING FOR ELECTRICAL POWER TRANSMISSION IN AUTONOMOUS MOBILE MICROGRIDS

Autonomous Mobile Microgrids provide electrical power to loads in environments where humans either can not, or would prefer not to, perform the task of positioning and connecting the power grid equipment. The contributions of this work compose an architecture for electrical power transmission by Unmanned Ground Vehicles (UGV). Purpose-specific UGV docking and cable deployment software algorithms, and hardware for electrical connection and cable management, has been deployed on Clearpath Husky robots. Software development leverages Robot Operating System (ROS) tools for navigation and rendezvous of the autonomous UGV robots, with task-specific visual feedback controllers for docking validated in Monte-Carlo outdoor trials with a 73% docking rate, and application to wireless power transmission demonstrated in an outdoor environment. An “Adjustable Cable Management Mechanism” (ACMM) was designed to meet low cost, compact-platform constraints for powered deployment and retraction by a UGV of electrical cable subject to disturbance, with feed rates up to 1 m/s. A probe-and-funnel AC/DC electrical connector system was de- veloped for deployment on UGVs, which does not substantially increase the cost or complexity of the UGV, while providing a repeatable and secure method of coupling electrical contacts subject to a docking miss-alignment of up to +/-2 cm laterally and +/-15 degrees axially. Cabled power transmission is accomplished by a feed-forward/feedback control method, which utilizes visual estimation of the cable state to deploy electrical cable without tension, in the obstacle-free track of the UGV as it transverses to connect power grid nodes. Cabling control response to step-input UGV chassis velocities in the forward, reverse, and zero-point-turn maneuvers are presented, as well as outdoor cable deployment. This power transmission capability is relevant to diverse domains including military Forward-Operating-Bases, disaster response, robotic persistent operation, underwater mining, or planetary exploration

Variable-Structure Cable-Driven Parallel Robots

Cable-driven parallel robots (CDPR) are a special class of robotic manipulators consisting of a rigid end effector suspended, constrained, and actuated by a number of length-varying cables. Since cable mass is typically negligible, it allows CDPRs to be built with extremely low-inertia, enabling high-accelerations and the ability to span distances that would otherwise be impossible using rigid structures. Where CDPRs suffer is their inability to perform in cluttered installation spaces due to the need to avoid collisions between cables and the environment. This thesis proposes a design alternative defined as `variable-structure CDPRs' (VSCR) to address the inherent limitations CDPRs have regarding their limited usable workspaces in cluttered environments. What makes VSCRs unique is their ability to instantaneously alter their dynamic structure through collisions between cables and objects fixed in the environment. It is shown that, unlike traditional CDPRs, VSCRs are able to produce non-convex reachable workspaces: a property that is especially useful for circumventing obstacles and has implications for a wide range of applications such as rehabilitation, agriculture, and warehousing. An extended cable model for representing collidable cables is developed along with a corresponding inverse kinematics method as a foundation for initiating the study of VSCRs. Next, an atlas-based approach for representing VSCR configuration spaces is introduced, along with a method for its computation. The proposed representation, referred to as the `structure atlas,' is shown to be a powerful tool for performing VSCR workspace analysis and inverse kinematics. Finally, an experimental testbed is constructed and used for conducting several experimental studies to validate the previously mentioned theoretical contributions and observe the real-world capabilities of VSCRs. Mathematically and experimentally, it is shown that VSCRs dramatically improve the reachability and accessible workspaces CDPRs can achieve in cluttered or irregular environments

Visual telemetry transmission in marine environment using Robot Operating System platform

Εθνικό Μετσόβιο Πολυτεχνείο--Μεταπτυχιακή Εργασία. Διεπιστημονικό-Διατμηματικό Πρόγραμμα Μεταπτυχιακών Σπουδών (Δ.Π.Μ.Σ.) “Συστήματα Αυτοματισμού

Optimized Design of Statically Equivalent Mooring and Catenary Ryser Systems

Due to size limitations of wave basins worldwide it is necessary to employ statically equivalent truncated mooring and riser systems to test floating systems to be deployed in deep and ultra-deep waters. A procedure for the optimized design of the statically equivalent truncated mooring and riser system was developed using a Genetic Algorithm, considering that the equivalent mooring/system needs to reproduce the net static forces and moments exerted by the prototype mooring/riser system on the floater in its six rigid body degrees of freedom (surge, sway, heave, roll, pitch and yaw). A fit-for-purpose program was developed to evaluate the three-dimensional static equilibrium of floating structures, considering the attached mooring and steel catenary riser systems. The static response is calculated for a set of offsets in the surge direction from the calm water equilibrium position up to a maximum user defined offset. Four study cases were considered to demonstrate the effectiveness and robustness of a Genetic Algorithm procedure developed for the optimize design of the statically equivalent mooring and riser system. The four study cases were a semisubmersible with a symmetric polyester mooring system, a semisubmersible with a symmetric steel wire mooring system, a semisubmersible with a non-symmetric polyester mooring and steel catenary riser system attached, and a spar with a non-symmetric polyester mooring and a steel catenary riser system attached. To gain insight on the distortion of the dynamic mooring forces exerted on the floater when dynamic effects are ignored in the design, a procedure to assess the mooring system inertia and damping force contributions to the floater was developed. The application of the procedure was demonstrated using two study cases corresponding to deepwater polyester and steel mooring systems