Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation

Robotic systems are ever more capable of automation and fulfilment of complex tasks, particularly with reliance on recent advances in intelligent systems, deep learning and artificial intelligence. However, as robots and humans come closer in their interactions, the matter of interpretability, or explainability of robot decision-making processes for the human grows in importance. A successful interaction and collaboration will only take place through mutual understanding of underlying representations of the environment and the task at hand. This is currently a challenge in deep learning systems. We present a hierarchical deep reinforcement learning system, consisting of a low-level agent handling the large actions/states space of a robotic system efficiently, by following the directives of a high-level agent which is learning the high-level dynamics of the environment and task. This high-level agent forms a representation of the world and task at hand that is interpretable for a human operator. The method, which we call Dot-to-Dot, is tested on a MuJoCo-based model of the Fetch Robotics Manipulator, as well as a Shadow Hand, to test its performance. Results show efficient learning of complex actions/states spaces by the low-level agent, and an interpretable representation of the task and decision-making process learned by the high-level agent

A study of FPGA-based System-on-Chip designs for real-time industrial application

This paper shows the benefits of the Field Programming Gate Array (FPGAs) in industrial control applications. The author starts by addressing the benefits of FPGA and where it is useful. As well as, the author has done some FPGA’s evaluation researches on the FPGA performing explaining the performance of the FPGA and the design tools. To show the benefits of the FPGA, an industrial application example has been used. The application is a real-time face detection and tracking using FPGA. Face tracking will depend on calculating the centroid of each detected region. A DE2-SoC Altera board has been used to implement this application. The application based on few algorithms that filter the captured images to detect them. These algorithms have been translated to a Verilog code to run it on the DE2-SoC boar

FPGAs in Industrial Control Applications

The aim of this paper is to review the state-of-the-art of Field Programmable Gate Array (FPGA) technologies and their contribution to industrial control applications. Authors start by addressing various research fields which can exploit the advantages of FPGAs. The features of these devices are then presented, followed by their corresponding design tools. To illustrate the benefits of using FPGAs in the case of complex control applications, a sensorless motor controller has been treated. This controller is based on the Extended Kalman Filter. Its development has been made according to a dedicated design methodology, which is also discussed. The use of FPGAs to implement artificial intelligence-based industrial controllers is then briefly reviewed. The final section presents two short case studies of Neural Network control systems designs targeting FPGAs

Implementasi Kontrol Bilateral Untuk 1 Dof Haptic Manipulator

Dalam melaksanakan pekerjaannya, manusia seringkali menemui kendala antara lain kendala jarak dan tingkat keamanan kondisi lingkungan kerja. Teknologi telerobotik yang memiliki fungsi haptic menjadi salah satu solusi permasalahan tersebut. Pada penelitian ini dibuat rancang bangun sistem 1 DOF haptic manipulator dengan konfigurasi master slave menggunakan kontrol bilateral haptic manipulator. Pada kontrol bilateral posisi dan informasi ditransmisikan secara dua arah antara dua manipulator. Informasi posisi didapatkan menggunakan potensiometer dan informasi gaya didapatkan dengan merepresentasikan gaya dengan arus menggunakan sensor WCS2702. Terdapat tiga metode kontrol yang digunakan, yaitu kontrol bilateral posisi simetris, kontrol bilateral refleksi gaya dan kontrol bilateral umpan balik gaya. Metode kontrol dengan hasil terbaik yaitu kontrol bilateral umpan balik gaya. Pada sistem kontrol bilateral umpan balik gaya, slave dapat bergerak mengikuti pergerakan master dengan error posisi ±2⁰. Ketika lengan slave ditahan obyek keras, lengan master juga tertahan dengan error posisi ±11⁰. Sedangkan ketika lengan slave ditahan obyek lunak, pantulan obyek dirasakan lengan master dengan error posisi ±10⁰. Pergerakan lengan master dan slave lebih ringan dibandingkan menggunakan kontrol bilateral posisi simetris dan kontrol bilateral refleksi gaya. ======================================================================================================== In daily life, human come across various problems in their works, such as far distance between workplaces, and the safety level of the working environment. One of the solution for those problems is telerobotic technology with haptic function. In this research, the design of 1 DOF haptic manipulator is realized using bilateral control. Master slave configuration is used in the manipulator. For the bilateral control, position and information are transmitted bidirectionally between two manipulators. The position is measured using a potentiometer and force is measured by electrical current representing the force using WCS2702 sensor. Three control methods are applied, namely simetriscal position bilateral control, force reflection bilateral control, and force feedback bilateral control. Among the three, force feedback bilateral control produces the best result of which slave is able to copy the movement of the master with position error of ±2o. Moreover when the arm of the slave is obstructed by a solid object, the arm of the master is also stop with a position error of ±11o. Furthermore when the arm of the slave is obstructed by a soft object, stiffness of the object is also sensed by user operating the master with a position error of ±10o. Last, but not least, the arm movement of both manipulators are lighter than when simetriscal position bilateral control and force reflection bilateral control are employed

A field programmable gate array based motion control platform

The expectations from motion control systems have been rising day by day. As the system becomes more complex, conventional motion control systems can not achieve to meet all the specifications with optimized results. This creates the need of re-designing the control platform in order to meet the new specifications. Field programmable gate arrays (FPGA) offer reconfigurable hardware, which would result in overcoming this re-designing issue. The hardware structure of the system can be reconfigured, even though the hardware is deployed. As the functionality is provided by the hardware, the performance is enhanced. The dedicated hardware also improves the power consumption. The board size also shrinks, as the discrete components can be implemented in FPGA. The shrinkage of the board size also lowers the cost. As a trade-off, FPGA programming is more complicated than software programming. The aim of this thesis is to create a level of abstraction in order to diminish the requirement of advanced hardware description language knowledge for implementing motion control algorithms on FPGA's. The hardware library is introduced which is specifically implemented for motion control purposes. In order to have a thorough motion control platform, other parts of the system like, user interface, kinematics calculations and trajectory generation, have been implemented as a software library. The control algorithms are tested, and the system is verified by experimenting on a parallel mechanism

Online model estimation and haptic characterization for robotic-assisted minimally invasive surgery

Online soft tissue characterization is important for robotic-assisted minimally invasive surgery (RAMIS) to achieve a precise and stable robotic control with haptic feedback. The traditional linear regression method (i.e. the recursive least square (RLS) method) is inappropriate to handle nonlinear Hunt-Crossley (H-C) model since its linearization process involves unacceptable errors. This thesis presents a new nonlinear estimation method for online soft tissue characterization. To deal with nonlinear and dynamic conditions involved in soft tissue characterization, the approach expands the nonlinearity and dynamics of the H-C model by treating parameter p as an independent variable. Based on this, an unscented Kalman filter (UKF) was adapted for online nonlinear soft tissue characterization. A comparison analysis of the UKF and RLS methods was conducted to validate the performance of the UKF-based method. The UKF-based method suffers from two major problems. The first one is that it requires prior noise statistics of the corresponding system to be precisely known. However, due to uncertainties in the dynamic environment of RAMIS, it is difficult to accurately describe noise characteristics. This leads to biased or even divergent UKF solutions. Therefore, in order to attain accurate estimation results from the UKF-based approach, it is necessary to estimate noise statistics online to restrain the disturbance of noise uncertainty. Secondly, the UKF performance depends on the pre-defined system and measurement models. If the models involve stochastic errors, the UKF-based solution will be unstable. In fact, the measurement model’s accuracy can be guaranteed by using high-precision measurement equipment together with a high volume of available measurement data. On the other hand, the system model is more often involved with the inaccuracy problem. In RAMIS, the system model is a theoretical approximation of the physical contact between robotic tool and biological soft tissue. The approximation is intended to fulfil the requirement of real-time performance in RAMIS. Therefore, it is essential to improve the UKF performance in the presence of system model (the contact model) uncertainty. To address the UKF problem for inaccurate noise statistics, this thesis further presents a new recursive adaptive UKF (RAUKF) method for online nonlinear soft tissue characterization. It was developed, based on the H-C model, to estimate system noise statistics in real-time with windowing approximation. The method was developed under the condition that system noises are of small variation. In order to account for the inherent relationship between the current and previous states of soft tissue deformation involved in RAMIS, a recursive formulation was further constructed by introducing a fading scaling factor. This factor was further modified to accommodate noise statistics of a large variation, which may be caused by rupture events or geometric discontinuities in RAMIS. Simulations and comparison analyses verified the performance of the proposed RAUKF. The second UKF limitation regarding the requirement of the accurate system model was also addressed. A random weighting strong tracking unscented Kalman filter (RWSTUKF) was developed based on the Hunt-Crossley model for online nonlinear soft tissue characterization. This RWSTUKF overcomes the problem of performance degradation in the UKF due to system model errors. It adopts a scaling factor in the predicted state covariance to compensate the inaccuracy of the system model. This scaling factor was derived by combining the orthogonality principle with the random weighting concept to prevent the cumbersome computation from Jacobian matrix and offer the reliable estimation for innovation covariances. Simulation and comparison analyses demonstrated that the proposed RWSTUKF can characterise soft tissue parameters in the presence of system model error for RAMIS in on online mode. Using the proposed methods, a master-slave robotic system has been developed with a nonlinear state observer for soft tissue characterization. Robotic indentation and needle insertion tests conducted to evaluate performances of the proposed methods. Further, a rupture detection approach was established based on the RWSTUKF. It was also integrated into the master-slave robotic system to detect rupture events occurred during needle insertion. The experiment results demonstrated that the RWSTUKF outperforms RLS, UKF and RAUKF for soft tissue characterization

Design og styring av smarte robotsystemer for applikasjoner innen biovitenskap: biologisk prøvetaking og jordbærhøsting

This thesis aims to contribute knowledge to support fully automation in life-science applications, which includes design, development, control and integration of robotic systems for sample preparation and strawberry harvesting, and is divided into two parts. Part I shows the development of robotic systems for the preparation of fungal samples for Fourier transform infrared (FTIR) spectroscopy. The first step in this part developed a fully automated robot for homogenization of fungal samples using ultrasonication. The platform was constructed with a modified inexpensive 3D printer, equipped with a camera to distinguish sample wells and blank wells. Machine vision was also used to quantify the fungi homogenization process using model fitting, suggesting that homogeneity level to ultrasonication time can be well fitted with exponential decay equations. Moreover, a feedback control strategy was proposed that used the standard deviation of local homogeneity values to determine the ultrasonication termination time. The second step extended the first step to develop a fully automated robot for the whole process preparation of fungal samples for FTIR spectroscopy by adding a newly designed centrifuge and liquid-handling module for sample washing, concentration and spotting. The new system used machine vision with deep learning to identify the labware settings, which frees the users from inputting the labware information manually. Part II of the thesis deals with robotic strawberry harvesting. This part can be further divided into three stages. i) The first stage designed a novel cable-driven gripper with sensing capabilities, which has high tolerance to positional errors and can reduce picking time with a storage container. The gripper uses fingers to form a closed space that can open to capture a fruit and close to push the stem to the cutting area. Equipped with internal sensors, the gripper is able to control a robotic arm to correct for positional errors introduced by the vision system, improving the robustness. The gripper and a detection method based on color thresholding were integrated into a complete system for strawberry harvesting. ii) The second stage introduced the improvements and updates to the first stage where the main focus was to address the challenges in unstructured environment by introducing a light-adaptive color thresholding method for vision and a novel obstacle-separation algorithm for manipulation. At this stage, the new fully integrated strawberry-harvesting system with dual-manipulator was capable of picking strawberries continuously in polytunnels. The main scientific contribution of this stage is the novel obstacle-separation path-planning algorithm, which is fundamentally different from traditional path planning where obstacles are typically avoided. The algorithm uses the gripper to push aside surrounding obstacles from an entrance, thus clearing the way for it to swallow the target strawberry. Improvements were also made to the gripper, the arm, and the control. iii) The third stage improved the obstacle-separation method by introducing a zig-zag push for both horizontal and upward directions and a novel dragging operation to separate upper obstacles from the target. The zig-zag push can help the gripper capture a target since the generated shaking motion can break the static contact force between the target and obstacles. The dragging operation is able to address the issue of mis-capturing obstacles located above the target, in which the gripper drags the target to a place with fewer obstacles and then pushes back to move the obstacles aside for further detachment. The separation paths are determined by the number and distribution of obstacles based on the downsampled point cloud in the region of interest.Denne avhandlingen tar sikte på å bidra med kunnskap om automatisering og robotisering av applikasjoner innen livsvitenskap. Avhandlingen er todelt, og tar for seg design, utvikling, styring og integrering av robotsystemer for prøvetaking og jordbærhøsting. Del I omhandler utvikling av robotsystemer til bruk under forberedelse av sopprøver for Fourier-transform infrarød (FTIR) spektroskopi. I første stadium av denne delen ble det utviklet en helautomatisert robot for homogenisering av sopprøver ved bruk av ultralyd-sonikering. Plattformen ble konstruert ved å modifisere en billig 3D-printer og utstyre den med et kamera for å kunne skille prøvebrønner fra kontrollbrønner. Maskinsyn ble også tatt i bruk for å estimere soppens homogeniseringsprosess ved hjelp av matematisk modellering, noe som viste at homogenitetsnivået faller eksponensielt med tiden. Videre ble det foreslått en strategi for regulering i lukker sløyfe som brukte standardavviket for lokale homogenitetsverdier til å bestemme avslutningstidspunkt for sonikeringen. I neste stadium ble den første plattformen videreutviklet til en helautomatisert robot for hele prosessen som forbereder prøver av sopprøver for FTIR-spektroskopi. Dette ble gjort ved å legge til en nyutviklet sentrifuge- og væskehåndteringsmodul for vasking, konsentrering og spotting av prøver. Det nye systemet brukte maskinsyn med dyp læring for å identifisere innstillingene for laboratorieutstyr, noe som gjør at brukerne slipper å registrere innstillingene manuelt.Norwegian University of Life SciencespublishedVersio