Forward Kinematic Modelling with Radial Basis Function Neural Network Tuned with a Novel Meta-Heuristic Algorithm for Robotic Manipulators

The complexity of forward kinematic modelling increases with the increase in the degrees of freedom for a manipulator. To reduce the computational weight and time lag for desired output transformation, this paper proposes a forward kinematic model mapped with the help of the Radial Basis Function Neural Network (RBFNN) architecture tuned by a novel meta-heuristic algorithm, namely, the Cooperative Search Optimisation Algorithm (CSOA). The architecture presented is able to automatically learn the kinematic properties of the manipulator. Learning is accomplished iteratively based only on the observation of the input–output relationship. Related simulations are carried out on a 3-Degrees of Freedom (DOF) manipulator on the Robot Operating System (ROS). The dataset created from the simulation is divided 65–35 for training–testing of the proposed model. The metrics used for model validation include spread value, cost and runtime for the training dataset, and Mean Relative Error, Normal Mean Square Error, and Mean Absolute Error for the testing dataset. A comparative analysis of the CSOA-RBFNN model is performed with an artificial neural network, support vector regression model, and with with other meta-heuristic RBFNN models, i.e., PSORBFNN and GWO-RBFNN, that show the effectiveness and superiority of the proposed technique.publishedVersio

A continuum robotic platform for endoscopic non-contact laser surgery: design, control, and preclinical evaluation

The application of laser technologies in surgical interventions has been accepted in the clinical domain due to their atraumatic properties. In addition to manual application of fibre-guided lasers with tissue contact, non-contact transoral laser microsurgery (TLM) of laryngeal tumours has been prevailed in ENT surgery. However, TLM requires many years of surgical training for tumour resection in order to preserve the function of adjacent organs and thus preserve the patient’s quality of life. The positioning of the microscopic laser applicator outside the patient can also impede a direct line-of-sight to the target area due to anatomical variability and limit the working space. Further clinical challenges include positioning the laser focus on the tissue surface, imaging, planning and performing laser ablation, and motion of the target area during surgery. This dissertation aims to address the limitations of TLM through robotic approaches and intraoperative assistance. Although a trend towards minimally invasive surgery is apparent, no highly integrated platform for endoscopic delivery of focused laser radiation is available to date. Likewise, there are no known devices that incorporate scene information from endoscopic imaging into ablation planning and execution. For focusing of the laser beam close to the target tissue, this work first presents miniaturised focusing optics that can be integrated into endoscopic systems. Experimental trials characterise the optical properties and the ablation performance. A robotic platform is realised for manipulation of the focusing optics. This is based on a variable-length continuum manipulator. The latter enables movements of the endoscopic end effector in five degrees of freedom with a mechatronic actuation unit. The kinematic modelling and control of the robot are integrated into a modular framework that is evaluated experimentally. The manipulation of focused laser radiation also requires precise adjustment of the focal position on the tissue. For this purpose, visual, haptic and visual-haptic assistance functions are presented. These support the operator during teleoperation to set an optimal working distance. Advantages of visual-haptic assistance are demonstrated in a user study. The system performance and usability of the overall robotic system are assessed in an additional user study. Analogous to a clinical scenario, the subjects follow predefined target patterns with a laser spot. The mean positioning accuracy of the spot is 0.5 mm. Finally, methods of image-guided robot control are introduced to automate laser ablation. Experiments confirm a positive effect of proposed automation concepts on non-contact laser surgery.Die Anwendung von Lasertechnologien in chirurgischen Interventionen hat sich aufgrund der atraumatischen Eigenschaften in der Klinik etabliert. Neben manueller Applikation von fasergeführten Lasern mit Gewebekontakt hat sich die kontaktfreie transorale Lasermikrochirurgie (TLM) von Tumoren des Larynx in der HNO-Chirurgie durchgesetzt. Die TLM erfordert zur Tumorresektion jedoch ein langjähriges chirurgisches Training, um die Funktion der angrenzenden Organe zu sichern und damit die Lebensqualität der Patienten zu erhalten. Die Positionierung des mikroskopis chen Laserapplikators außerhalb des Patienten kann zudem die direkte Sicht auf das Zielgebiet durch anatomische Variabilität erschweren und den Arbeitsraum einschränken. Weitere klinische Herausforderungen betreffen die Positionierung des Laserfokus auf der Gewebeoberfläche, die Bildgebung, die Planung und Ausführung der Laserablation sowie intraoperative Bewegungen des Zielgebietes. Die vorliegende Dissertation zielt darauf ab, die Limitierungen der TLM durch robotische Ansätze und intraoperative Assistenz zu adressieren. Obwohl ein Trend zur minimal invasiven Chirurgie besteht, sind bislang keine hochintegrierten Plattformen für die endoskopische Applikation fokussierter Laserstrahlung verfügbar. Ebenfalls sind keine Systeme bekannt, die Szeneninformationen aus der endoskopischen Bildgebung in die Ablationsplanung und -ausführung einbeziehen. Für eine situsnahe Fokussierung des Laserstrahls wird in dieser Arbeit zunächst eine miniaturisierte Fokussieroptik zur Integration in endoskopische Systeme vorgestellt. Experimentelle Versuche charakterisieren die optischen Eigenschaften und das Ablationsverhalten. Zur Manipulation der Fokussieroptik wird eine robotische Plattform realisiert. Diese basiert auf einem längenveränderlichen Kontinuumsmanipulator. Letzterer ermöglicht in Kombination mit einer mechatronischen Aktuierungseinheit Bewegungen des Endoskopkopfes in fünf Freiheitsgraden. Die kinematische Modellierung und Regelung des Systems werden in ein modulares Framework eingebunden und evaluiert. Die Manipulation fokussierter Laserstrahlung erfordert zudem eine präzise Anpassung der Fokuslage auf das Gewebe. Dafür werden visuelle, haptische und visuell haptische Assistenzfunktionen eingeführt. Diese unterstützen den Anwender bei Teleoperation zur Einstellung eines optimalen Arbeitsabstandes. In einer Anwenderstudie werden Vorteile der visuell-haptischen Assistenz nachgewiesen. Die Systemperformanz und Gebrauchstauglichkeit des robotischen Gesamtsystems werden in einer weiteren Anwenderstudie untersucht. Analog zu einem klinischen Einsatz verfolgen die Probanden mit einem Laserspot vorgegebene Sollpfade. Die mittlere Positioniergenauigkeit des Spots beträgt dabei 0,5 mm. Zur Automatisierung der Ablation werden abschließend Methoden der bildgestützten Regelung vorgestellt. Experimente bestätigen einen positiven Effekt der Automationskonzepte für die kontaktfreie Laserchirurgie

Real-Time Hybrid Visual Servoing of a Redundant Manipulator via Deep Reinforcement Learning

Fixtureless assembly may be necessary in some manufacturing tasks and environ-ments due to various constraints but poses challenges for automation due to non-deterministic characteristics not favoured by traditional approaches to industrial au-tomation. Visual servoing methods of robotic control could be effective for sensitive manipulation tasks where the desired end-effector pose can be ascertained via visual cues. Visual data is complex and computationally expensive to process but deep reinforcement learning has shown promise for robotic control in vision-based manipu-lation tasks. However, these methods are rarely used in industry due to the resources and expertise required to develop application-specific systems and prohibitive train-ing costs. Training reinforcement learning models in simulated environments offers a number of benefits for the development of robust robotic control algorithms by reducing training time and costs, and providing repeatable benchmarks for which algorithms can be tested, developed and eventually deployed on real robotic control environments. In this work, we present a new simulated reinforcement learning envi-ronment for developing accurate robotic manipulation control systems in fixtureless environments. Our environment incorporates a contemporary collaborative industrial robot, the KUKA LBR iiwa, with the goal of positioning its end effector in a generic fixtureless environment based on a visual cue. Observational inputs are comprised of the robotic joint positions and velocities, as well as two cameras, whose positioning reflect hybrid visual servoing with one camera attached to the robotic end-effector, and another observing the workspace respectively. We propose a state-of-the-art deep reinforcement learning approach to solving the task environment and make prelimi-nary assessments of the efficacy of this approach to hybrid visual servoing methods for the defined problem environment. We also conduct a series of experiments ex-ploring the hyperparameter space in the proposed reinforcement learning method. Although we could not prove the efficacy of a deep reinforcement approach to solving the task environment with our initial results, we remain confident that such an ap-proach could be feasible to solving this industrial manufacturing challenge and that our contributions in this work in terms of the novel software provide a good basis for the exploration of reinforcement learning approaches to hybrid visual servoing in accurate manufacturing contexts

Path and Motion Planning for Autonomous Mobile 3D Printing

Autonomous robotic construction was envisioned as early as the ‘90s, and yet, con- struction sites today look much alike ones half a century ago. Meanwhile, highly automated and efficient fabrication methods like Additive Manufacturing, or 3D Printing, have seen great success in conventional production. However, existing efforts to transfer printing technology to construction applications mainly rely on manufacturing-like machines and fail to utilise the capabilities of modern robotics. This thesis considers using Mobile Manipulator robots to perform large-scale Additive Manufacturing tasks. Comprised of an articulated arm and a mobile base, Mobile Manipulators, are unique in their simultaneous mobility and agility, which enables printing-in-motion, or Mobile 3D Printing. This is a 3D printing modality, where a robot deposits material along larger-than-self trajectories while in motion. Despite profound potential advantages over existing static manufacturing-like large- scale printers, Mobile 3D printing is underexplored. Therefore, this thesis tack- les Mobile 3D printing-specific challenges and proposes path and motion planning methodologies that allow this printing modality to be realised. The work details the development of Task-Consistent Path Planning that solves the problem of find- ing a valid robot-base path needed to print larger-than-self trajectories. A motion planning and control strategy is then proposed, utilising the robot-base paths found to inform an optimisation-based whole-body motion controller. Several Mobile 3D Printing robot prototypes are built throughout this work, and the overall path and motion planning strategy proposed is holistically evaluated in a series of large-scale 3D printing experiments

Task-space dynamic control of underwater robots

This thesis is concerned with the control aspects for underwater tasks performed by marine robots. The mathematical models of an underwater vehicle and an underwater vehicle with an onboard manipulator are discussed together with their associated properties. The task-space regulation problem for an underwater vehicle is addressed where the desired target is commonly specified as a point. A new control technique is proposed where the multiple targets are defined as sub-regions. A fuzzy technique is used to handle these multiple sub-region criteria effectively. Due to the unknown gravitational and buoyancy forces, an adaptive term is adopted in the proposed controller. An extension to a region boundary-based control law is then proposed for an underwater vehicle to illustrate the flexibility of the region reaching concept. In this novel controller, a desired target is defined as a boundary instead of a point or region. For a mapping of the uncertain restoring forces, a least-squares estimation algorithm and the inverse Jacobian matrix are utilised in the adaptive control law. To realise a new tracking control concept for a kinematically redundant robot, subregion tracking control schemes with a sub-tasks objective are developed for a UVMS. In this concept, the desired objective is specified as a moving sub-region instead of a trajectory. In addition, due to the system being kinematically redundant, the controller also enables the use of self-motion of the system to perform sub-tasks (drag minimisation, obstacle avoidance, manipulability and avoidance of mechanical joint limits)

Robotic manipulators for single access surgery

This thesis explores the development of cooperative robotic manipulators for enhancing surgical precision and patient outcomes in single-access surgery and, specifically, Transanal Endoscopic Microsurgery (TEM). During these procedures, surgeons manipulate a heavy set of instruments via a mechanical clamp inserted in the patient’s body through a surgical port, resulting in imprecise movements, increased patient risks, and increased operating time. Therefore, an articulated robotic manipulator with passive joints is initially introduced, featuring built-in position and force sensors in each joint and electronic joint brakes for instant lock/release capability. The articulated manipulator concept is further improved with motorised joints, evolving into an active tool holder. The joints allow the incorporation of advanced robotic capabilities such as ultra-lightweight gravity compensation and hands-on kinematic reconfiguration, which can optimise the placement of the tool holder in the operating theatre. Due to the enhanced sensing capabilities, the application of the active robotic manipulator was further explored in conjunction with advanced image guidance approaches such as endomicroscopy. Recent advances in probe-based optical imaging such as confocal endomicroscopy is making inroads in clinical uses. However, the challenging manipulation of imaging probes hinders their practical adoption. Therefore, a combination of the fully cooperative robotic manipulator with a high-speed scanning endomicroscopy instrument is presented, simplifying the incorporation of optical biopsy techniques in routine surgical workflows. Finally, another embodiment of a cooperative robotic manipulator is presented as an input interface to control a highly-articulated robotic instrument for TEM. This master-slave interface alleviates the drawbacks of traditional master-slave devices, e.g., using clutching mechanics to compensate for the mismatch between slave and master workspaces, and the lack of intuitive manipulation feedback, e.g. joint limits, to the user. To address those drawbacks a joint-space robotic manipulator is proposed emulating the kinematic structure of the flexible robotic instrument under control.Open Acces

Hybrid fuzzy sliding mode control for motorised space tether spin-up when coupled with axial and torsional oscillation

A specialised hybrid controller is applied to the control of a motorised space tether spin-up space coupled with an axial and a torsional oscillation phenomenon. A seven-degree-of-freedom (7-DOF) dynamic model of a motorised momentum exchange tether is used as the basis for interplanetary payload exchange in the context of control. The tether comprises a symmetrical double payload configuration, with an outrigger counter inertia and massive central facility. It is shown that including axial and torsional elasticity permits an enhanced level of performance prediction accuracy and a useful departure from the usual rigid body representations, particularly for accurate payload positioning at strategic points. A simulation with given initial condition data has been devised in a connecting programme between control code written in MATLAB and dynamics simulation code constructed within MATHEMATICA. It is shown that there is an enhanced level of spin-up control for the 7-DOF motorised momentum exchange tether system using the specialised hybrid controller. hybrid controller

Affordable flexible hybrid manipulator for miniaturised product assembly

Miniaturised assembly systems are capable of assembling parts of a few millimetres in size with an accuracy of a few micrometres. Reducing the size and the cost of such a system while increasing its flexibility and accuracy is a challenging issue. The introduction of hybrid manipulation, also called coarse/fine manipulation, within an assembly system is the solution investigated in this thesis. A micro-motion stage (MMS) is designed to be used as the fine positioning mechanism of the hybrid assembly system. MMSs often integrate compliant micro-motion stages (CMMSs) to achieve higher performances than the conventional MMSs. CMMSs are mechanisms that transmit an output force and displacement through the deformation of their structure. Although widely studied, the design and modelling techniques of these mechanisms still need to be improved and simplified. Firstly, the linear modelling of CMMSs is evaluated and two polymer prototypes are fabricated and characterised. It is found that polymer based designs have a low fabrication cost but not suitable for construction of a micro-assembly system. A simplified nonlinear model is then derived and integrated within an analytical model, allowing for the full characterisation of the CMMS in terms of stiffness and range of motion. An aluminium CMMS is fabricated based on the optimisation results from the analytical model and is integrated within an MMS. The MMS is controlled using dual-range positioning to achieve a low-cost positioning accuracy better than 2µm within a workspace of 4.4×4.4mm2. Finally, a hybrid manipulator is designed to assemble mobile-phone cameras and sensors automatically. A conventional robot manipulator is used to pick and place the parts in coarse mode while the aluminium CMMS based MMS is used for fine alignment of the parts. A high-resolution vision system is used to locate the parts on the substrate and to measure the relative position of the manipulator above MMS using a calibration grid with square patterns. The overall placement accuracy of the assembly system is ±24µm at 3σ and can reach 2µm, for a total cost of less than £50k, thus demonstrating the suitability of hybrid manipulation for desktop-size miniaturised assembly systems. The precision of the existing system could be significantly improved by making the manipulator stiffer (i.e. preloaded bearings…) and adjustable to compensate for misalignment. Further improvement could also be made on the calibration of the vision system. The system could be either scaled up or down using the same architecture while adapting the controllers to the scale.Engineering and Physical Sciences Research Council (EPSRC

Appropriate Design of Parallel Manipulators

International audienceAlthough parallel structures have found a niche market in many applications such as machine tools, telescope positioning or food packaging, they are not as successful as expected. The main reason of this relative lack of success is that the study and hardware of parallel structures have clearly not reached the same level of completeness than the one of serial structures. Among the main issues that have to be addressed, the design problem is crucial. Indeed, the performances that can be expected from a parallel robot are heavily dependent upon the choice of the mechanical structure and even more from its dimensioning. In this chapter, we show that classical design methodologies are not appropriate for such closed-loop mechanism and examine what alternatives are possible