Transfer Learning for Unseen Robot Detection and Joint Estimation on a Multi-Objective Convolutional Neural Network

A significant problem of using deep learning techniques is the limited amount of data available for training. There are some datasets available for the popular problems like item recognition and classification or self-driving cars, however, it is very limited for the industrial robotics field. In previous work, we have trained a multi-objective Convolutional Neural Network (CNN) to identify the robot body in the image and estimate 3D positions of the joints by using just a 2D image, but it was limited to a range of robots produced by Universal Robots (UR). In this work, we extend our method to work with a new robot arm - Kuka LBR iiwa, which has a significantly different appearance and an additional joint. However, instead of collecting large datasets once again, we collect a number of smaller datasets containing a few hundred frames each and use transfer learning techniques on the CNN trained on UR robots to adapt it to a new robot having different shapes and visual features. We have proven that transfer learning is not only applicable in this field, but it requires smaller well-prepared training datasets, trains significantly faster and reaches similar accuracy compared to the original method, even improving it on some aspects.Comment: Regular paper submission to 2018 IEEE International Conference on Intelligence and Safety Robotics (ISR). Camera Ready pape

Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks

Many works in collaborative robotics and human-robot interaction focuses on identifying and predicting human behaviour while considering the information about the robot itself as given. This can be the case when sensors and the robot are calibrated in relation to each other and often the reconfiguration of the system is not possible, or extra manual work is required. We present a deep learning based approach to remove the constraint of having the need for the robot and the vision sensor to be fixed and calibrated in relation to each other. The system learns the visual cues of the robot body and is able to localise it, as well as estimate the position of robot joints in 3D space by just using a 2D color image. The method uses a cascaded convolutional neural network, and we present the structure of the network, describe our own collected dataset, explain the network training and achieved results. A fully trained system shows promising results in providing an accurate mask of where the robot is located and a good estimate of its joints positions in 3D. The accuracy is not good enough for visual servoing applications yet, however, it can be sufficient for general safety and some collaborative tasks not requiring very high precision. The main benefit of our method is the possibility of the vision sensor to move freely. This allows it to be mounted on moving objects, for example, a body of the person or a mobile robot working in the same environment as the robots are operating in.Comment: Submission for IEEE AIM 2018 conference, 7 pages, 7 figures, ROBIN group, University of Osl

Multi-Objective Convolutional Neural Networks for Robot Localisation and 3D Position Estimation in 2D Camera Images

The field of collaborative robotics and human-robot interaction often focuses on the prediction of human behaviour, while assuming the information about the robot setup and configuration being known. This is often the case with fixed setups, which have all the sensors fixed and calibrated in relation to the rest of the system. However, it becomes a limiting factor when the system needs to be reconfigured or moved. We present a deep learning approach, which aims to solve this issue. Our method learns to identify and precisely localise the robot in 2D camera images, so having a fixed setup is no longer a requirement and a camera can be moved. In addition, our approach identifies the robot type and estimates the 3D position of the robot base in the camera image as well as 3D positions of each of the robot joints. Learning is done by using a multi-objective convolutional neural network with four previously mentioned objectives simultaneously using a combined loss function. The multi-objective approach makes the system more flexible and efficient by reusing some of the same features and diversifying for each objective in lower layers. A fully trained system shows promising results in providing an accurate mask of where the robot is located and an estimate of its base and joint positions in 3D. We compare the results to our previous approach of using cascaded convolutional neural networks.Comment: Ubiquitous Robots 2018 Regular paper submissio

UWB in 3D Indoor Positioning and Base Station Calibration

There are several technologies available for object locating and tracking in outdoor and indoor environments but performance requirements are getting tighter and precise object tracking is still largely an open challenge for researchers. Ultra wideband technology (UWB) has been identified as one of the most promising techniques to enhance a mobile node with accurate ranging and tracking capabilities. For indoor applications almost all positioning technologies require physical installation of fixed infrastructure. This infrastructure is usually expensive to deploy and maintain. The aim of this thesis is to improve the accessibility of the RF-positioning systems by lowering the configuration cost. Real time localisation and tracking systems (RTLS) based on RF technologies pose challenges especially for the deployment of positioning system over large areas or throughout buildings within a number of rooms. If calibration is done manually by providing information about the exact position of the base stations, the initial set-up is particularly time consuming and laborious. In this thesis a method for estimating the position and orientation (x, y, z, yaw, pitch and roll) of a base station of a real time localization system is presented. The algorithm uses two-dimensional Angle of Arrival information (i.e. azimuth and elevation measurements). This allows more inaccurate manual initial survey of the base stations and improves the final accuracy of the positioning. The thesis presents an implementation of the algorithm, simulations and empirical results. In the experiments, hardware and software procured from Ubisense was used. The Ubisense RTLS bases on UWB technology and utilises Angle of Arrival and Time Difference of Arrival techniques. Performance and functionality of the Ubisense RTLS were measured in various radio environments as well as the implementation of the calibration algorithm. Simulations and experiment studies showed that camera calibration method can be successfully adapted to position systems based on UWB technology and that the base stations can be calibrated in a sufficient accuracy. Because of more flexible calibration, the final positioning accuracy of the Ubisense system was as whole in average better.fi=Opinnäytetyö kokotekstinä PDF-muodossa.|en=Thesis fulltext in PDF format.|sv=Lärdomsprov tillgängligt som fulltext i PDF-format