Satellite pose estimation with deep landmark regression and nonlinear pose refinement

We propose an approach to estimate the 6DOF pose of a satellite, relative to a canonical pose, from a single image. Such a problem is crucial in many space proximity operations, such as docking, debris removal, and inter-spacecraft communications. Our approach combines machine learning and geometric optimisation, by predicting the coordinates of a set of landmarks in the input image, associating the landmarks to their corresponding 3D points on an a priori reconstructed 3D model, then solving for the object pose using non-linear optimisation. Our approach is not only novel for this specific pose estimation task, which helps to further open up a relatively new domain for machine learning and computer vision, but it also demonstrates superior accuracy and won the first place in the recent Kelvins Pose Estimation Challenge organised by the European Space Agency (ESA).Bo Chen, Jiewei Cao, Alvaro Parra and Tat-Jun Chi

Visual Servo Based Space Robotic Docking for Active Space Debris Removal

This thesis developed a 6DOF pose detection algorithm using machine learning capable of providing the orientation and location of an object in various lighting conditions and at different angles, for the purposes of space robotic rendezvous and docking control. The computer vision algorithm was paired with a virtual robotic simulation to test the feasibility of using the proposed algorithm for visual servo. This thesis also developed a method for generating virtual training images and corresponding ground truth data including both location and orientation information. Traditional computer vision techniques struggle to determine the 6DOF pose of an object when certain colors or edges are not found, therefore training a network is an optimal choice. The 6DOF pose detection algorithm was implemented on MATLAB and Python. The robotic simulation was implemented on Simulink and ROS Gazebo. Finally, the generation of training data was done with Python and Blender