Fast and robust image feature matching methods for computer vision applications

Service robotic systems are designed to solve tasks such as recognizing and manipulating objects, understanding natural scenes, navigating in dynamic and populated environments. It's immediately evident that such tasks cannot be modeled in all necessary details as easy as it is with industrial robot tasks; therefore, service robotic system has to have the ability to sense and interact with the surrounding physical environment through a multitude of sensors and actuators. Environment sensing is one of the core problems that limit the deployment of mobile service robots since existing sensing systems are either too slow or too expensive. Visual sensing is the most promising way to provide a cost effective solution to the mobile robot sensing problem. It's usually achieved using one or several digital cameras placed on the robot or distributed in its environment. Digital cameras are information rich sensors and are relatively inexpensive and can be used to solve a number of key problems for robotics and other autonomous intelligent systems, such as visual servoing, robot navigation, object recognition, pose estimation, and much more. The key challenges to taking advantage of this powerful and inexpensive sensor is to come up with algorithms that can reliably and quickly extract and match the useful visual information necessary to automatically interpret the environment in real-time. Although considerable research has been conducted in recent years on the development of algorithms for computer and robot vision problems, there are still open research challenges in the context of the reliability, accuracy and processing time. Scale Invariant Feature Transform (SIFT) is one of the most widely used methods that has recently attracted much attention in the computer vision community due to the fact that SIFT features are highly distinctive, and invariant to scale, rotation and illumination changes. In addition, SIFT features are relatively easy to extract and to match against a large database of local features. Generally, there are two main drawbacks of SIFT algorithm, the first drawback is that the computational complexity of the algorithm increases rapidly with the number of key-points, especially at the matching step due to the high dimensionality of the SIFT feature descriptor. The other one is that the SIFT features are not robust to large viewpoint changes. These drawbacks limit the reasonable use of SIFT algorithm for robot vision applications since they require often real-time performance and dealing with large viewpoint changes. This dissertation proposes three new approaches to address the constraints faced when using SIFT features for robot vision applications, Speeded up SIFT feature matching, robust SIFT feature matching and the inclusion of the closed loop control structure into object recognition and pose estimation systems. The proposed methods are implemented and tested on the FRIEND II/III service robotic system. The achieved results are valuable to adapt SIFT algorithm to the robot vision applications

Robotic assembly of threaded fasteners in a non-structured environment

Over the past two decades, a major part of the manufacturing and assembly market has been driven by the increasing demand for customised products. This has created the need for smaller batch sizes, shorter production times, lower costs, and the flexibility to produce families of products—or to assemble different parts—with the same sets of equipment. Consequently, manufacturing companies have deployed various automation systems and production strategies to improve their resource efficiency and move towards right-first-time production. Threaded fastening operations are widely used in assembly and are typically time-consuming and costly. In high-volume production, fastening operations are commonly automated using jigs, fixtures, and semi-automated tools. However, in low-volume, high-value manufacturing, fastening operations are carried out manually by skilled workers. The existing approaches are found to be less flexible and robust for performing assembly in a less structured industrial environment. This motivated the development of a flexible solution, which does not require fixtures and is adaptable to variation in part locations and lighting conditions. As a part of this research, a novel 3D threaded hole detection and a fast bolt detection algorithms are proposed and reported in this article, which offer substantial enhancement to the accuracy, repeatability, and the speed of the processes in comparison with the existing methods. Hence, the proposed method is more suitable for industrial applications. The development of an automated bolt fastening demonstrator is also described in this article to test and validate the proposed identification algorithms on complex components located in 3D space