Bin-Picking Solution for Randomly Placed Automotive Connectors Based on Machine Learning Techniques

This paper presents the development of a bin-picking solution based on low-cost vision systems for the manipulation of automotive electrical connectors using machine learning techniques. The automotive sector has always been in a state of constant growth and change, which also implies constant challenges in the wire harnesses sector, and the emerging growth of electric cars is proof of this and represents a challenge for the industry. Traditionally, this sector is based on strong human work manufacturing and the need arises to make the digital transition, supported in the context of Industry 4.0, allowing the automation of processes and freeing operators for other activities with more added value. Depending on the car model and its feature packs, a connector can interface with a different number of wires, but the connector holes are the same. Holes not connected with wires need to be sealed, mainly to guarantee the tightness of the cable. Seals are inserted manually or, more recently, through robotic stations. Due to the huge variety of references and connector configurations, layout errors sometimes occur during seal insertion due to changed references or problems with the seal insertion machine. Consequently, faulty connectors are dumped into boxes, piling up different types of references. These connectors are not trash and need to be reused. This article proposes a bin-picking solution for classification, selection and separation, using a two-finger gripper, of these connectors for reuse in a new operation of removal and insertion of seals. Connectors are identified through a 3D vision system, consisting of an Intel RealSense camera for object depth information and the YOLOv5 algorithm for object classification. The advantage of this approach over other solutions is the ability to accurately detect and grasp small objects through a low-cost 3D camera even when the image resolution is low, benefiting from the power of machine learning algorithms.info:eu-repo/semantics/publishedVersio

Framework for extracting and solving combination puzzles

Selles töös uuritakse, kuidas arvuti nägemisega seotud algoritme on võimalik rakendada objektide tuvastuse probleemile. Täpsemalt, kas arvuti nägemist on võimalik kasutada päris maailma kombinatoorsete probleemide lahendamiseks. Idee kasutada arvuti rakendust probleemide lahendamiseks, tulenes tähelepanekust, et probleemide lahenduse protsessid on kõik enamasti algoritmid. Sellest võib järeldada, et arvutid sobivad algoritmiliste probleemide lahendamiseks paremini kui inimesed, kellel võib sama ülesande peale kuluda kordades kauem. Siiski ei vaatle arvutid probleeme samamoodi nagu inimesed ehk nad ei saa probleeme analüüsida. Niisiis selle töö panuseks saab olema erinevate arvuti nägemise algoritmide uurimine, mille eesmärgiks on päris maailma kombinatoorsete probleemide tõlgendamine abstraktseteks struktuurideks, mida arvuti on võimeline mõistma ning lahendama.Praegu on antud valdkonnas vähe materiali, mis annab hea võimaluse panustada sellesse valdkonda. Seda saavutatakse läbi empiirilise uurimise testide kogumiku kujul selleks, et veenduda millised lähenemised on kõige paremad. Nende eesmärkide saavutamiseks töötati läbi suur hulk arvuti nägemisega seotud materjale ning teooriat. Lisaks võeti ka arvesse reaalaja toimingute tähtsus, mida võib näha erinevate liikumisest struktuuri eraldavate algoritmide(SLAM, PTAM) õpingutest, mida hiljem edukalt kasutati navigatsiooni ja liitreaalsuse probleemide lahendamiseks. Siiski tuleb mainida, et neid algoritme ei kasutatud objektide omaduste tuvastamiseks.See töö uurib, kuidas saab erinevaid lähenemisi kasutada selleks, et aidata vähekogenud kasutajaid kombinatoorsete päris maailma probleemide lahendamisel. Lisaks tekib selle töö tulemusena võimalus tuvastada objektide liikumist (translatsioon, pöörlemine), mida saab kasutada koos virutaalse probleemi mudeliga, et parandada kasutaja kogemust.This thesis describes and investigates how computer vision algorithms and stereo vision algorithms may be applied to the problem of object detection. In particular, if computer vision can aid on puzzle solving. The idea to use computer application for puzzle solving came from the fact that all solution techniques are algorithms in the end. This fact leads to the conclusion that algorithms are well solved by machines, for instance, a machine requires milliseconds to compute the solution while a human can handle this in minutes or hours. Unfortunately, machines cannot see puzzles from human perspective thus cannot analyze them. Hence, the contribution of this thesis is to study different computer vision approaches from non-related solutions applied to the problem of translating the physical puzzle model into the abstract structure that can be understood and solved by a machine.Currently, there is a little written on this subject, therefore, there is a great chance to contribute. This is achieved through empirical research represented as a set of experiments in order to ensure which approaches are suitable. To accomplish these goals huge amount of computer vision theory has been studied. In addition, the relevance of real-time operations was taken into account. This was manifested through the Different real-time Structure from Motion algorithms (SLAM, PTAM) studies that were successfully applied for navigation or augmented reality problems; however, none of them for object characteristics extraction.This thesis examines how these different approaches can be applied to the given problem to help inexperienced users solve the combination puzzles. Moreover, it produces a side effect which is a possibility to track objects movement (rotation, translation) that can be used for manipulating a rendered game puzzle and increase interactivity and engagement of the user

Cognitive computing: algorithm design in the intersection of cognitive science and emerging computer architectures

For the first time in decades computers are evolving into a fundamentally new class of machine. Transistors are still getting smaller, more economical, and more power-efficient, but operating frequencies leveled off in the mid-2000's. Today, improving performance requires placing a larger number of slower processing cores on each of many chips. Software written for such machines must scale out over many cores rather than scaling up with a faster single core. Biological computation is an extreme manifestation of such a many-slow-core architecture and therefore offers a potential source of ideas for leveraging new hardware. This dissertation addresses several problems in the intersection of emerging computer architectures and biological computation, termed Cognitive Computing: What mechanisms are necessary to maintain stable representations in a large distributed learning system? How should complex biologically-inspired algorithms be tested? How do visual sensing limitations like occlusion influence performance of classification algorithms? Neurons have a limited dynamic output range, but must process real-world signals over a wide dynamic range without saturating or succumbing to endogenous noise. Many existing neural network models leverage spatial competition to address this issue, but require hand-tuning of several parameters for a specific, fixed distribution of inputs. Integrating spatial competition with a stabilizing learning process produces a neural network model capable of autonomously adapting to a non-stationary distribution of inputs. Human-engineered complex systems typically include a number of architectural features to curtail complexity and simplify testing. Biological systems do not obey these constraints. Biologically-inspired algorithms are thus dramatically more difficult to engineer. Augmenting standard tools from the software engineering community with features targeted towards biologically-inspired systems is an effective mitigation. Natural visual environments contain objects that are occluded by other objects. Such occlusions are under-represented in the standard benchmark datasets for testing classification algorithms. This bias masks the negative effect of occlusion on performance. Correcting the bias with a new dataset demonstrates that occlusion is a dominant variable in classification performance. Modifying a state-of-the-art algorithm with mechanisms for occlusion resistance doubles classification performance in high-occlusion cases without penalty for unoccluded objects

Advances in Monocular Exemplar-based Human Body Pose Analysis: Modeling, Detection and Tracking

Esta tesis contribuye en el análisis de la postura del cuerpo humano a partir de secuencias de imágenes adquiridas con una sola cámara. Esta temática presenta un amplio rango de potenciales aplicaciones en video-vigilancia, video-juegos o aplicaciones biomédicas. Las técnicas basadas en patrones han tenido éxito, sin embargo, su precisión depende de la similitud del punto de vista de la cámara y de las propiedades de la escena entre las imágenes de entrenamiento y las de prueba. Teniendo en cuenta un conjunto de datos de entrenamiento capturado mediante un número reducido de cámaras fijas, paralelas al suelo, se han identificado y analizado tres escenarios posibles con creciente nivel de dificultad: 1) una cámara estática paralela al suelo, 2) una cámara de vigilancia fija con un ángulo de visión considerablemente diferente, y 3) una secuencia de video capturada con una cámara en movimiento o simplemente una sola imagen estática

From Constraints to Opportunities: Efficient Object Detection Learning for Humanoid Robots

Reliable perception and efficient adaptation to novel conditions are priority skills for robots that function in ever-changing environments. Indeed, autonomously operating in real world scenarios raises the need of identifying different context\u2019s states and act accordingly. Moreover, the requested tasks might not be known a-priori, requiring the system to update on-line. Robotic platforms allow to gather various types of perceptual information due to the multiple sensory modalities they are provided with. Nonetheless, latest results in computer vision motivate a particular interest in visual perception. Specifically, in this thesis, I mainly focused on the object detection task since it can be at the basis of more sophisticated capabilities. The vast advancements in latest computer vision research, brought by deep learning methods, are appealing in a robotic setting. However, their adoption in applied domains is not straightforward since adapting them to new tasks is strongly demanding in terms of annotated data, optimization time and computational resources. These requirements do not generally meet current robotics constraints. Nevertheless, robotic platforms and especially humanoids present opportunities that can be exploited. The sensors they are provided with represent precious sources of additional information. Moreover, their embodiment in the workspace and their motion capabilities allow for a natural interaction with the environment. Motivated by these considerations, in this Ph.D project, I mainly aimed at devising and developing solutions able to integrate the worlds of computer vision and robotics, by focusing on the task of object detection. Specifically, I dedicated a large amount of effort in alleviating state-of-the-art methods requirements in terms of annotated data and training time, preserving their accuracy by exploiting robotics opportunity

Design and Development of Robotic Part Assembly System under Vision Guidance

Robots are widely used for part assembly across manufacturing industries to attain high productivity through automation. The automated mechanical part assembly system contributes a major share in production process. An appropriate vision guided robotic assembly system further minimizes the lead time and improve quality of the end product by suitable object detection methods and robot control strategies. An approach is made for the development of robotic part assembly system with the aid of industrial vision system. This approach is accomplished mainly in three phases. The first phase of research is mainly focused on feature extraction and object detection techniques. A hybrid edge detection method is developed by combining both fuzzy inference rule and wavelet transformation. The performance of this edge detector is quantitatively analysed and compared with widely used edge detectors like Canny, Sobel, Prewitt, mathematical morphology based, Robert, Laplacian of Gaussian and wavelet transformation based. A comparative study is performed for choosing a suitable corner detection method. The corner detection technique used in the study are curvature scale space, Wang-Brady and Harris method. The successful implementation of vision guided robotic system is dependent on the system configuration like eye-in-hand or eye-to-hand. In this configuration, there may be a case that the captured images of the parts is corrupted by geometric transformation such as scaling, rotation, translation and blurring due to camera or robot motion. Considering such issue, an image reconstruction method is proposed by using orthogonal Zernike moment invariants. The suggested method uses a selection process of moment order to reconstruct the affected image. This enables the object detection method efficient. In the second phase, the proposed system is developed by integrating the vision system and robot system. The proposed feature extraction and object detection methods are tested and found efficient for the purpose. In the third stage, robot navigation based on visual feedback are proposed. In the control scheme, general moment invariants, Legendre moment and Zernike moment invariants are used. The selection of best combination of visual features are performed by measuring the hamming distance between all possible combinations of visual features. This results in finding the best combination that makes the image based visual servoing control efficient. An indirect method is employed in determining the moment invariants for Legendre moment and Zernike moment. These moments are used as they are robust to noise. The control laws, based on these three global feature of image, perform efficiently to navigate the robot in the desire environment

Articulated human tracking and behavioural analysis in video sequences

Recently, there has been a dramatic growth of interest in the observation and tracking of human subjects through video sequences. Arguably, the principal impetus has come from the perceived demand for technological surveillance, however applications in entertainment, intelligent domiciles and medicine are also increasing. This thesis examines human articulated tracking and the classi cation of human movement, rst separately and then as a sequential process. First, this thesis considers the development and training of a 3D model of human body structure and dynamics. To process video sequences, an observation model is also designed with a multi-component likelihood based on edge, silhouette and colour. This is de ned on the articulated limbs, and visible from a single or multiple cameras, each of which may be calibrated from that sequence. Second, for behavioural analysis, we develop a methodology in which actions and activities are described by semantic labels generated from a Movement Cluster Model (MCM). Third, a Hierarchical Partitioned Particle Filter (HPPF) was developed for human tracking that allows multi-level parameter search consistent with the body structure. This tracker relies on the articulated motion prediction provided by the MCM at pose or limb level. Fourth, tracking and movement analysis are integrated to generate a probabilistic activity description with action labels. The implemented algorithms for tracking and behavioural analysis are tested extensively and independently against ground truth on human tracking and surveillance datasets. Dynamic models are shown to predict and generate synthetic motion, while MCM recovers both periodic and non-periodic activities, de ned either on the whole body or at the limb level. Tracking results are comparable with the state of the art, however the integrated behaviour analysis adds to the value of the approach.Overseas Research Students Awards Scheme (ORSAS

Proceedings of the 2010 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

On the annual Joint Workshop of the Fraunhofer IOSB and the Karlsruhe Institute of Technology (KIT), Vision and Fusion Laboratory, the students of both institutions present their latest research findings on image processing, visual inspection, pattern recognition, tracking, SLAM, information fusion, non-myopic planning, world modeling, security in surveillance, interoperability, and human-computer interaction. This book is a collection of 16 reviewed technical reports of the 2010 Joint Workshop