A 3D descriptor to detect task-oriented grasping points in clothing

© 2016. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/Manipulating textile objects with a robot is a challenging task, especially because the garment perception is difficult due to the endless configurations it can adopt, coupled with a large variety of colors and designs. Most current approaches follow a multiple re-grasp strategy, in which clothes are sequentially grasped from different points until one of them yields a recognizable configuration. In this work we propose a method that combines 3D and appearance information to directly select a suitable grasping point for the task at hand, which in our case consists of hanging a shirt or a polo shirt from a hook. Our method follows a coarse-to-fine approach in which, first, the collar of the garment is detected and, next, a grasping point on the lapel is chosen using a novel 3D descriptor. In contrast to current 3D descriptors, ours can run in real time, even when it needs to be densely computed over the input image. Our central idea is to take advantage of the structured nature of range images that most depth sensors provide and, by exploiting integral imaging, achieve speed-ups of two orders of magnitude with respect to competing approaches, while maintaining performance. This makes it especially adequate for robotic applications as we thoroughly demonstrate in the experimental section.Peer ReviewedPostprint (author's final draft

THE APPLICATION OF COMPUTER VISION, MACHINE AND DEEP LEARNING ALGORITHMS UTILIZING MATLAB

MATLAB is a multi-paradigm proprietary programming language and numerical computing environment developed by MathWorks. Within MATLAB Integrated Development Environment (IDE) you can perform Computer-aided design (CAD), different matrix manipulations, plotting of functions and data, implementation algorithms, creation of user interfaces, and has the ability to interface with programs written in other languages1. Since, its launch in 1984 MATLAB software has not particularly been associated within the field of data science. In 2013, that changed with the launch of their new data science concentrated toolboxes that included Deep Learning, Image Processing, Computer Vision, and then a year later Statistics and Machine Learning. The main objective of my thesis was to research and explore the field of data science. More specifically pertaining to the development of an object recognition application that could be built entirely using MATLAB IDE and have a positive social impact on the deaf community. And in doing so, answering the question, could MATLAB be utilized for development of this type of application? To simultaneously answer this question while addressing my main objectives, I constructed two different object recognition protocols utilizing MATLAB_R2019 with the add-on data science tool packages. I named the protocols ASLtranslate (I) and (II). This allowed me to experiment with all of MATLAB data science toolboxes while learning the differences, benefits, and disadvantages of using multiple approaches to the same problem. The methods and approaches for the design of both versions was very similar. ASLtranslate takes in 2D image of American Sign Language (ASL) hand gestures as an input, classifies the image and then outputs its corresponding alphabet character. ASLtranslate (I) was an implementation of image category classification using machine learning methods. ASLtranslate (II) was implemented by using a deep learning method called transfer learning, done by fine-tuning a pre-trained convolutional neural network (CNN), AlexNet, to perform classification on a new collection of images