Superquadric representation of scenes from multi-view range data

Object representation denotes representing three-dimensional (3D) real-world objects with known graphic or mathematic primitives recognizable to computers. This research has numerous applications for object-related tasks in areas including computer vision, computer graphics, reverse engineering, etc. Superquadrics, as volumetric and parametric models, have been selected to be the representation primitives throughout this research. Superquadrics are able to represent a large family of solid shapes by a single equation with only a few parameters. This dissertation addresses superquadric representation of multi-part objects and multiobject scenes. Two issues motivate this research. First, superquadric representation of multipart objects or multi-object scenes has been an unsolved problem due to the complex geometry of objects. Second, superquadrics recovered from single-view range data tend to have low confidence and accuracy due to partially scanned object surfaces caused by inherent occlusions. To address these two problems, this dissertation proposes a multi-view superquadric representation algorithm. By incorporating both part decomposition and multi-view range data, the proposed algorithm is able to not only represent multi-part objects or multi-object scenes, but also achieve high confidence and accuracy of recovered superquadrics. The multi-view superquadric representation algorithm consists of (i) initial superquadric model recovery from single-view range data, (ii) pairwise view registration based on recovered superquadric models, (iii) view integration, (iv) part decomposition, and (v) final superquadric fitting for each decomposed part. Within the multi-view superquadric representation framework, this dissertation proposes a 3D part decomposition algorithm to automatically decompose multi-part objects or multiobject scenes into their constituent single parts consistent with human visual perception. Superquadrics can then be recovered for each decomposed single-part object. The proposed part decomposition algorithm is based on curvature analysis, and includes (i) Gaussian curvature estimation, (ii) boundary labeling, (iii) part growing and labeling, and (iv) post-processing. In addition, this dissertation proposes an extended view registration algorithm based on superquadrics. The proposed view registration algorithm is able to handle deformable superquadrics as well as 3D unstructured data sets. For superquadric fitting, two objective functions primarily used in the literature have been comprehensively investigated with respect to noise, viewpoints, sample resolutions, etc. The objective function proved to have better performance has been used throughout this dissertation. In summary, the three algorithms (contributions) proposed in this dissertation are generic and flexible in the sense of handling triangle meshes, which are standard surface primitives in computer vision and graphics. For each proposed algorithm, the dissertation presents both theory and experimental results. The results demonstrate the efficiency of the algorithms using both synthetic and real range data of a large variety of objects and scenes. In addition, the experimental results include comparisons with previous methods from the literature. Finally, the dissertation concludes with a summary of the contributions to the state of the art in superquadric representation, and presents possible future extensions to this research

UGV Navigation in ROS using LIDAR 3D

This works addresses to give a step forward the achievement of robust Unmanned Ground Vehicles (UGVs), which can drive in urban environments. More specifically, it focuses in the management of a four wheeled vehicle in ROS using mainly the inputs provided by a LIDAR 3D. Simulations were carried out in ad-hoc scenarios designed and run using GAZEBO. Visual information provided by sensors is processed through PCL library. Thanks to this processing the needed parameters to manage the UGV are obtained and its guidance can be carried out though a PID controller.El foco de este trabajo consiste en avanzar un paso hacia la consecución de vehículos terrestres no tripulados robustos, que puedan circular en zonas urbanas. Más concretamente se centra en el manejo de un vehículo de cuatro ruedas en ROS usando, sobre todo, las entradas proporcionadas por un LIDAR 3D. Las simulaciones se llevaron a cabo en escenarios ad-hoc diseñados y ejecutados usando GAZEBO. La información visual de los sensores es procesada mediante la librería PCL. Gracias a este procesamiento se obtienen los parámetros para conducir el UGV y su guiado puede ser llevado a cabo mediante un controlador PID.Máster Universitario en Ingeniería Industrial (M141

Part decomposition of 3D surfaces

This dissertation describes a general algorithm that automatically decomposes realworld scenes and objects into visual parts. The input to the algorithm is a 3 D triangle mesh that approximates the surfaces of a scene or object. This geometric mesh completely specifies the shape of interest. The output of the algorithm is a set of boundary contours that dissect the mesh into parts where these parts agree with human perception. In this algorithm, shape alone defines the location of a bom1dary contour for a part. The algorithm leverages a human vision theory known as the minima rule that states that human visual perception tends to decompose shapes into parts along lines of negative curvature minima. Specifically, the minima rule governs the location of part boundaries, and as a result the algorithm is known as the Minima Rule Algorithm. Previous computer vision methods have attempted to implement this rule but have used pseudo measures of surface curvature. Thus, these prior methods are not true implementations of the rule. The Minima Rule Algorithm is a three step process that consists of curvature estimation, mesh segmentation, and quality evaluation. These steps have led to three novel algorithms known as Normal Vector Voting, Fast Marching Watersheds, and Part Saliency Metric, respectively. For each algorithm, this dissertation presents both the supporting theory and experimental results. The results demonstrate the effectiveness of the algorithm using both synthetic and real data and include comparisons with previous methods from the research literature. Finally, the dissertation concludes with a summary of the contributions to the state of the art

The design and control of an actively restrained passive mechatronic system for safety-critical applications

Development of manipulators that interact closely with humans has been a focus of research in fields such as robot-assisted surgery and haptic interfaces for many years. Recent introduction of powered surgical-assistant devices into the operating theatre has meant that robot manipulators have been required to interact with both patients and surgeons. Most of these manipulators are modified industrial robots. However, the use of high-powered mechanisms in the operating theatre could compromise safety of the patient, surgeon, and operating room staff. As a solution to the safety problem, the use of actively restrained passive arms has been proposed. Clutches or brakes at each joint are used to restrict the motion of the end-effector to restrain it to a pre-defined region or path. However, these devices have only had limited success in following pre-defined paths under human guidance. In this research, three major limitations of existing passive devices actively restrained are addressed. [Continues.

Digital system of quarry management as a SAAS solution: mineral deposit module

Purpose. Improving the efficiency of functioning the mining enterprises and aggregation of earlier obtained results into a unified digital system of designing and operative management by quarry operation. Methods. Both the traditional (analysis of scientific and patent literature, analytical methods of deposit parameters research, analysis of experience and exploitation of quarries, conducting the passive experiment and processing the statistical data) and new forms of scientific research - deposit modeling on the basis of classical and neural network methods of approximation – are used in the work. For the purpose of the software product realization on the basis of cloud technologies, there were used: for back-end implementation – server-based scripting language php; for the front-end – multi-paradigm programming language javascript, javascript framework jQuery and asynchronous data exchange technology Ajax. Findings. The target audience of the system has been identified, SWOT-analysis has been carried out, conceptual directions of 3D-quarry system development have been defined. The strategies of development and promotion of the software product, as well as the strategies of safety and reliability of the application both for the client and the owner of the system have been formulated. The modular structure of the application has been developed, and the system functions have been divided to implement both back-end and front-end applications. The Mineral Deposit Module has been developed: the geological structure of the deposit has been simulated and its block model has been constructed. It has been proved that the use of neural network algorithms does not give an essential increase in the accuracy of the block model for the deposits of 1 and 2 groups in terms of the geological structure complexity. The possibility and prospects of constructing the systems for subsoil users on the basis of cloud technologies and the concept of SaaS have been substantiated. Originality. For the first time, the modern software products for solving the problems of designing and operational management of mining operations have been successfully developed on the basis of the SaaS concept. Practical implications. The results are applicable for enterprises-subsoil users, working with deposits of 1 and 2 groups in terms of the geological structure complexity: design organizations, as well as mining and processing plants.Мета. Підвищення ефективності функціонування гірничорудних підприємств та агрегація раніше отриманих результатів в єдину цифрову систему проектування і оперативного управління роботою кар’єрів. Методика. У роботі використані як традиційні (аналіз науково-патентної літератури, аналітичні методи дослідження параметрів родовища, аналіз досвіду й експлуатації кар’єрів, проведення пасивного експерименту та статистичної обробки даних), так і нові форми наукового дослідження – моделювання родовища на основі класичних і нейромережевих методів апроксимації. Для реалізації програмного продукту на основі хмарних технологій використані: для реалізації back-end – серверна скриптова мова програмування php; для front-end – мультипарадігменна мова програмування javascript, javascript framework jQuery і технологія асинхронного обміну даними Ajax. Результати. Виявлено цільову аудиторію системи, проведено SWOT-аналіз, визначено концептуальні напрями розвитку системи 3D-кар’єр, розроблені стратегії розвитку та просування програмного продукту, розроблені стратегії безпеки й надійності додатки як для клієнта, так і власника системи. Розроблено модульну структуру програми, вироблено розподіл функцій системи для реалізації як back-end і front-end додатки. Розроблено модуль “Родовище”: проведено моделювання геологічної структури родовища та побудована його блокова модель. Доведено, що використання нейромережевих алгоритмів не дає принципового підвищення точності блокової моделі для родовищ 1 і 2 груп за складністю геологічної будови. Виявлено недоліки нейромережевих алгоритмів, такі як високі витрати обчислювальних ресурсів сервера і проблеми візуалізації великих масивів геоданих при використанні web-рішень, знайдені шляхи їх вирішення. Доведено можливість і перспективність побудови систем для надрокористувачів на основі хмарних технологій і концепції SaaS. Наукова новизна. Вперше на основі концепції ASP успішно побудовані сучасні програмні продукти для вирішення завдань проектування та оперативного керування гірничими роботами. Практична значимість. Результати корисні для підприємств-надрокористувачів, які працюють з родовищами 1 і 2 груп за складністю геологічної будови – проектних організацій і ГЗК.Цель. Повышение эффективности функционирования горнорудных предприятий и агрегация ранее полученных результатов в единую цифровую систему проектирования и оперативного управления работой карьеров. Методика. В работе использованы как традиционные (анализ научно-патентной литературы, аналитические методы исследования параметров месторождения, анализ опыта и эксплуатации карьеров, проведение пассивного эксперимента и статистической обработкой данных), так и новые формы научного исследования – моделирование месторождения на основе классических и нейросетевых методов аппроксимации. Для реализации программного продукта на основе облачных технологий использованы: для реализации back-end – серверный скриптовый язык программирования php; для front-end – мультипарадигменный язык программирования javascript, javascript framework jQuery и технология асинхронного обмена данными Ajax. Результаты. Выявлена целевая аудитория системы, проведен SWOT-анализ, определены концептуальные направления развития системы 3D-карьер, разработаны стратегии развития и продвижения программного продукта, разработаны стратегии безопасности и надежности приложения как для клиента, так и владельца системы. Разработана модульная структура приложения, произведено деление функций системы для реализации как back-end и front-end приложения. Разработан модуль “Месторождение”: проведено моделирование геологической структуры месторождения и построена его блочная модель. Доказано, что использование нейросетевых алгоритмов не дает принципиального повышения точности блочной модели для месторождений 1 и 2 групп по сложности геологического строения. Выявлены недостатки нейросетевых алгоритмов, такие как высокие затраты вычислительных ресурсов сервера и проблемы визуализации больших массивов геоданных при использовании web-решений, найдены пути их решения. Доказана возможность и перспективность построения систем для недропользователей на основе облачных технологий и концепции SaaS. Научная новизна. Впервые на основе концепции ASP успешно построены современные программные продукты для решения задач проектирования и оперативного управления горными работами. Практическая значимость. Результаты применимы для предприятий-недропользователей, работающих с месторождениями 1 и 2 групп по сложности геологического строения – проектных организаций и ГОКов.We express our profound gratitude to A.B. Naizabekov for his assistance in scientific research, to A.F. Tsekhovoy, P.A. Tsekhovoy, D.Sh. Akhmedov, V. V. Yankovenko and D.V. Nikitas for scientific advice in implementation of the program code. The research was carried out within the framework of the initiative research theme “Improving the Efficiency of Mining Enterprises” on the basis of the RSE at the Rudny Industrial Institute of the Ministry of Education and Science of the Republic of Kazakhstan

Design of an Automotive IoT Device to Improve Driver Fault Detection Through Road Class Estimation

Unsafe driver habits pose a serious threat to all vehicles on the road. This thesis outlines the development of an automotive IoT device capable of monitoring and reporting adverse driver habits to mitigate the occurrence of unsafe practices. The driver habits targeted are harsh braking, harsh acceleration, harsh cornering, speeding and over revving the vehicle. With the intention of evaluating and expanding upon the industry method of fault detection, a working prototype is designed to handle initialization, data collection, vehicle state tracking, fault detection and communication. A method of decoding the broadcasted messages on the vehicle bus is presented and unsafe driver habits are detected using static limits. An analysis of the initial design’s performance revealed that the industry method of detecting faults fails to account for the vehicle’s speed and is unable to detect faults on all roadways. A framework for analyzing fault profiles at varying speeds is presented and yields the relationship between fault magnitude and speed. A method of detecting the type of road driven was developed to dynamically assign fault limits while the vehicle traveled on a highway, city street or in traffic. The improved design correctly detected faults along all types of roads and proved to greatly expand upon the current method of fault detection used by the automotive IoT industry today

Surface Modeling and Analysis Using Range Images: Smoothing, Registration, Integration, and Segmentation

This dissertation presents a framework for 3D reconstruction and scene analysis, using a set of range images. The motivation for developing this framework came from the needs to reconstruct the surfaces of small mechanical parts in reverse engineering tasks, build a virtual environment of indoor and outdoor scenes, and understand 3D images. The input of the framework is a set of range images of an object or a scene captured by range scanners. The output is a triangulated surface that can be segmented into meaningful parts. A textured surface can be reconstructed if color images are provided. The framework consists of surface smoothing, registration, integration, and segmentation. Surface smoothing eliminates the noise present in raw measurements from range scanners. This research proposes area-decreasing flow that is theoretically identical to the mean curvature flow. Using area-decreasing flow, there is no need to estimate the curvature value and an optimal step size of the flow can be obtained. Crease edges and sharp corners are preserved by an adaptive scheme. Surface registration aligns measurements from different viewpoints in a common coordinate system. This research proposes a new surface representation scheme named point fingerprint. Surfaces are registered by finding corresponding point pairs in an overlapping region based on fingerprint comparison. Surface integration merges registered surface patches into a whole surface. This research employs an implicit surface-based integration technique. The proposed algorithm can generate watertight models by space carving or filling the holes based on volumetric interpolation. Textures from different views are integrated inside a volumetric grid. Surface segmentation is useful to decompose CAD models in reverse engineering tasks and help object recognition in a 3D scene. This research proposes a watershed-based surface mesh segmentation approach. The new algorithm accurately segments the plateaus by geodesic erosion using fast marching method. The performance of the framework is presented using both synthetic and real world data from different range scanners. The dissertation concludes by summarizing the development of the framework and then suggests future research topics

Clothoid-based Planning and Control in Intelligent Vehicles (Autonomous and Manual-Assisted Driving)

[EN] Nowadays, there are many electronic products that incorporate elements and features coming from the research in the field of mobile robotics. For instance, the well-known vacuum cleaning robot Roomba by iRobot, which belongs to the field of service robotics, one of the most active within the sector. There are also numerous autonomous robotic systems in industrial warehouses and plants. It is the case of Autonomous Guided Vehicles (AGVs), which are able to drive completely autonomously in very structured environments. Apart from industry and consumer electronics, within the automotive field there are some devices that give intelligence to the vehicle, derived in most cases from advances in mobile robotics. In fact, more and more often vehicles incorporate Advanced Driver Assistance Systems (ADAS), such as navigation control with automatic speed regulation, lane change and overtaking assistant, automatic parking or collision warning, among other features. However, despite all the advances there are some problems that remain unresolved and can be improved. Collisions and rollovers stand out among the most common accidents of vehicles with manual or autonomous driving. In fact, it is almost impossible to guarantee driving without accidents in unstructured environments where vehicles share the space with other moving agents, such as other vehicles and pedestrians. That is why searching for techniques to improve safety in intelligent vehicles, either autonomous or manual-assisted driving, is still a trending topic within the robotics community. This thesis focuses on the design of tools and techniques for planning and control of intelligent vehicles in order to improve safety and comfort. The dissertation is divided into two parts, the first one on autonomous driving and the second one on manual-assisted driving. The main link between them is the use of clothoids as mathematical formulation for both trajectory generation and collision detection. Among the problems solved the following stand out: obstacle avoidance, rollover avoidance and advanced driver assistance to avoid collisions with pedestrians.[ES] En la actualidad se comercializan infinidad de productos de electrónica de consumo que incorporan elementos y características procedentes de avances en el sector de la robótica móvil. Por ejemplo, el conocido robot aspirador Roomba de la empresa iRobot, el cual pertenece al campo de la robótica de servicio, uno de los más activos en el sector. También hay numerosos sistemas robóticos autónomos en almacenes y plantas industriales. Es el caso de los vehículos autoguiados (AGVs), capaces de conducir de forma totalmente autónoma en entornos muy estructurados. Además de en la industria y en electrónica de consumo, dentro del campo de la automoción también existen dispositivos que dotan de cierta inteligencia al vehículo, derivados la mayoría de las veces de avances en robótica móvil. De hecho, cada vez con mayor frecuencia los vehículos incorporan sistemas avanzados de asistencia al conductor (ADAS por sus siglas en inglés), tales como control de navegación con regulación automática de velocidad, asistente de cambio de carril y adelantamiento, aparcamiento automático o aviso de colisión, entre otras prestaciones. No obstante, pese a todos los avances siguen existiendo problemas sin resolver y que pueden mejorarse. La colisión y el vuelco destacan entre los accidentes más comunes en vehículos con conducción tanto manual como autónoma. De hecho, la dificultad de conducir en entornos desestructurados compartiendo el espacio con otros agentes móviles, tales como coches o personas, hace casi imposible garantizar la conducción sin accidentes. Es por ello que la búsqueda de técnicas para mejorar la seguridad en vehículos inteligentes, ya sean de conducción autónoma o manual asistida, es un tema que siempre está en auge en la comunidad robótica. La presente tesis se centra en el diseño de herramientas y técnicas de planificación y control de vehículos inteligentes, para la mejora de la seguridad y el confort. La disertación se ha dividido en dos partes, la primera sobre conducción autónoma y la segunda sobre conducción manual asistida. El principal nexo de unión es el uso de clotoides como elemento de generación de trayectorias y detección de colisiones. Entre los problemas que se resuelven destacan la evitación de obstáculos, la evitación de vuelcos y la asistencia avanzada al conductor para evitar colisiones con peatones.[CA] En l'actualitat es comercialitzen infinitat de productes d'electrònica de consum que incorporen elements i característiques procedents d'avanços en el sector de la robòtica mòbil. Per exemple, el conegut robot aspirador Roomba de l'empresa iRobot, el qual pertany al camp de la robòtica de servici, un dels més actius en el sector. També hi ha nombrosos sistemes robòtics autònoms en magatzems i plantes industrials. És el cas dels vehicles autoguiats (AGVs), els quals són capaços de conduir de forma totalment autònoma en entorns molt estructurats. A més de en la indústria i en l'electrònica de consum, dins el camp de l'automoció també existeixen dispositius que doten al vehicle de certa intel·ligència, la majoria de les vegades derivats d'avanços en robòtica mòbil. De fet, cada vegada amb més freqüència els vehicles incorporen sistemes avançats d'assistència al conductor (ADAS per les sigles en anglés), com ara control de navegació amb regulació automàtica de velocitat, assistent de canvi de carril i avançament, aparcament automàtic o avís de col·lisió, entre altres prestacions. No obstant això, malgrat tots els avanços segueixen existint problemes sense resoldre i que poden millorar-se. La col·lisió i la bolcada destaquen entre els accidents més comuns en vehicles amb conducció tant manual com autònoma. De fet, la dificultat de conduir en entorns desestructurats compartint l'espai amb altres agents mòbils, tals com cotxes o persones, fa quasi impossible garantitzar la conducció sense accidents. És per això que la recerca de tècniques per millorar la seguretat en vehicles intel·ligents, ja siguen de conducció autònoma o manual assistida, és un tema que sempre està en auge a la comunitat robòtica. La present tesi es centra en el disseny d'eines i tècniques de planificació i control de vehicles intel·ligents, per a la millora de la seguretat i el confort. La dissertació s'ha dividit en dues parts, la primera sobre conducció autònoma i la segona sobre conducció manual assistida. El principal nexe d'unió és l'ús de clotoides com a element de generació de trajectòries i detecció de col·lisions. Entre els problemes que es resolen destaquen l'evitació d'obstacles, l'evitació de bolcades i l'assistència avançada al conductor per evitar col·lisions amb vianants.Girbés Juan, V. (2016). Clothoid-based Planning and Control in Intelligent Vehicles (Autonomous and Manual-Assisted Driving) [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/65072TESI