Robust 2D Assembly Sequencing via Geometric Planning with Learned Scores

To compute robust 2D assembly plans, we present an approach that combines geometric planning with a deep neural network. We train the network using the Box2D physics simulator with added stochastic noise to yield robustness scores--the success probabilities of planned assembly motions. As running a simulation for every assembly motion is impractical, we train a convolutional neural network to map assembly operations, given as an image pair of the subassemblies before and after they are mated, to a robustness score. The neural network prediction is used within a planner to quickly prune out motions that are not robust. We demonstrate this approach on two-handed planar assemblies, where the motions are one-step translations. Results suggest that the neural network can learn robustness to plan robust sequences an order of magnitude faster than physics simulation.Comment: Presented at the 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE

The evaluation of a novel haptic machining VR-based process planning system using an original process planning usability method

This thesis provides an original piece of work and contribution to knowledge by creating a new process planning system; Haptic Aided Process Planning (HAPP). This system is based on the combination of haptics and virtual reality (VR). HAPP creates a simulative machining environment where Process plans are automatically generated from the real time logging of a user’s interaction. Further, through the application of a novel usability test methodology, a deeper study of how this approach compares to conventional process planning was undertaken. An abductive research approach was selected and an iterative and incremental development methodology chosen. Three development cycles were undertaken with evaluation studies carried out at the end of each. Each study, the pre-pilot, pilot and industrial, identified progressive refinements to both the usability of HAPP and the usability evaluation method itself. HAPP provided process planners with an environment similar to which they are already familiar. Visual images were used to represent tools and material whilst a haptic interface enabled their movement and positioning by an operator in a manner comparable to their native setting. In this way an intuitive interface was developed that allowed users to plan the machining of parts consisting of features that can be machined on a pillar drill, 21/2D axis milling machine or centre lathe. The planning activities included single or multiple set ups, fixturing and sequencing of cutting operations. The logged information was parsed and output to a process plan including route sheets, operation sheets, tool lists and costing information, in a human readable format. The system evaluation revealed that HAPP, from an expert planners perspective is perceived to be 70% more satisfying to use, 66% more efficient in completing process plans, primarily due to the reduced cognitive load, is more effective producing a higher quality output of information and is 20% more learnable than a traditional process planning approach

Monocular 3d Object Recognition

Object recognition is one of the fundamental tasks of computer vision. Recent advances in the field enable reliable 2D detections from a single cluttered image. However, many challenges still remain. Object detection needs timely response for real world applications. Moreover, we are genuinely interested in estimating the 3D pose and shape of an object or human for the sake of robotic manipulation and human-robot interaction. In this thesis, a suite of solutions to these challenges is presented. First, Active Deformable Part Models (ADPM) is proposed for fast part-based object detection. ADPM dramatically accelerates the detection by dynamically scheduling the part evaluations and efficiently pruning the image locations. Second, we unleash the power of marrying discriminative 2D parts with an explicit 3D geometric representation. Several methods of such scheme are proposed for recovering rich 3D information of both rigid and non-rigid objects from monocular RGB images. (1) The accurate 3D pose of an object instance is recovered from cluttered images using only the CAD model. (2) A global optimal solution for simultaneous 2D part localization, 3D pose and shape estimation is obtained by optimizing a unified convex objective function. Both appearance and geometric compatibility are jointly maximized. (3) 3D human pose estimation from an image sequence is realized via an Expectation-Maximization algorithm. The 2D joint location uncertainties are marginalized out during inference and 3D pose smoothness is enforced across frames. By bridging the gap between 2D and 3D, our methods provide an end-to-end solution to 3D object recognition from images. We demonstrate a range of interesting applications using only a single image or a monocular video, including autonomous robotic grasping with a single image, 3D object image pop-up and a monocular human MoCap system. We also show empirical start-of-art results on a number of benchmarks on 2D detection and 3D pose and shape estimation

A Contribution to Conveying Quality Criteria in Mechanical CAD Models and Assemblies through Rubrics and Comprehensive Design Intent Quantification

Esta investigación examinó el uso de rúbricas de ensamblaje, describiendo su evolución a partir de rúbricas de piezas, y estudió cómo afectan a la autoevaluación de los estudiantes. También se valoró la evaluación de los estudiantes por los instructores, encontrando que, mientras que las rúbricas de ensamblaje fueron parcialmente comprendidas y utilizadas de manera eficiente por los estudiantes, éstas fueron usadas con más éxito por los instructores. En esta investigación se han abordado estrategias diseñadas para mejorar la comunicación de la intención de diseño en modelos CAD, acrecentando así su calidad, con directrices dirigidas a evaluar su eficiencia. Es evidente que se necesitan métricas dirigidas hacia la instrucción de la intención de diseño, ya que la intención de diseño transferida a través de modelos CAD puede realizarse en tres etapas con criterios contradictorios que deben ser equilibrados para llegar a la mejor estrategia de modelado. La investigación ha incluido el desarrollo de un método de validación que demuestra que las rúbricas son dispositivos útiles para garantizar una comunicación consistente de la intención de diseño, y son fundamentales no sólo para evaluar, sino también para comunicar las expectativas del instructor. En esta investigación se examinó cómo definir claramente las cualidades de la intención de diseño para permitir una más fácil evaluación de un ensamblaje CAD. Para todas las dimensiones de la rúbrica, se encontró más concordancia y correlación entre instructores que entre instructores y estudiantes. Existe una correlación moderada/fuerte entre los instructores para las dimensiones de la validez, completitud, concisión y claridad, mientras que existe una ligera correlación para las dimensiones de consistencia e intención del diseño. En segundo lugar, las rúbricas también pueden ser descritas como estáticas o dinámicas. Las rúbricas estáticas, existen sólo en papel, no proporcionan retroalimentación inmediata al educando. Las rúbricas dinámicas realizan cálculos que proporcionan observaciones de evaluación inmediatas al usuario. Además, pueden adaptarse a situaciones específicas dependiendo de la capacidad del usuario. Las rúbricas electrónicas son ideales para rúbricas dinámicas, y permiten el uso y desarrollo de rúbricas adaptativas y adaptables, como se describe a continuación. En tercer lugar, las rúbricas deben ser adaptables lo que debería hacerlas fácilmente comprensibles y fáciles de usar, y adaptativas. Las rúbricas de evaluación se usan cuando un experto determina el progreso pedagógico de un educando, mientras que las rúbricas formativas son empleadas por los propios estudiantes, para trazar su progreso e identificar las deficiencias escolares para las que necesitan apoyo. Las rúbricas se deben refinar y mejorar de forma continuada, en un proceso iterativo y colaborativo, hasta que se alcance un acuerdo satisfactorio, tanto entre evaluadores como entre evaluadores y alumnos. Por ello, se desarrollaron mapas de aserciones que ilustran el modo en que la estrategia de expansión-contracción adapta las rúbricas al progreso del aprendiz de CAD, a la vez que ayudan a comprender las diferentes dimensiones de la rúbrica. Basándose en los experimentos con las rúbricas de ensamblajes, es evidente que las pequeñas diferencias entre los instructores sugieren que la rúbrica de ensamblajes propuesta es lo suficientemente sofisticada como para proporcionar una evaluación acumulativa imparcial del desempeño del alumno. En consecuencia, se puede afirmar con confianza que los evaluadores pueden usarse indistintamente sin sacrificar la precisión. Sin embargo, la rúbrica de ensamblaje posee una eficacia finita para producir una autoevaluación formativa de las habilidades de ensamblaje CAD para nuevos alumnos.This research examined the use of assembly rubrics, described how they evolved from parts rubrics, and studied how they affect student self-evaluation. Instructor assessment of students was also evaluated, finding that while the assembly rubrics were partially understood and effectively used by the students, they were more successfully utilized by the instructors. Strategies designed to improve design intent communication in CAD models, in order to enhance their quality, with guidelines targeted to evaluate efficiency, have been addressed with this research. It is apparent that metrics directed toward the instruction of design intent are needed, since design intent transferred through CAD models can be performed at three stages with competing tradeoffs that must be balanced to arrive at the best modeling strategy. Research included the development of a validation approach that reflects that rubrics are valuable devices to expedite consistent design intent communication, and are vital not only for evaluation, but also for the communication of instructor expectations. This research examined how to clearly define qualities of design intent to enable easier CAD assembly assessment. It has been found that there is more inter-rater agreement and correlation between instructors than between instructors and students, for all rubric dimensions. There is strong to moderate correlation between instructors for the dimensions of validity, completeness, conciseness, and clarity, while slight correlation exists for the dimensions of consistency and design intent. Secondly, rubrics can also be described as being either static or dynamic. Static rubrics, existing in paper form only, do not provide immediate feedback to the learner. Dynamic rubrics perform calculations that provide immediate evaluative observations to the user. Besides, they can be independently adapted to specific situations depending on the capability of the user. Electronic rubrics are ideally suited for dynamic rubrics, and permit the use and development of both adaptable and adaptive rubrics, as described next. Thirdly, rubrics need to be adaptable which should make them easily understood and user-friendly, and adaptive (rubric can change itself, depending on the usage pattern). Evaluative rubrics are used when an expert determines the pedagogical progress of a learner, while formative rubrics are employed by the learners themselves, in order to chart their progress and identify scholastic deficiencies that are in need of remediation. Rubrics must be continually refined and improved, in an iterative, collaborative process, until satisfactory agreement is attained, both between raters, but also between raters and learners. Thus, assertions maps were developed, illustrating how the expand-contract strategy adapts the rubrics to CAD trainee progress, while assisting the understanding of the different rubric dimensions. Based on the assembly rubric experiments, it is apparent that the small differences between instructors suggests that the proposed assemblies rubric is sufficiently sophisticated to furnish an unbiased accumulative assessment of student performance. Accordingly, it can be confidently stated that raters can be used interchangeably without sacrificing accuracy. However, the assembly rubric possesses finite efficacy to produce formative self-evaluation of CAD assembly skills for new learners.Aquesta investigació examinà l'ús de rúbriques de acoblament, descrivint la seua evolució a partir de rúbriques de peces, i estudià cóm afecten a la autoavaluació dels estudiants. També es va valorar la avaluació dels estudiants per els instructors, trobant que, mentre que les rúbriques de acoblament van ser parcialment compreses i fetes servir de manera eficient per els estudiants, van ser usades amb mes èxit per els instructors. En aquesta investigació s'han abordat estrategues dissenyades per a millorar la comunicació de la intenció de disseny en models CAD, creixentat així la seua qualitat, amb directrius dirigides a avaluar la seua eficiència. Es evident que es necessiten mètriques dirigides cap a la instrucció de la intenció de disseny, ja que la intenció de disseny transferida a través de models CAD pot realitzar-se en tres etapes amb criteris contradictoris que deuen ser equilibrats per a arribar a la millor estratègia de modelatge. La investigació ha inclòs el desenvolupament de un mètode de validació que demostra que las rúbriques son dispositius útils per a garantir una comunicació consistent de la intenció de disseny, i son fonamentals no només per a avaluar, però també per a comunicar les expectatives de l'instructor. En aquesta investigació s'examinà com definir clarament les qualitats de la intenció de disseny per a permetre una mes fàcil avaluació de un acoblament CAD. Per a totes les dimensions de la rúbrica, es va trobar mes concordança i correlació entre instructors que no pas entre instructors i estudiants. Existeix una correlació moderada/fort entre els instructors per a les dimensions de la validesa, completesa, concisió i claredat, mentre que existeix una lleugera correlació per a les dimensions de consistència i intenció del disseny. En segon lloc, les rúbriques també poden ser descrites com estàtiques o dinàmiques. Les rúbriques estàtiques, existeixen sòls en paper, no proporcionen retroalimentació immediata a l'educand. Les rúbriques dinàmiques realitzen càlculs que proporcionen observacions de avaluació immediates al usuari. A mes, poden adaptar-se a situacions específiques dependent de la capacitat de l'usuari. Les rúbriques electròniques son ideals per a rúbriques dinàmiques, i permeten l'ús i desenvolupament de rúbriques adaptatives i adaptables, como es descriu a continuació. En tercer lloc, les rúbriques deuen ser adaptables, el que deuria ferles fàcilment comprensibles i fàcils d'usar, i adaptatives. Les rúbriques d'avaluació se usen quant un expert determina el progrés pedagògic de un educand, mentre que les rúbriques formatives son fetes servir per els propis estudiants, per a traçar el seu progrés i identificar les deficiències escolars para a les que necessiten suport. Les rúbriques es deuen refinar i millorar de forma continuada, en un procés iteratiu i col·laboratori, fins que se arriba a un acord satisfactori, tant entre avaluadors como entre avaluadors i alumnes. Por això, es desenvoluparen mapes de assercions que il·lustren el mode en que la estratègia de expansió-contracció adapta les rúbriques al progres del aprenent de CAD, a la vegada que ajuden a comprendre les diferents dimensions de la rúbrica. Basant-se en els experiments amb les rúbriques de acoblaments, es evident que les xicotetes diferencies entre els instructors suggereixen que la rúbrica de acoblaments proposta es lo suficientment sofisticada com per a proporcionar una avaluació acumulativa imparcial del acompliment de l'alumne. En conseqüència, es pot afirmar amb confiança que els avaluadors poden usar-se indistintament sense sacrificar la precisió. No obstant, la rúbrica de acoblaments posseeix una eficàcia finita per a produir una autoavaluació formativa de les habilitats de acoblament CAD per a nous alumnes.Otey, JM. (2017). A Contribution to Conveying Quality Criteria in Mechanical CAD Models and Assemblies through Rubrics and Comprehensive Design Intent Quantification [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/94627TESI

GNNs for Knowledge Transfer in Robotic Assembly Sequence Planning

Automated Assembly Sequence Planning (ASP) is a crucial task for robotic systems in manufacturing settings as it increases flexibility and efficiency. However, ASP is still largely performed manually, which can be time-consuming and prone to errors. The ASP process consists of two main steps: determining possible assembly sequences and confirming the feasibility of these sequences with the capabilities and restrictions of the target robot system. While several methods have been developed to automate ASP in recent years, they have limitations such as a lack of flexibility with regard to the properties of the assembly, a long training process, and a dependence on both positive and negative examples for feasibility detection. To address these issues, we propose a graph-based approach that divides the ASP problem into two tasks: Sequence Prediction and Feasibility Prediction. For the Sequence Prediction task, we model assemblies as graphs of part surfaces and apply a Graph Neural Network (GNN) to efficiently extract meaningful information from the input. An expert demonstrator guides us through the sequence prediction process, step-by-step, predicting which parts can be placed in their position at each state of the assembly. Our method is flexible and able to transfer knowledge between different assembly tasks. For Feasibility Prediction, we frame the task as an Anomaly Detection (AD) problem and use our GNN as a feature extractor to create latent representations for each assembly graph. We train a Normalizing Flows (NF) model to model the distribution of feasible assemblies and detect infeasible assemblies as Out-of-Distribution (OoD). Although our method performs better than a baseline one-class classifier, it still lags behind a binary classifier trained on both feasible and infeasible assemblies. However, it is not trivial to obtain a sufficient amount of infeasible assemblies for training. To better understand the limitations of our approach, we conduct ablation studies. Our approach requires only a few seconds to derive a feasible assembly sequence and could be integrated into future ASP frameworks to dramatically reduce planning time while using fewer computational resources compared to previous methods. To our knowledge, we are the first to use NF for graph-level Anomaly Detection. Overall, our method has the potential to significantly improve the efficiency and effectiveness of ASP in manufacturing settings

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways