Component Segmentation of Engineering Drawings Using Graph Convolutional Networks

We present a data-driven framework to automate the vectorization and machine interpretation of 2D engineering part drawings. In industrial settings, most manufacturing engineers still rely on manual reads to identify the topological and manufacturing requirements from drawings submitted by designers. The interpretation process is laborious and time-consuming, which severely inhibits the efficiency of part quotation and manufacturing tasks. While recent advances in image-based computer vision methods have demonstrated great potential in interpreting natural images through semantic segmentation approaches, the application of such methods in parsing engineering technical drawings into semantically accurate components remains a significant challenge. The severe pixel sparsity in engineering drawings also restricts the effective featurization of image-based data-driven methods. To overcome these challenges, we propose a deep learning based framework that predicts the semantic type of each vectorized component. Taking a raster image as input, we vectorize all components through thinning, stroke tracing, and cubic bezier fitting. Then a graph of such components is generated based on the connectivity between the components. Finally, a graph convolutional neural network is trained on this graph data to identify the semantic type of each component. We test our framework in the context of semantic segmentation of text, dimension and, contour components in engineering drawings. Results show that our method yields the best performance compared to recent image, and graph-based segmentation methods.Comment: Preprint accepted to Computers in Industr

New trends on digitisation of complex engineering drawings

Engineering drawings are commonly used across different industries such as oil and gas, mechanical engineering and others. Digitising these drawings is becoming increasingly important. This is mainly due to the legacy of drawings and documents that may provide rich source of information for industries. Analysing these drawings often requires applying a set of digital image processing methods to detect and classify symbols and other components. Despite the recent significant advances in image processing, and in particular in deep neural networks, automatic analysis and processing of these engineering drawings is still far from being complete. This paper presents a general framework for complex engineering drawing digitisation. A thorough and critical review of relevant literature, methods and algorithms in machine learning and machine vision is presented. Real-life industrial scenario on how to contextualise the digitised information from specific type of these drawings, namely piping and instrumentation diagrams, is discussed in details. A discussion of how new trends on machine vision such as deep learning could be applied to this domain is presented with conclusions and suggestions for future research directions

Semi-automatic geometric digital twinning for existing buildings based on images and CAD drawings

Despite the emerging new data capturing technologies and advanced modelling systems, the process of geometric digital twin modelling for existing buildings still lacks a systematic and completed framework to streamline. As-is Building Information Model (BIM) is one of the commonly used geometric digital twin modelling approaches. However, the process of as-is BIM construction is time-consuming and needed to improve. To address this challenge, in this paper, a semi-automatic approach is developed to establish a systematic, accurate and convenient digital twinning system based on images and CAD drawings. With this ultimate goal, this paper summarises the state-of-the-art geometric digital twinning methods and elaborates on the methodological framework of this semi-automatic geometric digital twinning approach. The framework consists of three modules. The Building Framework Construction and Geometry Information Extraction (Module 1) defines the locations of each structural component through recognising special symbols in a floor plan and then extracting data from CAD drawings using the Optical Character Recognition (OCR) technology. Meaningful text information is further filtered based on predefined rules. In order to integrate with completed building information, the Building Information Complementary (Module 2) is developed based on neuro-fuzzy system (NFS) and the image processing procedure to supplement additional building components. Finally, the Information Integration and IFC Creation (Module 3) integrates information from Module 1 and 2 and creates as-is Industry Foundation Classes (IFC) BIM based on IFC schema. A case study using part of an office building and the results of its analysis are provided and discussed from the perspectives of applicability and accuracy. Future works and limitations are also addressed

Using artificial intelligence to find design errors in the engineering drawings

Artificial intelligence is increasingly becoming important to businesses because many companies have realized the benefits of applying machine learning (ML) and deep learning (DL) in their operations. ML and DL have become attractive technologies for organizations looking to automate repetitive tasks to reduce manual work and free up resources for innovation. Unlike rule-based automation, typically used for standardized and predictable processes, machine learning, especially deep learning, can handle more complex tasks and learn over time, leading to greater accuracy and efficiency improvements. One of such promising applications is to use AI to reduce manual engineering work. This paper discusses a particular case within McDermott where the research team developed a DL model to do a quality check of complex blueprints. We describe the development and the final product of this case—AI-based software for the engineering, procurement, and construction (EPC) industry that helps to find the design mistakes buried inside very complex engineering drawings called piping and instrumentation diagrams (P&IDs). We also present a cost-benefit analysis and potential scale-up of the developed software. Our goal is to share the successful experience of AI-based product development that can substantially reduce the engineering hours and, therefore, reduce the project\u27s overall costs. The developed solution can also be potentially applied to other EPC companies doing a similar design for complex installations with high safety standards like oil and gas or petrochemical plants because the design errors it captures are common within this industry. It also could motivate practitioners and researchers to create similar products for the various fields within engineering industry

Using empirical studies to mitigate symbol overload in iStar extensions

UID/CEC/04516/2019Modelling languages are frequently extended to include new constructs to be used together with the original syntax. New constructs may be proposed by adding textual information, such as UML stereotypes, or by creating new graphical representations. Thus, these new symbols need to be expressive and proposed in a careful way to increase the extension’s adoption. A method to create symbols for the original constructs of a modelling language was proposed and has been used to create the symbols when a new modelling language is designed. We argue this method can be used to recommend new symbols for the extension’s constructs. However, it is necessary to make some adjustments since the new symbols will be used with the existing constructs of the modelling language original syntax. In this paper, we analyse the usage of this adapted method to propose symbols to mitigate the occurrence of overloaded symbols in the existing iStar extensions. We analysed the existing iStar extensions in an SLR and identified the occurrence of symbol overload among the existing constructs. We identified a set of fifteen overloaded symbols in existing iStar extensions. We used these concepts with symbol overload in a multi-stage experiment that involved users in the visual notation design process. The study involved 262 participants, and its results revealed that most of the new graphical representations were better than those proposed by the extensions, with regard to semantic transparency. Thus, the new representations can be used to mitigate this kind of conflict in iStar extensions. Our results suggest that next extension efforts should consider user-generated notation design techniques in order to increase the semantic transparency.authorsversionpublishe

Enriching BIM models with fire safety equipment using keypoint-based symbol detection in escape plans

In the context of fire safety inspections, Building Information Modeling (BIM) models enriched with Fire Safety Equipment (FSE) components can be used to complete compliance checks and other analyses. However, BIM models often lack the required FSE information. To address this issue, escape plans are a convenient source of data, as they show the position and type of FSE on floor plans. Therefore, this study proposes an automated method to analyze escape plans and extract FSE component information to enrich existing BIM models. The method employs the deep learning model Keypoint R-CNN for symbol detection. Symbol locations are then translated into physical positions within the BIM model. Through a real-building case study, the method demonstrates promising results. Future research may focus on improving the symbol detection performance and the registration between the BIM models and fire escape plans, as well as utilizing the extracted information for actual fire safety analyses

Interactive interpretation of structured documents: Application to the recognition of handwritten architectural plans

International audienceThis paper addresses a whole architecture, including the IMISketch method. IMISketch method incorporates two aspects: document analysis and interactivity. This paper describes a global vision of all the parts of the project. IMISketch is a generic method for an interactive interpretation of handwritten sketches. The analysis of complex documents requires the management of uncertainty. While, in practice the similar methods often induce a large combinatorics, IMISketch method presents several optimization strategies to reduce the combinatorics. The goal of these optimizations is to have a time analysis compatible with user expectations. The decision process is able to solicit the user in the case of strong ambiguity: when it is not sure to make the right decision, the user explicitly validates the right decision to avoid a fastidious a posteriori verification phase due to propagation of errors.This interaction requires solving two major problems: how interpretation results will be presented to the user, and how the user will interact with analysis process. We propose to study the effects of those two aspects. The experiments demonstrate that (i) a progressive presentation of the analysis results, (ii) user interventions during it and (iii) the user solicitation by the analysis process are an efficient strategy for the recognition of complex off-line documents.To validate this interactive analysis method, several experiments are reported on off-line handwritten 2D architectural floor plans