Component Segmentation of Engineering Drawings Using Graph Convolutional Networks

We present a data-driven framework to automate the vectorization and machine interpretation of 2D engineering part drawings. In industrial settings, most manufacturing engineers still rely on manual reads to identify the topological and manufacturing requirements from drawings submitted by designers. The interpretation process is laborious and time-consuming, which severely inhibits the efficiency of part quotation and manufacturing tasks. While recent advances in image-based computer vision methods have demonstrated great potential in interpreting natural images through semantic segmentation approaches, the application of such methods in parsing engineering technical drawings into semantically accurate components remains a significant challenge. The severe pixel sparsity in engineering drawings also restricts the effective featurization of image-based data-driven methods. To overcome these challenges, we propose a deep learning based framework that predicts the semantic type of each vectorized component. Taking a raster image as input, we vectorize all components through thinning, stroke tracing, and cubic bezier fitting. Then a graph of such components is generated based on the connectivity between the components. Finally, a graph convolutional neural network is trained on this graph data to identify the semantic type of each component. We test our framework in the context of semantic segmentation of text, dimension and, contour components in engineering drawings. Results show that our method yields the best performance compared to recent image, and graph-based segmentation methods.Comment: Preprint accepted to Computers in Industr

Image-Based Force Estimation and Haptic Rendering For Robot-Assisted Cardiovascular Intervention

Clinical studies have indicated that the loss of haptic perception is the prime limitation of robot-assisted cardiovascular intervention technology, hindering its global adoption. It causes compromised situational awareness for the surgeon during the intervention and may lead to health risks for the patients. This doctoral research was aimed at developing technology for addressing the limitation of the robot-assisted intervention technology in the provision of haptic feedback. The literature review showed that sensor-free force estimation (haptic cue) on endovascular devices, intuitive surgeon interface design, and haptic rendering within the surgeon interface were the major knowledge gaps. For sensor-free force estimation, first, an image-based force estimation methods based on inverse finite-element methods (iFEM) was developed and validated. Next, to address the limitation of the iFEM method in real-time performance, an inverse Cosserat rod model (iCORD) with a computationally efficient solution for endovascular devices was developed and validated. Afterward, the iCORD was adopted for analytical tip force estimation on steerable catheters. The experimental studies confirmed the accuracy and real-time performance of the iCORD for sensor-free force estimation. Afterward, a wearable drift-free rotation measurement device (MiCarp) was developed to facilitate the design of an intuitive surgeon interface by decoupling the rotation measurement from the insertion measurement. The validation studies showed that MiCarp had a superior performance for spatial rotation measurement compared to other modalities. In the end, a novel haptic feedback system based on smart magnetoelastic elastomers was developed, analytically modeled, and experimentally validated. The proposed haptics-enabled surgeon module had an unbounded workspace for interventional tasks and provided an intuitive interface. Experimental validation, at component and system levels, confirmed the usability of the proposed methods for robot-assisted intervention systems

The Effect of Malaysia General Election on Financial Network: An Evidence from Shariah-Compliant Stocks on Bursa Malaysia

Instead of focusing the volatility of the market, the market participants should consider on how the general election affects the correlation between the stocks during 14th general election Malaysia. The 14th general election of Malaysia was held on 9th May 2018. This event has a great impact towards the stocks listed on Bursa Malaysia. Thus, this study investigates the effect of 14th general election Malaysia towards the correlation between stock in Bursa Malaysia specifically the shariah-compliant stock. In addition, this paper examines the changes in terms of network topology for the duration, sixth months before and after the general election. The minimum spanning tree was used to visualize the correlation between the stocks. Also, the centrality measure, namely degree, closeness and betweenness were computed to identify if any changes of stocks that plays a crucial role in the network for the duration of before and after 14th general election Malaysia

The kinematic theory and minimum principles in motor control : a conceptual comparison

This paper presents a comparative study of two motor control theories that have put forward mathematical expressions to describe the stereotypical velocity profiles of rapid movements: the Kinematic Theory and the Minimization Theory. Among the various forms of the latter, the Minimum Square- Derivatives (MSD) principle and the Minimum-Time model are analyzed. It is shown that their concepts are linked and describe, with different arguments, a paradigm similar to the one used in the Kinematic Theory to model a velocity profile with a Delta-Lognormal equation. This unifying paradigm represents the functioning of a neuromuscular system by the convolution product of an infinite number of subsystem impulse responses. A second finding emerging from the present study is that the analytical models of velocity profiles, as described by the minimum principles under study correspond, with more or less accuracy, to an approximation of the Delta-Lognormal equation. Overall, the Kinematic Theory can be seen as relying on a general optimization principle and the use of the Minimization Theory in motor control gets new insights

Text Detection and Recognition in the Wild

Text detection and recognition (TDR) in highly structured environments with a clean background and consistent fonts (e.g., office documents, postal addresses and bank cheque) is a well understood problem (i.e., OCR), however this is not the case for unstructured environments. The main objective for scene text detection is to locate text within images captured in the wild. For scene text recognition, the techniques map each detected or cropped word image into string. Nowadays, convolutional neural networks (CNNs) and Recurrent Neural Networks (RNN) deep learning architectures dominate most of the recent state-of-the-art (SOTA) scene TDR methods. Most of the reported respective accuracies of current SOTA TDR methods are in the range of 80% to 90% on benchmark datasets with regular and clear text instances. However, those detecting and/or recognizing results drastically deteriorate 10% and 30% - in terms of F-measure detection and word recognition accuracy performances with irregular or occluded text images. Transformers and their variations are new deep learning architectures that mitigate the above-mentioned issues for CNN and RNN-based pipelines.Unlike Recurrent Neural Networks (RNNs), transformers are models that learn how to encode and decode data by looking not only backward but also forward in order to extract relevant information from a whole sequence. This thesis utilizes the transformer architecture to address the irregular (multi-oriented and arbitrarily shaped) and occluded text challenges in the wild images. Our main contributions are as follows: (1) We first targeted solving the irregular TDR in two separate architectures as follows: In Chapter 4, unlike the SOTA text detection frameworks that have complex pipelines and use many hand-designed components and post-processing stages, we design a conceptually more straightforward and trainable end-to-end architecture of transformer-based detector for multi-oriented scene text detection, which can directly predict the set of detections (i.e., text and box regions) of the input image. A central contribution to our work is introducing a loss function tailored to the rotated text detection problem that leverages a rotated version of a generalized intersection over union score to capture the rotated text instances adequately. In Chapter 5, we extend our previous architecture to arbitrary shaped scene text detection. We design a new text detection technique that aims to better infer n-vertices of a polygon or the degree of a Bezier curve to represent irregular-text instances. We also propose a loss function that combines a generalized-split-intersection-over union loss defined over the piece-wise polygons. In Chapter 6, we show that our transformer-based architecture without rectifying the input curved text instances is more suitable than SOTA RNN-based frameworks equipped with rectification modules for irregular text recognition in the wild images. Our main contribution to this chapter is leveraging a 2D Learnable Sinusoidal frequencies Positional Encoding (2LSPE) with a modified feed-forward neural network to better encode the 2D spatial dependencies of characters in the irregular text instances. (2) Since TDR tasks encounter the same challenging problems (e.g., irregular text, illumination variations, low-resolution text, etc.), we present a new transformer model that can detect and recognize individual characters of text instances in an end-to-end manner. Reading individual characters later makes a robust occlusion and arbitrarily shaped text spotting model without needing polygon annotation or multiple stages of detection and recognition modules used in SOTA text spotting architectures. In Chapter 7, unlike SOTA methods that combine two different pipelines of detection and recognition modules for a complete text reading, we utilize our text detection framework by leveraging a recent transformer-based technique, namely Deformable Patch-based Transformer (DPT), as a feature extracting backbone, to robustly read the class and box coordinates of irregular characters in the wild images. (3) Finally, we address the occlusion problem by using a multi-task end-to-end scene text spotting framework. In Chapter 8, we leverage a recent transformer-based framework in deep learning, namely Masked Auto Encoder (MAE), as a backbone for scene text recognition and end-to-end scene text spotting pipelines to overcome the partial occlusion limitation. We design a new multitask End-to-End transformer network that directly outputs characters, word instances, and their bounding box representations, saving the computational overhead as it eliminates multiple processing steps. The unified proposed framework can also detect and recognize arbitrarily shaped text instances without using polygon annotations

Integration of sketch-based ideation and 3D modeling with CAD systems

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This thesis is concerned with the study of how sketch-based systems can be improved to enhance idea generation process in conceptual design stage. It is also concerned with achieving a kind of integration between sketch-based systems and CAD systems to complete the digitization of the design process as sketching phase is still not integrated with other phases due to the different nature of it and the incomplete digitization of sketching phase itself. Previous studies identified three main related issues: sketching process, sketch-based modeling, and the integration between the digitized design phases. Here, the thesis is motivated from the desire to improve sketch-based modeling to support idea generation process but unlike previous studies that only focused on the technical or drawing part of sketching, this thesis attempts to concentrate more on the mental part of the sketching process which play a key role in developing ideas in design. Another motivation of this thesis is to produce a kind of integration between sketch-based systems and CAD systems to enable 3D models produced by sketching to be edited in detailed design stage. As such, there are two main contributions have been addressed in this thesis. The first contribution is the presenting of a new approach in designing sketch-based systems that enable more support for idea generation by separating thinking and developing ideas from the 3D modeling process. This kind of separation allows designers to think freely and concentrate more on their ideas rather than 3D modeling. the second contribution is achieving a kind of integration between gesture-based systems and CAD systems by using an IGES file in exchanging data between systems and a new method to organize data within the file in an order that make it more understood by feature recognition embedded in commercial CAD systems.This study is funded by the Ministry of Higher Education of Egypt

Intraoperative, Quantitative, and Non-Contact Blood Volume Flow Measurement via Indocyanine Green Fluorescence Angiography

In vielen Fällen unterziehen sich Patienten einer Revaskularisationsoperation wenn sie an einer zerebrovaskulären Erkrankung leiden, die eine Hypoperfusion des Gehirns verursacht. Dieser chirurgische Eingriff wird häufig als offene Operation durchgeführt und hat das Ziel, die Gefäßfunktion, insbesondere den Blutfluss, wiederherzustellen. Hierzu wird eine Anastomose (Verbindung von Arterien) angelegt, um den Fluss zu einem hypoperfundierten Gehirnareal zu erhöhen. In ungefähr 10% der Eingriffe treten nach der Operation Komplikationen auf, die zum Teil auf eine unzureichende Durchflusssteigerung zurückgeführt werden. Daher sollte der Blutfluss intraoperativ überprüft werden, um die Qualität des Eingriffs im Operationssaal zu beurteilen und schnell eingreifen zu können. Damit könnte ein negativer Ausgang für den Patienten verhindert werden. Der derzeitige Stand der Technik in der intraoperativen und quantitativen Blutflussmessung ist die Nutzung der Ultraschall-Transitzeit-Durchflusssonde. Sie gibt einen quantitativen Flusswert an, muss jedoch das Gefäß umschließen. Dies ist einerseits umständlich für den Chirurgen und andererseits birgt es das Risiko von Kontaminationen, Gefäßquetschungen und der Gefäßruptur. Eine alternative Methode ist die Indocyaningrün (ICG) Fluoreszenzangiographie (FA), welche eine kamerabasierte Methode ist. Sie ist der Stand der Technik in der hochauflösenden anatomischen Visualisierung des Situs und kann zusätzlich dem Chirurgen eine qualitative funktionelle Darstellung der Gefäße im Sichtfeld liefern. Der Stand der Wissenschaft zur Quantifizierung des Blutflusses mittels ICG-FA konnten bisher keine verlässlichen Fluss- werte liefern. Die vorliegende Arbeit analysiert und verbessert die Eignung von ICG FA zu Bereitstellung von verlässlichen und quantitativen Blutflusswerten, indem 1. geklärt wird, wie akkurat die Messung durchgeführt werden kann. 2. Methoden zur Verbesserung der Genauigkeit entwickelt werden. 3. die Existenz eines systematischen Fehlers abgeleitet wird. 4. eine Methode zur Kompensation des systematischen Fehlers entwickelt wird. 5. ein Algorithmus zur Verarbeitung der eingehenden Videodaten für eine Ausgabe eines Durchflusswertes bereitgestellt wird. 6. die Validierung der vorgeschlagenen Methoden und des Arbeitsablaufs in einer ex vivo und in vivo Studie durchgeführt wird. Die in dieser Arbeit vorgeschlagene Messung basiert auf dem systemic mean transit time theorem für Systeme mit einem Eingang und einem Ausgang. Um den Fluss zu berechnen müssen die Transitzeit eines ICG-Bolus für eine zu bestimmenden Strecke und die Querschnittsfläche des Gefäßes ermittelt werden. Es wurden Methoden entwickelt, um den Blutvolumenstrom zu messen und um Fehlerquellen bei dieser Messung der einzelnen Parameter zu identifizieren, quantifizieren und reduzieren. Die statistischen Fehler bei der Messung der Transitstrecke und der Transitzeit des ICG- Bolus sowie der Querschnittsfläche des Gefäßes werden in der Forschung oft vernachlässigt. In dieser Arbeit wurden die Fehler mit Hilfe von in silico Modellen quantifiziert. Es zeigte sich, dass der Fehler zu groß für eine zuverlässige Blutflussmessung ist und daher Methoden zu seiner Reduzierung benötigt werden. Um den Fehler bei der Längenmessung deutlich zu reduzieren, wurde eine Methode vorgestellt, welche die diskrete Mittellinie wieder in eine kontinuierliche überführt. Dabei wird der Fehler in der Längenmessung signifikant reduziert und der Fehler von der räumlichen Orientierung der Struktur entkoppelt. In ähnlicher Weise wurde eine Methode vorgestellt, welche die gemessenen diskreten Indikatorverdünnungskurven (IDCs) ebenso in kontinuierliche überführt, um den Fehler in der Laufzeitmessung des ICG-Bolus zu reduzieren. Der propagierte statistische Fehler der Blutflussmessung wurde auf einen akzeptablen und praktikablen Wert von 20 % bis 30 % reduziert. Die Präsenz eines systematischen Fehlers bei der optischen Messung des Blutflusses wurde identifiziert und aus der Definition des Volumenflusses theoretisch abgeleitet. Folgend wird eine Methode zur Kompensation des Fehlers vorgestellt. Im ersten Schritt wird eine Fluid-Strömungssimulation genutzt, um die räumlich-zeitliche Konzentration des ICG in einem Blutgefäß zu berechnen. Anschließend wird die Konzentration an ein neu entwickeltes Fluoreszenz-Monte-Carlo-Multizylinder (FMCMC) Modell übergeben, das die Ausbreitung von Photonen in einem Gefäß simuliert. Dabei wird der Ort der Fluoreszenzereignisse der emittierten Photonen ermittelt und der systematische Fehler bestimmt. Dies ermöglicht die Kompensation des systematischen Fehlers. Es zeigte sich, dass dieser Fehler unabhängig von dem Volumenfluss ist, solange die Strömung laminar ist, aber abhängig vom Durchmesser des Gefäßes und dem Zeitpunkt der Messung. Die Abhängigkeit vom Durchmesser ist reduziert bei Messungen zu einem früheren Zeitpunkt. Daher ist es vorteilhaft, die erste Ankunft des ICG-Bolus zur Bestimmung der Transitzeit zu verwenden, um den Einfluss des Durchmessers auf den Fehler zu verringern und somit die Messung robuster durchzuführen. Um die Genauigkeit der Messung in einem Experiment zu beweisen, wurde ein ex vivo Experiment unter Verwendung von Schweineblut und Kaninchen Aorten konzipiert und durchgeführt. Es zeigte sich, dass der durch den vorgeschlagenen Algorithmus ermittelte Fluss mit der Referenzmessung (einem industriellem Durchflussmesser) übereinstimmt. Die statistische Streuung der gemessenen Flussdaten durch den Algorithmus stimmte mit der zuvor ermittelten statistischen Fehlerspanne überein, was den in silico Ansatz validiert. Es wurde eine retrospektive in vivo Studie an Menschen durchgeführt, die sich einer extrakraniellen-zu-intrakraniellen (EC-IC) Bypass Operation unterzogen hatten. Die Analyse der FA-Daten ergab eine gute Übereinstimmung mit der klinischen Referenzmethode, jedoch mit dem großen Vorteil, dass kein Kontakt zum Gewebe erforderlich war. Zusätzlich wurde gezeigt, dass simultan Flusswerte für mehrere Gefäße im Sichtfeld der Kamera gemessen werden können. Die vorgestellten Ergebnisse sind ein Proof of Concept für die Eignung der vorgestellten intraoperativen, quantitativen und optischen Messung des Blutvolumenstroms mittels ICG FA. Diese Arbeit ebnet den Weg für den klinischen Einsatz dieser Methode in Ergänzung zum aktuellen klinischen Stand der Technik. Sie könnte zukünftig dem Chirurgen eine neuartige Messung des Blutvolumenstroms zur Verfügung stellen und dabei potentiell das Risiko einer Komplikation reduzieren und damit das Wohl der Patienten verbessern

AutoGraff: towards a computational understanding of graffiti writing and related art forms.

The aim of this thesis is to develop a system that generates letters and pictures with a style that is immediately recognizable as graffiti art or calligraphy. The proposed system can be used similarly to, and in tight integration with, conventional computer-aided geometric design tools and can be used to generate synthetic graffiti content for urban environments in games and in movies, and to guide robotic or fabrication systems that can materialise the output of the system with physical drawing media. The thesis is divided into two main parts. The first part describes a set of stroke primitives, building blocks that can be combined to generate different designs that resemble graffiti or calligraphy. These primitives mimic the process typically used to design graffiti letters and exploit well known principles of motor control to model the way in which an artist moves when incrementally tracing stylised letter forms. The second part demonstrates how these stroke primitives can be automatically recovered from input geometry defined in vector form, such as the digitised traces of writing made by a user, or the glyph outlines in a font. This procedure converts the input geometry into a seed that can be transformed into a variety of calligraphic and graffiti stylisations, which depend on parametric variations of the strokes