8,449 research outputs found

    A deep learning-enhanced digital twin framework for improving safety and reliability in human-robot collaborative manufacturing

    Get PDF
    In Industry 5.0, Digital Twins bring in flexibility and efficiency for smart manufacturing. Recently, the success of artificial intelligence techniques such as deep learning has led to their adoption in manufacturing and especially in human–robot collaboration. Collaborative manufacturing tasks involving human operators and robots pose significant safety and reliability concerns. In response to these concerns, a deep learning-enhanced Digital Twin framework is introduced through which human operators and robots can be detected and their actions can be classified during the manufacturing process, enabling autonomous decision making by the robot control system. Developed using Unreal Engine 4, our Digital Twin framework complies with the Robotics Operating System specification, and supports synchronous control and communication between the Digital Twin and the physical system. In our framework, a fully-supervised detector based on a faster region-based convolutional neural network is firstly trained on synthetic data generated by the Digital Twin, and then tested on the physical system to demonstrate the effectiveness of the proposed Digital Twin-based framework. To ensure safety and reliability, a semi-supervised detector is further designed to bridge the gap between the twin system and the physical system, and improved performance is achieved by the semi-supervised detector compared to the fully-supervised detector that is simply trained on either synthetic data or real data. The evaluation of the framework in multiple scenarios in which human operators collaborate with a Universal Robot 10 shows that it can accurately detect the human and robot, and classify their actions under a variety of conditions. The data from this evaluation have been made publicly available, and can be widely used for research and operational purposes. Additionally, a semi-automated annotation tool from the Digital Twin framework is published to benefit the collaborative robotics community

    Graduate Catalog of Studies, 2023-2024

    Get PDF

    Beam scanning by liquid-crystal biasing in a modified SIW structure

    Get PDF
    A fixed-frequency beam-scanning 1D antenna based on Liquid Crystals (LCs) is designed for application in 2D scanning with lateral alignment. The 2D array environment imposes full decoupling of adjacent 1D antennas, which often conflicts with the LC requirement of DC biasing: the proposed design accommodates both. The LC medium is placed inside a Substrate Integrated Waveguide (SIW) modified to work as a Groove Gap Waveguide, with radiating slots etched on the upper broad wall, that radiates as a Leaky-Wave Antenna (LWA). This allows effective application of the DC bias voltage needed for tuning the LCs. At the same time, the RF field remains laterally confined, enabling the possibility to lay several antennas in parallel and achieve 2D beam scanning. The design is validated by simulation employing the actual properties of a commercial LC medium

    Introduction to Facial Micro Expressions Analysis Using Color and Depth Images: A Matlab Coding Approach (Second Edition, 2023)

    Full text link
    The book attempts to introduce a gentle introduction to the field of Facial Micro Expressions Recognition (FMER) using Color and Depth images, with the aid of MATLAB programming environment. FMER is a subset of image processing and it is a multidisciplinary topic to analysis. So, it requires familiarity with other topics of Artifactual Intelligence (AI) such as machine learning, digital image processing, psychology and more. So, it is a great opportunity to write a book which covers all of these topics for beginner to professional readers in the field of AI and even without having background of AI. Our goal is to provide a standalone introduction in the field of MFER analysis in the form of theorical descriptions for readers with no background in image processing with reproducible Matlab practical examples. Also, we describe any basic definitions for FMER analysis and MATLAB library which is used in the text, that helps final reader to apply the experiments in the real-world applications. We believe that this book is suitable for students, researchers, and professionals alike, who need to develop practical skills, along with a basic understanding of the field. We expect that, after reading this book, the reader feels comfortable with different key stages such as color and depth image processing, color and depth image representation, classification, machine learning, facial micro-expressions recognition, feature extraction and dimensionality reduction. The book attempts to introduce a gentle introduction to the field of Facial Micro Expressions Recognition (FMER) using Color and Depth images, with the aid of MATLAB programming environment.Comment: This is the second edition of the boo

    Virtual Model Building for Multi-Axis Machine Tools Using Field Data

    Get PDF
    Accurate machine dynamic models are the foundation of many advanced machining technologies such as virtual process planning and machine condition monitoring. Viewing recent designs of modern high-performance machine tools, to enhance the machine versatility and productivity, the machine axis configuration is becoming more complex and diversified, and direct drive motors are more commonly used. Due to the above trends, coupled and nonlinear multibody dynamics in machine tools are gaining more attention. Also, vibration due to limited structural rigidity is an important issue that must be considered simultaneously. Hence, this research aims at building high-fidelity machine dynamic models that are capable of predicting the dynamic responses, such as the tracking error and motor current signals, considering a wide range of dynamic effects such as structural flexibility, inter-axis coupling, and posture-dependency. Building machine dynamic models via conventional bottom-up approaches may require extensive investigation on every single component. Such approaches are time-consuming or sometimes infeasible for the machine end-users. Alternatively, as the recent trend of Industry 4.0, utilizing data via Computer Numerical Controls (CNCs) and/or non-intrusive sensors to build the machine model is rather favorable for industrial implementation. Thus, the methods proposed in this thesis are top-down model building approaches, utilizing available data from CNCs and/or other auxiliary sensors. The achieved contributions and results of this thesis are summarized below. As the first contribution, a new modeling and identification technique targeting a closed-loop control system of coupled rigid multi-axis feed drives has been developed. A multi-axis closed-loop control system, including the controller and the electromechanical plant, is described by a multiple-input multiple-output (MIMO) linear time-invariant (LTI) system, coupled with a generalized disturbance input that represents all the nonlinear dynamics. Then, the parameters of the open-loop and closed-loop dynamic models are respectively identified by a strategy that combines linear Least Squares (LS) and constrained global optimization. This strategy strikes a balance between model accuracy and computational efficiency. This proposed method was validated using an industrial 5-axis laser drilling machine and an experimental feed drive, achieving 2.38% and 5.26% root mean square (RMS) prediction error, respectively. Inter-axis coupling effects, i.e., the motion of one axis causing the dynamic responses of another axis, are correctly predicted. Also, the tracking error induced by motor ripple and nonlinear friction is correctly predicted as well. As the second contribution, the above proposed methodology is extended to also consider structural flexibility, which is a crucial behavior of large-sized industrial 5-axis machine tools. More importantly, structural vibration is nonlinear and posture-dependent due to the nature of a multibody system. In this thesis, prominent cases of flexibility-induced vibrations in a linear feed drive are studied and modeled by lumped mass-spring-damper system. Then, a flexible linear drive coupled with a rotary drive is systematically analyzed. It is found that the case with internal structural vibration between the linear and rotary drives requires an additional motion sensor for the proposed model identification method. This particular case is studied with an experimental setup. The thesis presents a method to reconstruct such missing internal structural vibration using the data from the embedded encoders as well as a low-cost micro-electromechanical system (MEMS) inertial measurement unit (IMU) mounted on the machine table. It is achieved by first synchronizing the data, aligning inertial frames, and calibrating mounting misalignments. Finally, the unknown internal vibration is reconstructed by comparing the rigid and flexible machine kinematic models. Due to the measurement limitation of MEMS IMUs and geometric assembly error, the reconstructed angle is unfortunately inaccurate. Nevertheless, the vibratory angular velocity and acceleration are consistently reconstructed, which is sufficient for the identification with reasonable model simplification. Finally, the reconstructed internal vibration along with the gathered servo data are used to identify the proposed machine dynamic model. Due to the separation of linear and nonlinear dynamics, the vibratory dynamics can be simply considered by adding complex pole pairs into the MIMO LTI system. Experimental validation shows that the identified model is able to predict the dynamic responses of the tracking error and motor force/torque to the input command trajectory and external disturbances, with 2% ~ 6% RMS error. Especially, the vibratory inter-axis coupling effect and posture-dependent effect are accurately depicted. Overall, this thesis presents a dynamic model-building approach for multi-axis feed drive assemblies. The proposed model is general and can be configured according to the kinematic configuration. The model-building approach only requires the data from the servo system or auxiliary motion sensors, e.g., an IMU, which is non-intrusive and in favor of industrial implementation. Future research includes further investigation of the IMU measurement, geometric error identification, validation using more complicated feed drive system, and applications to the planning and monitoring of 5-axis machining process

    Assessing Atmospheric Pollution and Its Impacts on the Human Health

    Get PDF
    This reprint contains articles published in the Special Issue entitled "Assessing Atmospheric Pollution and Its Impacts on the Human Health" in the journal Atmosphere. The research focuses on the evaluation of atmospheric pollution by statistical methods on the one hand, and on the other hand, on the evaluation of the relationship between the level of pollution and the extent of its effect on the population's health, especially on pulmonary diseases

    Applications and Properties of Magnetic Nanoparticles

    Get PDF
    This Special Issue aimed to cover the new developments in the synthesis and characterization of magnetic nanoconstructs ranging from conventional metal oxide nanoparticles to novel molecule-based or hybrid multifunctional nano-objects. At the same time, the focus was on the potential of these novel magnetic nanoconstructs in several possible applications, e.g. sensing, energy storage, and nanomedicine

    Ciguatoxins

    Get PDF
    Ciguatoxins (CTXs), which are responsible for Ciguatera fish poisoning (CFP), are liposoluble toxins produced by microalgae of the genera Gambierdiscus and Fukuyoa. This book presents 18 scientific papers that offer new information and scientific evidence on: (i) CTX occurrence in aquatic environments, with an emphasis on edible aquatic organisms; (ii) analysis methods for the determination of CTXs; (iii) advances in research on CTX-producing organisms; (iv) environmental factors involved in the presence of CTXs; and (v) the assessment of public health risks related to the presence of CTXs, as well as risk management and mitigation strategies

    Multimaterial 3D/4D Printing by Integrating Digital Light Processing and Direct Ink Writing

    Get PDF
    Driven by the growing demand of applications in robotics, electronics, biomedical devices and wearable devices, multi-material 3D printing has now become a trend to offer solutions with a wide choice of materials with various mechanical, chemical, thermal-mechanical, or electrical properties. However, it remains a challenge to find an approach, with a wide choice of materials, to realize high-resolution multi-material 3D printing efficiently. In this study, an innovative hybrid multi-material 3D printing system is developed, which integrates digital light processing (DLP), and direct ink writing (DIW). Here, DLP can efficiently provide a high-resolution matrix, with complex geometry and multicolor appearance, while DIW can add functionality to the component due to the wide choice of functional materials, such as shape memory photopolymers, conductive inks, and liquid crystal elastomers (LCE). With this hybrid 3D printing system, multicolor functional devices, circuit-embedding architectures, soft sensors, hybrid active lattices, active tensegrities, functionally graded actuators, and pure LCE lattices were successfully fabricated, showing a great prospect in the area of electronics, smart wearable devices, soft robots and actuators.Ph.D

    Human Gait Analysis using Spatiotemporal Data Obtained from Gait Videos

    Get PDF
    Mit der Entwicklung von Deep-Learning-Techniken sind Deep-acNN-basierte Methoden zum Standard fĂŒr Bildverarbeitungsaufgaben geworden, wie z. B. die Verfolgung menschlicher Bewegungen und PosenschĂ€tzung, die Erkennung menschlicher AktivitĂ€ten und die Erkennung von Gesichtern. Deep-Learning-Techniken haben den Entwurf, die Implementierung und den Einsatz komplexer und vielfĂ€ltiger Anwendungen verbessert, die nun in einer Vielzahl von Bereichen, einschließlich der Biomedizintechnik, eingesetzt werden. Die Anwendung von Computer-Vision-Techniken auf die medizinische Bild- und Videoanalyse hat zu bemerkenswerten Ergebnissen bei der Erkennung von Ereignissen gefĂŒhrt. Die eingebaute FĂ€higkeit von convolutional neural network (CNN), Merkmale aus komplexen medizinischen Bildern zu extrahieren, hat in Verbindung mit der FĂ€higkeit von long short term memory network (LSTM), die zeitlichen Informationen zwischen Ereignissen zu erhalten, viele neue Horizonte fĂŒr die medizinische Forschung geschaffen. Der Gang ist einer der kritischen physiologischen Bereiche, der viele Störungen im Zusammenhang mit Alterung und Neurodegeneration widerspiegeln kann. Eine umfassende und genaue Ganganalyse kann Einblicke in die physiologischen Bedingungen des Menschen geben. Bestehende Ganganalyseverfahren erfordern eine spezielle Umgebung, komplexe medizinische GerĂ€te und geschultes Personal fĂŒr die Erfassung der Gangdaten. Im Falle von tragbaren Systemen kann ein solches System die kognitiven FĂ€higkeiten beeintrĂ€chtigen und fĂŒr die Patienten unangenehm sein. Außerdem wurde berichtet, dass die Patienten in der Regel versuchen, wĂ€hrend des Labortests bessere Leistungen zu erbringen, was möglicherweise nicht ihrem tatsĂ€chlichen Gang entspricht. Trotz technologischer Fortschritte stoßen wir bei der Messung des menschlichen Gehens in klinischen und Laborumgebungen nach wie vor an Grenzen. Der Einsatz aktueller Ganganalyseverfahren ist nach wie vor teuer und zeitaufwĂ€ndig und erschwert den Zugang zu SpezialgerĂ€ten und Fachwissen. Daher ist es zwingend erforderlich, ĂŒber Methoden zu verfĂŒgen, die langfristige Daten ĂŒber den Gesundheitszustand des Patienten liefern, ohne doppelte kognitive Aufgaben oder Unannehmlichkeiten bei der Verwendung tragbarer Sensoren. In dieser Arbeit wird daher eine einfache, leicht zu implementierende und kostengĂŒnstige Methode zur Erfassung von Gangdaten vorgeschlagen. Diese Methode basiert auf der Aufnahme von Gehvideos mit einer Smartphone-Kamera in einer hĂ€uslichen Umgebung unter freien Bedingungen. Deep neural network (NN) verarbeitet dann diese Videos, um die Gangereignisse zu extrahieren. Die erkannten Ereignisse werden dann weiter verwendet, um verschiedene rĂ€umlich-zeitliche Parameter des Gangs zu quantifizieren, die fĂŒr jedes Ganganalysesystem wichtig sind. In dieser Arbeit wurden Gangvideos verwendet, die mit einer Smartphone-Kamera mit geringer Auflösung außerhalb der Laborumgebung aufgenommen wurden. Viele Deep- Learning-basierte NNs wurden implementiert, um die grundlegenden Gangereignisse wie die Fußposition in Bezug auf den Boden aus diesen Videos zu erkennen. In der ersten Studie wurde die Architektur von AlexNet verwendet, um das Modell anhand von Gehvideos und öffentlich verfĂŒgbaren DatensĂ€tzen von Grund auf zu trainieren. Mit diesem Modell wurde eine Gesamtgenauigkeit von 74% erreicht. Im nĂ€chsten Schritt wurde jedoch die LSTM-Schicht in dieselbe Architektur integriert. Die eingebaute FĂ€higkeit von LSTM in Bezug auf die zeitliche Information fĂŒhrte zu einer verbesserten Vorhersage der Etiketten fĂŒr die Fußposition, und es wurde eine Genauigkeit von 91% erreicht. Allerdings gibt es Schwierigkeiten bei der Vorhersage der richtigen Bezeichnungen in der letzten Phase des Schwungs und der Standphase jedes Fußes. Im nĂ€chsten Schritt wird das Transfer-Lernen eingesetzt, um die Vorteile von bereits trainierten tiefen NNs zu nutzen, indem vortrainierte Gewichte verwendet werden. Zwei bekannte Modelle, inceptionresnetv2 (IRNV-2) und densenet201 (DN-201), wurden mit ihren gelernten Gewichten fĂŒr das erneute Training des NN auf neuen Daten verwendet. Das auf Transfer-Lernen basierende vortrainierte NN verbesserte die Vorhersage von Kennzeichnungen fĂŒr verschiedene Fußpositionen. Es reduzierte insbesondere die Schwankungen in den Vorhersagen in der letzten Phase des Gangschwungs und der Standphase. Bei der Vorhersage der Klassenbezeichnungen der Testdaten wurde eine Genauigkeit von 94% erreicht. Da die Abweichung bei der Vorhersage des wahren Labels hauptsĂ€chlich ein Bild betrug, konnte sie bei einer Bildrate von 30 Bildern pro Sekunde ignoriert werden. Die vorhergesagten Markierungen wurden verwendet, um verschiedene rĂ€umlich-zeitliche Parameter des Gangs zu extrahieren, die fĂŒr jedes Ganganalysesystem entscheidend sind. Insgesamt wurden 12 Gangparameter quantifiziert und mit der durch Beobachtungsmethoden gewonnenen Grundwahrheit verglichen. Die NN-basierten rĂ€umlich-zeitlichen Parameter zeigten eine hohe Korrelation mit der Grundwahrheit, und in einigen FĂ€llen wurde eine sehr hohe Korrelation erzielt. Die Ergebnisse belegen die NĂŒtzlichkeit der vorgeschlagenen Methode. DerWert des Parameters ĂŒber die Zeit ergab eine Zeitreihe, eine langfristige Darstellung des Ganges. Diese Zeitreihe konnte mit verschiedenen mathematischen Methoden weiter analysiert werden. Als dritter Beitrag in dieser Dissertation wurden Verbesserungen an den bestehenden mathematischen Methoden der Zeitreihenanalyse von zeitlichen Gangdaten vorgeschlagen. Zu diesem Zweck werden zwei Verfeinerungen bestehender entropiebasierter Methoden zur Analyse von Schrittintervall-Zeitreihen vorgeschlagen. Diese Verfeinerungen wurden an Schrittintervall-Zeitseriendaten von normalen und neurodegenerativen Erkrankungen validiert, die aus der öffentlich zugĂ€nglichen Datenbank PhysioNet heruntergeladen wurden. Die Ergebnisse zeigten, dass die von uns vorgeschlagene Methode eine klare Trennung zwischen gesunden und kranken Gruppen ermöglicht. In Zukunft könnten fortschrittliche medizinische UnterstĂŒtzungssysteme, die kĂŒnstliche Intelligenz nutzen und von den hier vorgestellten Methoden abgeleitet sind, Ärzte bei der Diagnose und langfristigen Überwachung des Gangs von Patienten unterstĂŒtzen und so die klinische Arbeitsbelastung verringern und die Patientensicherheit verbessern
    • 

    corecore