Demonstration of Semantic Web-based Medical Ontologies and Clinical Decision Support Systems

Master's thesis in Information- and communication technology IKT590 - University of Agder 2016Konfidensiell til / confidential until 01.01.202

SCRO: A Domain Ontology for Describing Steel Cold Rolling Processes towards Industry 4.0

This paper introduces the Steel Cold Rolling Ontology (SCRO) to model and capture domain knowledge of cold rolling processes and activities within a steel plant. A case study is set up that uses real-world cold rolling data sets to validate the performance and functionality of SCRO. This includes using the Ontop framework to deploy virtual knowledge graphs for data access, data integration, data querying, and condition-based maintenance purposes. SCRO is evaluated using OOPS!, the ontology pitfall detection system, and feedback from domain experts from Tata Steel

Semantic Asset Administration Shells in Industry 4.0: A Survey

The Asset Administration Shell (AAS) is a fundamental concept in the Reference Architecture Model for Industry 4.0 (RAMI 4.0), that provides a virtual and digital representation of all information and functions of a physical asset in a manufacturing environment. Recently, Semantic AASs have emerged that add knowledge representation formalisms to enhance the digital representation of physical assets. In this paper, we provide a comprehensive survey of the scientific contributions to Semantic AASs that model the Information and Communication Layer within RAMI 4.0, and summarise and demonstrate their structure, communication, functionalities, and use cases. We also highlight the challenges of future development of Semantic AASs

Exchanging-based Multimodal Fusion with Transformer

We study the problem of multimodal fusion in this paper. Recent exchanging-based methods have been proposed for vision-vision fusion, which aim to exchange embeddings learned from one modality to the other. However, most of them project inputs of multimodalities into different low-dimensional spaces and cannot be applied to the sequential input data. To solve these issues, in this paper, we propose a novel exchanging-based multimodal fusion model MuSE for text-vision fusion based on Transformer. We first use two encoders to separately map multimodal inputs into different low-dimensional spaces. Then we employ two decoders to regularize the embeddings and pull them into the same space. The two decoders capture the correlations between texts and images with the image captioning task and the text-to-image generation task, respectively. Further, based on the regularized embeddings, we present CrossTransformer, which uses two Transformer encoders with shared parameters as the backbone model to exchange knowledge between multimodalities. Specifically, CrossTransformer first learns the global contextual information of the inputs in the shallow layers. After that, it performs inter-modal exchange by selecting a proportion of tokens in one modality and replacing their embeddings with the average of embeddings in the other modality. We conduct extensive experiments to evaluate the performance of MuSE on the Multimodal Named Entity Recognition task and the Multimodal Sentiment Analysis task. Our results show the superiority of MuSE against other competitors. Our code and data are provided at https://github.com/RecklessRonan/MuSE

WavJourney: Compositional Audio Creation with Large Language Models

Large Language Models (LLMs) have shown great promise in integrating diverse expert models to tackle intricate language and vision tasks. Despite their significance in advancing the field of Artificial Intelligence Generated Content (AIGC), their potential in intelligent audio content creation remains unexplored. In this work, we tackle the problem of creating audio content with storylines encompassing speech, music, and sound effects, guided by text instructions. We present WavJourney, a system that leverages LLMs to connect various audio models for audio content generation. Given a text description of an auditory scene, WavJourney first prompts LLMs to generate a structured script dedicated to audio storytelling. The audio script incorporates diverse audio elements, organized based on their spatio-temporal relationships. As a conceptual representation of audio, the audio script provides an interactive and interpretable rationale for human engagement. Afterward, the audio script is fed into a script compiler, converting it into a computer program. Each line of the program calls a task-specific audio generation model or computational operation function (e.g., concatenate, mix). The computer program is then executed to obtain an explainable solution for audio generation. We demonstrate the practicality of WavJourney across diverse real-world scenarios, including science fiction, education, and radio play. The explainable and interactive design of WavJourney fosters human-machine co-creation in multi-round dialogues, enhancing creative control and adaptability in audio production. WavJourney audiolizes the human imagination, opening up new avenues for creativity in multimedia content creation.Comment: Project Page: https://audio-agi.github.io/WavJourney_demopage

Finishing the euchromatic sequence of the human genome

The sequence of the human genome encodes the genetic instructions for human physiology, as well as rich information about human evolution. In 2001, the International Human Genome Sequencing Consortium reported a draft sequence of the euchromatic portion of the human genome. Since then, the international collaboration has worked to convert this draft into a genome sequence with high accuracy and nearly complete coverage. Here, we report the result of this finishing process. The current genome sequence (Build 35) contains 2.85 billion nucleotides interrupted by only 341 gaps. It covers ∼99% of the euchromatic genome and is accurate to an error rate of ∼1 event per 100,000 bases. Many of the remaining euchromatic gaps are associated with segmental duplications and will require focused work with new methods. The near-complete sequence, the first for a vertebrate, greatly improves the precision of biological analyses of the human genome including studies of gene number, birth and death. Notably, the human enome seems to encode only 20,000-25,000 protein-coding genes. The genome sequence reported here should serve as a firm foundation for biomedical research in the decades ahead

Technologies sémantiques pour la modélisation de la maintenance prédictive pour un réseau de PME dans le cadre de l'industrie 4.0

In the manufacturing domain, the detection of anomalies such as mechanical faults and failures enables the launching of predictive maintenance tasks, which aim to predict future faults, errors, and failures and also enable maintenance actions. With the trend of Industry 4.0, predictive maintenance tasks are benefiting from advanced technologies such as Cyber-Physical Systems (CPS), the Internet of Things (IoT), and Cloud Computing. These advanced technologies enable the collection and processing of sensor data that contain measurements of physical signals of machinery, such as temperature, voltage, and vibration. However, due to the heterogeneous nature of industrial data, sometimes the knowledge extracted from industrial data is presented in a complex structure. Therefore formal knowledge representation methods are required to facilitate the understanding and exploitation of the knowledge. Furthermore, as the CPSs are becoming more and more knowledge-intensive, uniform knowledge representation of physical resources and reasoning capabilities for analytic tasks are needed to automate the decision-making processes in CPSs. These issues bring obstacles to machine operators to perform appropriate maintenance actions. To address the aforementioned challenges, in this thesis, we propose a novel semantic approach to facilitate predictive maintenance tasks in manufacturing processes. In particular, we propose four main contributions: i) a three-layered ontological framework that is the core component of a knowledge-based predictive maintenance system; ii) a novel hybrid semantic approach to automate machinery failure prediction tasks, which is based on the combined use of chronicles (a more descriptive type of sequential patterns) and semantic technologies; iii) a new approach that uses clustering methods with Semantic Web Rule Language (SWRL) rules to assess failures according to their criticality levels; iv) a novel rule base refinement approach that uses rule quality measures as references to refine a rule base within a knowledge-based predictive maintenance system. These approaches have been validated on both real-world and synthetic data sets.Dans le domaine de la fabrication, la détection d’anomalies telles que les défauts et les défaillances mécaniques permet de lancer des tâches de maintenance prédictive, qui visent à prévoir les défauts, les erreurs et les défaillances futurs et à permettre des actions de maintenance. Avec la tendance de l’industrie 4.0, les tâches de maintenance prédictive bénéficient de technologies avancées telles que les systèmes cyberphysiques (CPS), l’Internet des objets (IoT) et l’informatique dématérialisée (cloud computing). Ces technologies avancées permettent la collecte et le traitement de données de capteurs qui contiennent des mesures de signaux physiques de machines, tels que la température, la tension et les vibrations. Cependant, en raison de la nature hétérogène des données industrielles, les connaissances extraites des données industrielles sont parfois présentées dans une structure complexe. Des méthodes formelles de représentation des connaissances sont donc nécessaires pour faciliter la compréhension et l’exploitation des connaissances. En outre, comme les CPSs sont de plus en plus axées sur la connaissance, une représentation uniforme de la connaissance des ressources physiques et des capacités de raisonnement pour les tâches analytiques est nécessaire pour automatiser les processus de prise de décision dans les CPSs. Ces problèmes constituent des obstacles pour les opérateurs de machines qui doivent effectuer des opérations de maintenance appropriées. Pour relever les défis susmentionnés, nous proposons dans cette thèse une nouvelle approche sémantique pour faciliter les tâches de maintenance prédictive dans les processus de fabrication. En particulier, nous proposons quatre contributions principales: i) un cadre ontologique à trois niveaux qui est l’élément central d’un système de maintenance prédictive basé sur la connaissance; ii) une nouvelle approche sémantique hybride pour automatiser les tâches de prédiction des pannes de machines, qui est basée sur l’utilisation combinée de chroniques (un type plus descriptif de modèles séquentiels) et de technologies sémantiques; iii) a new approach that uses clustering methods with Semantic Web Rule Language (SWRL) rules to assess failures according to their criticality levels; iv) une nouvelle approche d’affinement de la base de règles qui utilise des mesures de qualité des règles comme références pour affiner une base de règles dans un système de maintenance prédictive basé sur la connaissance. Ces approches ont été validées sur des ensembles de données réelles et synthétiques