The Application of Data Analytics Technologies for the Predictive Maintenance of Industrial Facilities in Internet of Things (IoT) Environments

In industrial production environments, the maintenance of equipment has a decisive influence on costs and on the plannability of production capacities. In particular, unplanned failures during production times cause high costs, unplanned downtimes and possibly additional collateral damage. Predictive Maintenance starts here and tries to predict a possible failure and its cause so early that its prevention can be prepared and carried out in time. In order to be able to predict malfunctions and failures, the industrial plant with its characteristics, as well as wear and ageing processes, must be modelled. Such modelling can be done by replicating its physical properties. However, this is very complex and requires enormous expert knowledge about the plant and about wear and ageing processes of each individual component. Neural networks and machine learning make it possible to train such models using data and offer an alternative, especially when very complex and non-linear behaviour is evident. In order for models to make predictions, as much data as possible about the condition of a plant and its environment and production planning data is needed. In Industrial Internet of Things (IIoT) environments, the amount of available data is constantly increasing. Intelligent sensors and highly interconnected production facilities produce a steady stream of data. The sheer volume of data, but also the steady stream in which data is transmitted, place high demands on the data processing systems. If a participating system wants to perform live analyses on the incoming data streams, it must be able to process the incoming data at least as fast as the continuous data stream delivers it. If this is not the case, the system falls further and further behind in processing and thus in its analyses. This also applies to Predictive Maintenance systems, especially if they use complex and computationally intensive machine learning models. If sufficiently scalable hardware resources are available, this may not be a problem at first. However, if this is not the case or if the processing takes place on decentralised units with limited hardware resources (e.g. edge devices), the runtime behaviour and resource requirements of the type of neural network used can become an important criterion. This thesis addresses Predictive Maintenance systems in IIoT environments using neural networks and Deep Learning, where the runtime behaviour and the resource requirements are relevant. The question is whether it is possible to achieve better runtimes with similarly result quality using a new type of neural network. The focus is on reducing the complexity of the network and improving its parallelisability. Inspired by projects in which complexity was distributed to less complex neural subnetworks by upstream measures, two hypotheses presented in this thesis emerged: a) the distribution of complexity into simpler subnetworks leads to faster processing overall, despite the overhead this creates, and b) if a neural cell has a deeper internal structure, this leads to a less complex network. Within the framework of a qualitative study, an overall impression of Predictive Maintenance applications in IIoT environments using neural networks was developed. Based on the findings, a novel model layout was developed named Sliced Long Short-Term Memory Neural Network (SlicedLSTM). The SlicedLSTM implements the assumptions made in the aforementioned hypotheses in its inner model architecture. Within the framework of a quantitative study, the runtime behaviour of the SlicedLSTM was compared with that of a reference model in the form of laboratory tests. The study uses synthetically generated data from a NASA project to predict failures of modules of aircraft gas turbines. The dataset contains 1,414 multivariate time series with 104,897 samples of test data and 160,360 samples of training data. As a result, it could be proven for the specific application and the data used that the SlicedLSTM delivers faster processing times with similar result accuracy and thus clearly outperforms the reference model in this respect. The hypotheses about the influence of complexity in the internal structure of the neuronal cells were confirmed by the study carried out in the context of this thesis

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways

MicroRNA Dysregulation in Neuropsychiatric Disorders and Cognitive Dysfunction

MicroRNAs (miRNAs) are evolutionarily-conserved small non-coding RNAs that are important posttranscriptional regulators of gene expression. Genetic Variants may cause microRNA dysregulation and the concomitant aberrant target expression. The dysregulation of one or a few targets may in turn lead to functional consequences ranging from phenotypic variations to disease conditions. In this thesis, I present our studies of mouse models of two human genetic variants - a rare copy number variant (CNV), 22q11.2 microdeletions, and a common single nucleotide polymorphism (SNP), BDNF Val66Met. 22q11.2 microdeletions result in specific cognitive deficits and high risk to develop schizophrenia. Analysis of Df(16)A+/- mice, which model this microdeletion, revealed abnormalities in the formation of neuronal dendrites and spines as well as microRNA dysregulation in brain. We show a drastic reduction of miR- 185, which resides within the 22q11.2 locus, to levels more than expected by a hemizygous deletion and demonstrate that this reduction impairs dendritic and spine development. miR-185 targets and represses, through an evolutionary conserved target site, a previously unknown inhibitor of these processes that resides in the Golgi apparatus. Sustained derepression of this inhibitor after birth represents the most robust transcriptional disturbance in the brains of Df(16)A+/- mice and could affect the formation and maintenance of neural circuits. Reduction of miR-185 also has milder effects on the expression of a group of Golgi-related genes. One the other hand, BNDF Val66Met results in impaired activity-dependent secretion of BDNF from neuronal terminals and affects episodic memory and affective behaviors. We found a modest reduction of miR-146b which causes derepression of mRNA and/or protein levels of a few targets. Our findings add to the growing evidence of the pivotal involvement of miRNAs in the development of neuropsychiatric disorders and cognitive dysfunction. In addition, the identification of key players in miRNA dysregulation has implications for both basic and translational research in psychiatric disorders and cognitive dysfunction

Neural recommender models for sparse and skewed behavioral data

Modern online platforms offer recommendations and personalized search and services to a large and diverse user base while still aiming to acquaint users with the broader community on the platform. Prior work backed by large volumes of user data has shown that user retention is reliant on catering to their specific eccentric tastes, in addition to providing them popular services or content on the platform. Long-tailed distributions are a fundamental characteristic of human activity, owing to the bursty nature of human attention. As a result, we often observe skew in data facets that involve human interaction. While there are superficial similarities to Zipf's law in textual data and other domains, the challenges with user data extend further. Individual words may have skewed frequencies in the corpus, but the long-tail words by themselves do not significantly impact downstream text-mining tasks. On the contrary, while sparse users (a majority on most online platforms) contribute little to the training data, they are equally crucial at inference time. Perhaps more so, since they are likely to churn. In this thesis, we study platforms and applications that elicit user participation in rich social settings incorporating user-generated content, user-user interaction, and other modalities of user participation and data generation. For instance, users on the Yelp review platform participate in a follower-followee network and also create and interact with review text (two modalities of user data). Similarly, community question-answer (CQA) platforms incorporate user interaction and collaboratively authored content over diverse domains and discussion threads. Since user participation is multimodal, we develop generalizable abstractions beyond any single data modality. Specifically, we aim to address the distributional mismatch that occurs with user data independent of dataset specifics; While a minority of the users generates most training samples, it is insufficient only to learn the preferences of this subset of users. As a result, the data's overall skew and individual users' sparsity are closely interlinked: sparse users with uncommon preferences are under-represented. Thus, we propose to treat these problems jointly with a skew-aware grouping mechanism that iteratively sharpens the identification of preference groups within the user population. As a result, we improve user characterization; content recommendation and activity prediction (+6-22% AUC, +6-43% AUC, +12-25% RMSE over state-of-the-art baselines), primarily for users with sparse activity. The size of the item or content inventories compounds the skew problem. Recommendation models can achieve very high aggregate performance while recommending only a tiny proportion of the inventory (as little as 5%) to users. We propose a data-driven solution guided by the aggregate co-occurrence information across items in the dataset. We specifically note that different co-occurrences are not equally significant; For example, some co-occurring items are easily substituted while others are not. We develop a self-supervised learning framework where the aggregate co-occurrences guide the recommendation problem while providing room to learn these variations among the item associations. As a result, we improve coverage to ~100% (up from 5%) of the inventory and increase long-tail item recall up to 25%. We also note that the skew and sparsity problems repeat across data modalities. For instance, social interactions and review content both exhibit aggregate skew, although individual users who actively generate reviews may not participate socially and vice-versa. It is necessary to differentially weight and merge different data sources for each user towards inference tasks in such cases. We show that the problem is inherently adversarial since the user participation modalities compete to describe a user accurately. We develop a framework to unify these representations while algorithmically tackling mode collapse, a well-known pitfall with adversarial models. A more challenging but important instantiation of sparsity is the few-shot setting or cross-domain setting. We may only have a single or a few interactions for users or items in the sparse domains or partitions. We show that contextualizing user-item interactions helps us infer behavioral invariants in the dense domain, allowing us to correlate sparse participants to their active counterparts (resulting in 3x faster training, ~19% recall gains in multi-domain settings). Finally, we consider the multi-task setting, where the platform incorporates multiple distinct recommendations and prediction tasks for each user. A single-user representation is insufficient for users who exhibit different preferences along each dimension. At the same time, it is counter-productive to handle correlated prediction or inference tasks in isolation. We develop a multi-faceted representation approach grounded on residual learning with heterogeneous knowledge graph representations, which provides us an expressive data representation for specialized domains and applications with multimodal user data. We achieve knowledge sharing by unifying task-independent and task-specific representations of each entity with a unified knowledge graph framework. In each chapter, we also discuss and demonstrate how the proposed frameworks directly incorporate a wide range of gradient-optimizable recommendation and behavior models, maximizing their applicability and pertinence to user-centered inference tasks and platforms

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways

Characterisation of genetic risk factors for mental illness in rodent models, impact of Map2k7+/- and Fxyd6-/- mice on neural systems and working memory

Even in wealthy and seemingly prosperous countries like the United Kingdom, the spectre of mental illness and psychiatric disorders remains highly prevalent. These disorders present a huge economic burden to societies, where in the UK alone, mental disorders cost the economy an estimated €134 billion a year; along with the unmeasurable societal and human costs. This has led to an intense debate over the past few decades just as to what factors contribute to these illnesses. It is now understood that a number of biological and non-biological factors contribute. These include socio-economic pressures, early-life trauma, gestational and peri-natal infections; genetic and familial factors, and molecular and cellular factors. However, while the definitions and diagnostic criteria of mental disorders remain based in the subjective realms of the DSM and ICD, treatment and understanding of psychiatric illness has had little chance to progress over the last fifty years. As a result, neuroscientists are starting to direct psychiatric disorder research from the bottom-up; where genetic, cognitive and neuroconnectivity factors are being investigated to serve as a future basis for diagnosis and treatment. One of the most complex and debilitating psychiatric disorders, schizophrenia, exhibits a complex array of genetic, cognitive and neuroconnectivity abnormalities. Current challenges in schizophrenia research is to understand how identified genetic abnormalities contribute to neuroconnectivity and cognitive impairments which are prominent in schizophrenia. Recently, genetic association studies have implicated two genes as risk factors for schizophrenia - FXYD6 and MAP2K7. Currently it is unclear exactly how these genes contribute to schizophrenia pathology, particularly cognitive symptoms and neural circuitry.;This thesis investigates these two genes by utilising two mouse models, first a heterozygous mouse line of Map2k7+/- and second, a gene knock-out line of Fxyd6-/-. MAP2K7 is a gene that expresses a kinase that is involved in the c-Jun N-terminal kinase (JNK) pathway, which is implicated in neuronal activity, receptor function, and cortical and hippocampal plasticity. Recent studies have found a decreased expression of MA2PK7 in the PFC, ACC and hippocampal regions in schizophrenia patients; regions associated with memory and decision making. A component of the cognitive profile of MAP2K7 was therefore investigated using Map2k7+/- mouse lines in a working memory paradigm in the radial arm maze. This test is known as the n-back test or the retention interval test. For the first time this investigation reveals that Map2k7+/- mice exhibit a subtle yet significant spatial working memory deficit compared to WT mice; as judged by their average performance over the whole experiment. WT mice exhibited an overall average performance of 70% and MAP2K7+/- mice 66% (p<0.001). This indicates that MAP2K7 may play a subtle role in working memory function in rodents, and may represent a component of the aberrations in the genetic architecture that gives rise to working memory impairments in psychiatric disorders, particularly schizophrenia. This experiment also backs up previous evidence for this radial arm maze paradigm as a robust behavioural test for testing rodent working memory.;FXYD6 belongs to a group of proteins that are known to be involved in modulating NaKATPase activity. Previously, NaKATPase has been associated with bipolar disorder and depression, but has now also been implicated in schizophrenia. Previous studies have found that FXYD6 is also abnormally expressed in the PFC of schizophrenia patients, and therefore may contribute to the cognate symptoms of the disorder. This experiment, therefore, investigated how Fxyd6 contributes to local brain activation, particularly in neural systems relevant to cognition, using gene knockout Fxyd6-/- mouse models and semi quantitative 2DG autoradiographic imaging. Three regions showed a significant deviation in activity in Fxyd6-/- mice compared to WT mice. The subiculum, medial septum and lateral septum all exhibited significant reductions in activity in Fxyd6-/- mice compared to WT mice. Notably the subiculum is heavily implicated with memory functions, particularly working memory and disambiguation of previously learned memory. Indicating a possible role for FXYD6 and NaKATPase in working memory processing and memory disambiguation in the subiculum. Finally, the role of glutamate in relation to FXYD6 function and brain activity was assessed by administering the NMDA receptor antagonist ketamine and analysing regional brain activity using semi quantitative 2DG autoradiographic imaging. Generally, regions which were affected by ketamine in WT mice including PFC, thalamic and septal regions, were not affected in Fxyd6-/- mice. It is hypothesized that this may be down to a compensatory effect that knocking-out Fxyd6 may have on glutamate reuptake. Because NaKATPase is involved in glutamate reuptake into glia and neurons, the blockage of NMDA receptors may have less effect due to a reduction in glutamate reuptake, and therefore higher than normal postsynaptic glutamate concentrations. In conclusion, this investigation highlights two genes which may have roles in working memory functioning and neural circuitry that contribute to cognitive processes. While the evidence from this investigation does not explicitly associate these genes with symptoms of schizophrenia and other psychiatric disorders; the evidence does provide indication that they are involved in cognitive processes in rodents, and possibly humans. This investigation provides an interesting path of investigation for the potential roles of these genes regardless of their relationship to psychiatric disorders and will inform future research into the genetic architecture of neural circuits and cognition.Even in wealthy and seemingly prosperous countries like the United Kingdom, the spectre of mental illness and psychiatric disorders remains highly prevalent. These disorders present a huge economic burden to societies, where in the UK alone, mental disorders cost the economy an estimated €134 billion a year; along with the unmeasurable societal and human costs. This has led to an intense debate over the past few decades just as to what factors contribute to these illnesses. It is now understood that a number of biological and non-biological factors contribute. These include socio-economic pressures, early-life trauma, gestational and peri-natal infections; genetic and familial factors, and molecular and cellular factors. However, while the definitions and diagnostic criteria of mental disorders remain based in the subjective realms of the DSM and ICD, treatment and understanding of psychiatric illness has had little chance to progress over the last fifty years. As a result, neuroscientists are starting to direct psychiatric disorder research from the bottom-up; where genetic, cognitive and neuroconnectivity factors are being investigated to serve as a future basis for diagnosis and treatment. One of the most complex and debilitating psychiatric disorders, schizophrenia, exhibits a complex array of genetic, cognitive and neuroconnectivity abnormalities. Current challenges in schizophrenia research is to understand how identified genetic abnormalities contribute to neuroconnectivity and cognitive impairments which are prominent in schizophrenia. Recently, genetic association studies have implicated two genes as risk factors for schizophrenia - FXYD6 and MAP2K7. Currently it is unclear exactly how these genes contribute to schizophrenia pathology, particularly cognitive symptoms and neural circuitry.;This thesis investigates these two genes by utilising two mouse models, first a heterozygous mouse line of Map2k7+/- and second, a gene knock-out line of Fxyd6-/-. MAP2K7 is a gene that expresses a kinase that is involved in the c-Jun N-terminal kinase (JNK) pathway, which is implicated in neuronal activity, receptor function, and cortical and hippocampal plasticity. Recent studies have found a decreased expression of MA2PK7 in the PFC, ACC and hippocampal regions in schizophrenia patients; regions associated with memory and decision making. A component of the cognitive profile of MAP2K7 was therefore investigated using Map2k7+/- mouse lines in a working memory paradigm in the radial arm maze. This test is known as the n-back test or the retention interval test. For the first time this investigation reveals that Map2k7+/- mice exhibit a subtle yet significant spatial working memory deficit compared to WT mice; as judged by their average performance over the whole experiment. WT mice exhibited an overall average performance of 70% and MAP2K7+/- mice 66% (p<0.001). This indicates that MAP2K7 may play a subtle role in working memory function in rodents, and may represent a component of the aberrations in the genetic architecture that gives rise to working memory impairments in psychiatric disorders, particularly schizophrenia. This experiment also backs up previous evidence for this radial arm maze paradigm as a robust behavioural test for testing rodent working memory.;FXYD6 belongs to a group of proteins that are known to be involved in modulating NaKATPase activity. Previously, NaKATPase has been associated with bipolar disorder and depression, but has now also been implicated in schizophrenia. Previous studies have found that FXYD6 is also abnormally expressed in the PFC of schizophrenia patients, and therefore may contribute to the cognate symptoms of the disorder. This experiment, therefore, investigated how Fxyd6 contributes to local brain activation, particularly in neural systems relevant to cognition, using gene knockout Fxyd6-/- mouse models and semi quantitative 2DG autoradiographic imaging. Three regions showed a significant deviation in activity in Fxyd6-/- mice compared to WT mice. The subiculum, medial septum and lateral septum all exhibited significant reductions in activity in Fxyd6-/- mice compared to WT mice. Notably the subiculum is heavily implicated with memory functions, particularly working memory and disambiguation of previously learned memory. Indicating a possible role for FXYD6 and NaKATPase in working memory processing and memory disambiguation in the subiculum. Finally, the role of glutamate in relation to FXYD6 function and brain activity was assessed by administering the NMDA receptor antagonist ketamine and analysing regional brain activity using semi quantitative 2DG autoradiographic imaging. Generally, regions which were affected by ketamine in WT mice including PFC, thalamic and septal regions, were not affected in Fxyd6-/- mice. It is hypothesized that this may be down to a compensatory effect that knocking-out Fxyd6 may have on glutamate reuptake. Because NaKATPase is involved in glutamate reuptake into glia and neurons, the blockage of NMDA receptors may have less effect due to a reduction in glutamate reuptake, and therefore higher than normal postsynaptic glutamate concentrations. In conclusion, this investigation highlights two genes which may have roles in working memory functioning and neural circuitry that contribute to cognitive processes. While the evidence from this investigation does not explicitly associate these genes with symptoms of schizophrenia and other psychiatric disorders; the evidence does provide indication that they are involved in cognitive processes in rodents, and possibly humans. This investigation provides an interesting path of investigation for the potential roles of these genes regardless of their relationship to psychiatric disorders and will inform future research into the genetic architecture of neural circuits and cognition

ATHENA Research Book

The ATHENA European University is an alliance of nine Higher Education Institutions with the mission of fostering excellence in research and innovation by facilitating international cooperation. The ATHENA acronym stands for Advanced Technologies in Higher Education Alliance. The partner institutions are from France, Germany, Greece, Italy, Lithuania, Portugal, and Slovenia: the University of Orléans, the University of Siegen, the Hellenic Mediterranean University, the Niccolò Cusano University, the Vilnius Gediminas Technical University, the Polytechnic Institute of Porto, and the University of Maribor. In 2022 institutions from Poland and Spain joined the alliance: the Maria Curie-Skłodowska University and the University of Vigo. This research book presents a selection of the ATHENA university partners' research activities. It incorporates peer-reviewed original articles, reprints and student contributions. The ATHENA Research Book provides a platform that promotes joint and interdisciplinary research projects of both advanced and early-career researchers

Intelligent Systems

This book is dedicated to intelligent systems of broad-spectrum application, such as personal and social biosafety or use of intelligent sensory micro-nanosystems such as "e-nose", "e-tongue" and "e-eye". In addition to that, effective acquiring information, knowledge management and improved knowledge transfer in any media, as well as modeling its information content using meta-and hyper heuristics and semantic reasoning all benefit from the systems covered in this book. Intelligent systems can also be applied in education and generating the intelligent distributed eLearning architecture, as well as in a large number of technical fields, such as industrial design, manufacturing and utilization, e.g., in precision agriculture, cartography, electric power distribution systems, intelligent building management systems, drilling operations etc. Furthermore, decision making using fuzzy logic models, computational recognition of comprehension uncertainty and the joint synthesis of goals and means of intelligent behavior biosystems, as well as diagnostic and human support in the healthcare environment have also been made easier

In Search of a Common Thread: Enhancing the LBD Workflow with a view to its Widespread Applicability

Literature-Based Discovery (LBD) research focuses on discovering implicit knowledge linkages in existing scientific literature to provide impetus to innovation and research productivity. Despite significant advancements in LBD research, previous studies contain several open problems and shortcomings that are hindering its progress. The overarching goal of this thesis is to address these issues, not only to enhance the discovery component of LBD, but also to shed light on new directions that can further strengthen the existing understanding of the LBD work ow. In accordance with this goal, the thesis aims to enhance the LBD work ow with a view to ensuring its widespread applicability. The goal of widespread applicability is twofold. Firstly, it relates to the adaptability of the proposed solutions to a diverse range of problem settings. These problem settings are not necessarily application areas that are closely related to the LBD context, but could include a wide range of problems beyond the typical scope of LBD, which has traditionally been applied to scientific literature. Adapting the LBD work ow to problems outside the typical scope of LBD is a worthwhile goal, since the intrinsic objective of LBD research, which is discovering novel linkages in text corpora is valid across a vast range of problem settings. Secondly, the idea of widespread applicability also denotes the capability of the proposed solutions to be executed in new environments. These `new environments' are various academic disciplines (i.e., cross-domain knowledge discovery) and publication languages (i.e., cross-lingual knowledge discovery). The application of LBD models to new environments is timely, since the massive growth of the scientific literature has engendered huge challenges to academics, irrespective of their domain. This thesis is divided into five main research objectives that address the following topics: literature synthesis, the input component, the discovery component, reusability, and portability. The objective of the literature synthesis is to address the gaps in existing LBD reviews by conducting the rst systematic literature review. The input component section aims to provide generalised insights on the suitability of various input types in the LBD work ow, focusing on their role and potential impact on the information retrieval cycle of LBD. The discovery component section aims to intermingle two research directions that have been under-investigated in the LBD literature, `modern word embedding techniques' and `temporal dimension' by proposing diachronic semantic inferences. Their potential positive in uence in knowledge discovery is veri ed through both direct and indirect uses. The reusability section aims to present a new, distinct viewpoint on these LBD models by verifying their reusability in a timely application area using a methodical reuse plan. The last section, portability, proposes an interdisciplinary LBD framework that can be applied to new environments. While highly cost-e cient and easily pluggable, this framework also gives rise to a new perspective on knowledge discovery through its generalisable capabilities. Succinctly, this thesis presents novel and distinct viewpoints to accomplish five main research objectives, enhancing the existing understanding of the LBD work ow. The thesis offers new insights which future LBD research could further explore and expand to create more eficient, widely applicable LBD models to enable broader community benefits.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 202