ELECTRAMed: a new pre-trained language representation model for biomedical NLP

The overwhelming amount of biomedical scientific texts calls for the development of effective language models able to tackle a wide range of biomedical natural language processing (NLP) tasks. The most recent dominant approaches are domain-specific models, initialized with general-domain textual data and then trained on a variety of scientific corpora. However, it has been observed that for specialized domains in which large corpora exist, training a model from scratch with just in-domain knowledge may yield better results. Moreover, the increasing focus on the compute costs for pre-training recently led to the design of more efficient architectures, such as ELECTRA. In this paper, we propose a pre-trained domain-specific language model, called ELECTRAMed, suited for the biomedical field. The novel approach inherits the learning framework of the general-domain ELECTRA architecture, as well as its computational advantages. Experiments performed on benchmark datasets for several biomedical NLP tasks support the usefulness of ELECTRAMed, which sets the novel state-of-the-art result on the BC5CDR corpus for named entity recognition, and provides the best outcome in 2 over the 5 runs of the 7th BioASQ-factoid Challange for the question answering task

Fault Detection and Diagnosis with Imbalanced and Noisy Data: A Hybrid Framework for Rotating Machinery

Fault diagnosis plays an essential role in reducing the maintenance costs of rotating machinery manufacturing systems. In many real applications of fault detection and diagnosis, data tend to be imbalanced, meaning that the number of samples for some fault classes is much less than the normal data samples. At the same time, in an industrial condition, accelerometers encounter high levels of disruptive signals and the collected samples turn out to be heavily noisy. As a consequence, many traditional Fault Detection and Diagnosis (FDD) frameworks get poor classification performances when dealing with real-world circumstances. Three main solutions have been proposed in the literature to cope with this problem: (1) the implementation of generative algorithms to increase the amount of under-represented input samples, (2) the employment of a classifier being powerful to learn from imbalanced and noisy data, (3) the development of an efficient data pre-processing including feature extraction and data augmentation. This paper proposes a hybrid framework which uses the three aforementioned components to achieve an effective signal-based FDD system for imbalanced conditions. Specifically, it first extracts the fault features, using Fourier and wavelet transforms to make full use of the signals. Then, it employs Wasserstein Generative Adversarial Networks (WGAN) to generate synthetic samples to populate the rare fault class and enhance the training set. Moreover, to achieve a higher performance a novel combination of Convolutional Long Short-term Memory (CLSTM) and Weighted Extreme Learning Machine (WELM) is proposed. To verify the effectiveness of the developed framework, different datasets settings on different imbalance severities and noise degrees were used. The comparative results demonstrate that in different scenarios GAN-CLSTM-ELM outperforms the other state-of-the-art FDD frameworks.Comment: 23 pages, 11 figure

CoV-ABM: A stochastic discrete-event agent-based framework to simulate spatiotemporal dynamics of COVID-19

The paper develops a stochastic Agent-Based Model (ABM) mimicking the spread of infectious diseases in geographical domains. The model is designed to simulate the spatiotemporal spread of SARS-CoV2 disease, known as COVID-19. Our SARS-CoV2-based ABM framework (CoV-ABM) simulates the spread at any geographical scale, ranging from a village to a country and considers unique characteristics of SARS-CoV2 viruses such as its persistence in the environment. Therefore, unlike other simulators, CoV-ABM computes the density of active viruses inside each location space to get the virus transmission probability for each agent. It also uses the local census and health data to create health and risk factor profiles for each individual. The proposed model relies on a flexible timestamp scale to optimize the computational speed and the level of detail. In our framework each agent represents a person interacting with the surrounding space and other adjacent agents inside the same space. Moreover, families stochastic daily tasks are formulated to get tracked by the corresponding family members. The model also formulates the possibility of meetings for each subset of friendships and relatives. The main aim of the proposed framework is threefold: to illustrate the dynamics of SARS-CoV diseases, to identify places which have a higher probability to become infection hubs and to provide a decision-support system to design efficient interventions in order to fight against pandemics. The framework employs SEIHRD dynamics of viral diseases with different intervention scenarios. The paper simulates the spread of COVID-19 in the State of Delaware, United States, with near one million stochastic agents. The results achieved over a period of 15 weeks with a timestamp of 1 hour show which places become the hubs of infection. The paper also illustrates how hospitals get overwhelmed as the outbreak reaches its pick