An Investigation of Recurrent Neural Architectures for Drug Name Recognition

Drug name recognition (DNR) is an essential step in the Pharmacovigilance (PV) pipeline. DNR aims to find drug name mentions in unstructured biomedical texts and classify them into predefined categories. State-of-the-art DNR approaches heavily rely on hand crafted features and domain specific resources which are difficult to collect and tune. For this reason, this paper investigates the effectiveness of contemporary recurrent neural architectures - the Elman and Jordan networks and the bidirectional LSTM with CRF decoding - at performing DNR straight from the text. The experimental results achieved on the authoritative SemEval-2013 Task 9.1 benchmarks show that the bidirectional LSTM-CRF ranks closely to highly-dedicated, hand-crafted systems.Comment: Accepted for Oral Presentation at LOUHI 2016 : EMNLP 2016 Workshop - The Seventh International Workshop on Health Text Mining and Information Analysis (LOUHI 2016

Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition

© 2017 Elsevier Inc. Background Previous state-of-the-art systems on Drug Name Recognition (DNR) and Clinical Concept Extraction (CCE) have focused on a combination of text “feature engineering” and conventional machine learning algorithms such as conditional random fields and support vector machines. However, developing good features is inherently heavily time-consuming. Conversely, more modern machine learning approaches such as recurrent neural networks (RNNs) have proved capable of automatically learning effective features from either random assignments or automated word “embeddings”. Objectives (i) To create a highly accurate DNR and CCE system that avoids conventional, time-consuming feature engineering. (ii) To create richer, more specialized word embeddings by using health domain datasets such as MIMIC-III. (iii) To evaluate our systems over three contemporary datasets. Methods Two deep learning methods, namely the Bidirectional LSTM and the Bidirectional LSTM-CRF, are evaluated. A CRF model is set as the baseline to compare the deep learning systems to a traditional machine learning approach. The same features are used for all the models. Results We have obtained the best results with the Bidirectional LSTM-CRF model, which has outperformed all previously proposed systems. The specialized embeddings have helped to cover unusual words in DrugBank and MedLine, but not in the i2b2/VA dataset. Conclusions We present a state-of-the-art system for DNR and CCE. Automated word embeddings has allowed us to avoid costly feature engineering and achieve higher accuracy. Nevertheless, the embeddings need to be retrained over datasets that are adequate for the domain, in order to adequately cover the domain-specific vocabulary

DEEP LEARNING TECHNIQUES FOR DETECTION OF FALSE DATA INJECTION ATTACKS ON ELECTRIC POWER GRID

The electric power grid uses a set of measuring and switching devices for its operations and control. The data retrieved from the measuring instruments is assumed to be noisy, therefore a state estimator is used to estimate the correct values of state variables on which the system can take control actions. The modern electric power grid is dependent on communication networks for transferring these measurements, which are susceptible to intrusions from hackers. False data injection attacks (FDIA) are one of the most common attack strategies where an intruder tries to trick the underlying control system of the grid to cause disruptions without getting detected by native anomaly detection measures inbuilt in the state estimator. The native anomaly detection mechanism relies on threshold and residual based measure to flag a set of measurements as anomaly. Therefore, if the attack is devised in such a way that the intrusion can be performed without significantly affecting the residual error of state estimation it can go undetected. We propose a data augmented deep learning based solution to detect such attacks in real time. We propose methods of generating realistic random and targeted attack simulations on standard IEEE architectures and methods of detecting them using deep learning models. We propose recurrent neural network (RNN) based architectures to detect and locate FDIAs and devices compromised in real-time. For detection we propose a supervised and an unsupervised method. Similarly, for location we propose a method to find exact devices compromised which is less practical and then move on to a more feasible and practical solution in supervised and unsupervised conditions. Being an intrusion detection system it is critical to detect all attacks which means false negatives should be penalized heavily, whereas false positives can be accommodated. Therefore, we use recall as our primary performance metric and precision recall curve to find an optimal threshold of probability score. In addition, we demonstrate how our approach is better than a residual error and other previous detection models. We also compare the performance of our models with increasing number of devices being compromised

Detection of Stealthy False Data Injection Attacks Against State Estimation in Electric Power Grids Using Deep Learning Techniques

Since communication technologies are being integrated into smart grid, its vulnerability to false data injection is increasing. State estimation is a critical component which is used for monitoring the operation of power grid. However, a tailored attack could circumvent bad data detection of the state estimation, thus disturb the stability of the grid. Such attacks are called stealthy false data injection attacks (FDIAs). This thesis proposed a prediction-based detector using deep learning techniques to detect injected measurements. The proposed detector adopts both Convolutional Neural Networks and Recurrent Neural Networks, making full use of the spatial-temporal correlations in the measurement data. With its separable architecture, three discriminators with different feature extraction methods were designed for the predictor. Besides, a measurement restoration mechanism was proposed based on the prediction. The proposed detection mechanism was assessed by simulating FDIAs on the IEEE 39-bus system. The results demonstrated that the proposed mechanism could achieve a satisfactory performance compared with existing algorithms

A Novel Unsupervised Video Anomaly Detection Framework Based on Optical Flow Reconstruction and Erased Frame Prediction

Reconstruction-based and prediction-based approaches are widely used for video anomaly detection (VAD) in smart city surveillance applications. However, neither of these approaches can effectively utilize the rich contextual information that exists in videos, which makes it difficult to accurately perceive anomalous activities. In this paper, we exploit the idea of a training model based on the “Cloze Test” strategy in natural language processing (NLP) and introduce a novel unsupervised learning framework to encode both motion and appearance information at an object level. Specifically, to store the normal modes of video activity reconstructions, we first design an optical stream memory network with skip connections. Secondly, we build a space–time cube (STC) for use as the basic processing unit of the model and erase a patch in the STC to form the frame to be reconstructed. This enables a so-called ”incomplete event (IE)” to be completed. On this basis, a conditional autoencoder is utilized to capture the high correspondence between optical flow and STC. The model predicts erased patches in IEs based on the context of the front and back frames. Finally, we employ a generating adversarial network (GAN)-based training method to improve the performance of VAD. By distinguishing the predicted erased optical flow and erased video frame, the anomaly detection results are shown to be more reliable with our proposed method which can help reconstruct the original video in IE. Comparative experiments conducted on the benchmark UCSD Ped2, CUHK Avenue, and ShanghaiTech datasets demonstrate AUROC scores reaching 97.7%, 89.7%, and 75.8%, respectively

DEEP LEARNING TECHNIQUES FOR DETECTION OF FALSE DATA INJECTION ATTACKS ON ELECTRIC POWER GRID

The electric power grid uses a set of measuring and switching devices for its operations and control. The data retrieved from the measuring instruments is assumed to be noisy, therefore a state estimator is used to estimate the correct values of state variables on which the system can take control actions. The modern electric power grid is dependent on communication networks for transferring these measurements, which are susceptible to intrusions from hackers. False data injection attacks (FDIA) are one of the most common attack strategies where an intruder tries to trick the underlying control system of the grid to cause disruptions without getting detected by native anomaly detection measures inbuilt in the state estimator. The native anomaly detection mechanism relies on threshold and residual based measure to flag a set of measurements as anomaly. Therefore, if the attack is devised in such a way that the intrusion can be performed without significantly affecting the residual error of state estimation it can go undetected. We propose a data augmented deep learning based solution to detect such attacks in real time. We propose methods of generating realistic random and targeted attack simulations on standard IEEE architectures and methods of detecting them using deep learning models. We propose recurrent neural network (RNN) based architectures to detect and locate FDIAs and devices compromised in real-time. For detection we propose a supervised and an unsupervised method. Similarly, for location we propose a method to find exact devices compromised which is less practical and then move on to a more feasible and practical solution in supervised and unsupervised conditions. Being an intrusion detection system it is critical to detect all attacks which means false negatives should be penalized heavily, whereas false positives can be accommodated. Therefore, we use recall as our primary performance metric and precision recall curve to find an optimal threshold of probability score. In addition, we demonstrate how our approach is better than a residual error and other previous detection models. We also compare the performance of our models with increasing number of devices being compromised