Multilinear EigenECGs and FisherECGs for Individual Identification from Information Obtained by an Electrocardiogram Sensor

In this study, we present a third-order tensor-based multilinear eigenECG (MEECG) and multilinear Fisher ECG (MFECG) for individual identification based on the information obtained by an electrocardiogram (ECG) sensor. MEECG and MFECG are based on multilinear principal component analysis (MPCA) and multilinear linear discriminant analysis (MLDA) in the field of multilinear subspace learning (MSL), respectively. MSL directly extracts features without the vectorization of input data, while MSL extracts features without vectorizing the input data while maintaining most of the correlations shown in the original structure. In contrast with unsupervised linear subspace learning (LSL) techniques such as PCA (Principal Component Analysis) and LDA (Linear Discriminant Analysis), it is less susceptible to small-data problems because it learns more compact and potentially useful representations, and it can efficiently handle large tensors. Here, the third-order tensor is formed by reordering the one-dimensional ECG signal into a two-dimensional matrix, considering the time frame. The MSL consists of four steps. The first step is preprocessing, in which input samples are centered. The second step is initialization, in which eigen decomposition is performed and the most significant eigenvectors are selected. The third step is local optimization, in which input data is applied by eigenvectors from the second step, and new eigenvectors are calculated using the applied input data. The final step is projection, in which the resultant feature tensors after projection are obtained. The experiments are performed on two databases for performance evaluation. The Physikalisch-Technische Bundesanstalt (PTB)-ECG is a well-known database, and Chosun University (CU)-ECG is directly built for this study using the developed ECG sensor. The experimental results revealed that the tensor-based MEECG and MFECG showed good identification performance in comparison to PCA and LDA of LSL

Individual Identification by Late Information Fusion of EmgCNN and EmgLSTM from Electromyogram Signals

This paper is concerned with individual identification by late fusion of two-stream deep networks from Electromyogram (EMG) signals. EMG signal has more advantages on security compared to other biosignals exposed visually, such as the face, iris, and fingerprints, when used for biometrics, at least in the aspect of visual exposure, because it is measured through contact without any visual exposure. Thus, we propose an ensemble deep learning model by late information fusion of convolutional neural networks (CNN) and long short-term memory (LSTM) from EMG signals for robust and discriminative biometrics. For this purpose, in the ensemble model’s first stream, one-dimensional EMG signals were converted into time–frequency representation to train a two-dimensional convolutional neural network (EmgCNN). In the second stream, statistical features were extracted from one-dimensional EMG signals to train a long short-term memory (EmgLSTM) that uses sequence input. Here, the EMG signals were divided into fixed lengths, and feature values were calculated for each interval. A late information fusion is performed by the output scores of two deep learning models to obtain a final classification result. To confirm the superiority of the proposed method, we use an EMG database constructed at Chosun University and a public EMG database. The experimental results revealed that the proposed method showed performance improvement by 10.76% on average compared to a single stream and the previous methods

A Design for Genetically Oriented Rules-Based Incremental Granular Models and Its Application

In this paper, we develop a genetically oriented rule-based Incremental Granular Model (IGM). The IGM is designed using a combination of a simple Linear Regression (LR) model and a local Linguistic Model (LM) to predict the modeling error obtained by the LR. The IGM has been successfully applied to various examples. However, the disadvantage of IGM is that the number of clusters in each context is determined, with the same number, by trial and error. Moreover, a weighting exponent is set to the typical value. In order to solve these problems, the goal of this paper is to design an optimized rule-based IGM with the use of a Genetic Algorithm (GA) to simultaneously optimize the number of cluster centers in each context, the number of contexts, and the weighting exponent. The experimental results regarding a coagulant dosing process in a water purification plant, an automobile mpg (miles per gallon) prediction, and a Boston housing data set revealed that the proposed GA-based IGM showed good performance, when compared with the Radial Basis Function Neural Network (RBFNN), LM, Takagi–Sugeno–Kang (TSK)-Linguistic Fuzzy Model (LFM), GA-based LM, and IGM itself

Pre-Configured Deep Convolutional Neural Networks with Various Time-Frequency Representations for Biometrics from ECG Signals

We evaluated electrocardiogram (ECG) biometrics using pre-configured models of convolutional neural networks (CNNs) with various time-frequency representations. Biometrics technology records a person’s physical or behavioral characteristics in a digital signal via a sensor and analyzes it to identify the person. An ECG signal is obtained by detecting and amplifying a minute electrical signal flowing on the skin using a noninvasive electrode when the heart muscle depolarizes at each heartbeat. In biometrics, the ECG is especially advantageous in security applications because the heart is located within the body and moves while the subject is alive. However, a few body states generate noisy biometrics. The analysis of signals in the frequency domain has a robust effect on the noise. As the ECG is noise-sensitive, various studies have applied time-frequency transformations that are robust to noise, with CNNs achieving a good performance in image classification. Studies have applied time-frequency representations of the 1D ECG signals to 2D CNNs using transforms like MFCC (mel frequency cepstrum coefficient), spectrogram, log spectrogram, mel spectrogram, and scalogram. CNNs have various pre-configured models such as VGGNet, GoogLeNet, ResNet, and DenseNet. Combinations of the time-frequency representations and pre-configured CNN models have not been investigated. In this study, we employed the PTB (Physikalisch-Technische Bundesanstalt)-ECG and CU (Chosun University)-ECG databases. The MFCC accuracies were 0.45%, 2.60%, 3.90%, and 0.25% higher than the spectrogram, log spectrogram, mel spectrogram, and scalogram accuracies, respectively. The Xception accuracies were 3.91%, 0.84%, and 1.14% higher than the VGGNet-19, ResNet-101, and DenseNet-201 accuracies, respectively

Intelligent Deep Models Based on Scalograms of Electrocardiogram Signals for Biometrics

This paper conducts a comparative analysis of deep models in biometrics using scalogram of electrocardiogram (ECG). A scalogram is the absolute value of the continuous wavelet transform coefficients of a signal. Since biometrics using ECG signals are sensitive to noise, studies have been conducted by transforming signals into a frequency domain that is efficient for analyzing noisy signals. By transforming the signal from the time domain to the frequency domain using the wavelet, the 1-D signal becomes a 2-D matrix, and it could be analyzed at multiresolution. However, this process makes signal analysis morphologically complex. This means that existing simple classifiers could perform poorly. We investigate the possibility of using the scalogram of ECG as input to deep convolutional neural networks of deep learning, which exhibit optimal performance for the classification of morphological imagery. When training data is small or hardware is insufficient for training, transfer learning can be used with pretrained deep models to reduce learning time, and classify it well enough. In this paper, AlexNet, GoogLeNet, and ResNet are considered as deep models of convolutional neural network. The experiments are performed on two databases for performance evaluation. Physikalisch-Technische Bundesanstalt (PTB)-ECG is a well-known database, while Chosun University (CU)-ECG is directly built for this study using the developed ECG sensor. The ResNet was 0.73%—0.27% higher than AlexNet or GoogLeNet on PTB-ECG—and the ResNet was 0.94%—0.12% higher than AlexNet or GoogLeNet on CU-ECG

Design of Ensemble Stacked Auto-Encoder for Classification of Horse Gaits with MEMS Inertial Sensor Technology

This paper discusses the classification of horse gaits for self-coaching using an ensemble stacked auto-encoder (ESAE) based on wavelet packets from the motion data of the horse rider. For this purpose, we built an ESAE and used probability values at the end of the softmax classifier. First, we initialized variables such as hidden nodes, weight, and max epoch using the options of the auto-encoder (AE). Second, the ESAE model is trained by feedforward, back propagation, and gradient calculation. Next, the parameters are updated by a gradient descent mechanism as new parameters. Finally, once the error value is satisfied, the algorithm terminates. The experiments were performed to classify horse gaits for self-coaching. We constructed the motion data of a horse rider. For the experiment, an expert horse rider of the national team wore a suit containing 16 inertial sensors based on a wireless network. To improve and quantify the performance of the classification, we used three methods (wavelet packet, statistical value, and ensemble model), as well as cross entropy with mean squared error. The experimental results revealed that the proposed method showed good performance when compared with conventional algorithms such as the support vector machine (SVM)

Body and Hand–Object ROI-Based Behavior Recognition Using Deep Learning

Behavior recognition has applications in automatic crime monitoring, automatic sports video analysis, and context awareness of so-called silver robots. In this study, we employ deep learning to recognize behavior based on body and hand–object interaction regions of interest (ROIs). We propose an ROI-based four-stream ensemble convolutional neural network (CNN). Behavior recognition data are mainly composed of images and skeletons. The first stream uses a pre-trained 2D-CNN by converting the 3D skeleton sequence into pose evolution images (PEIs). The second stream inputs the RGB video into the 3D-CNN to extract temporal and spatial features. The most important information in behavior recognition is identification of the person performing the action. Therefore, if the neural network is trained by removing ambient noise and placing the ROI on the person, feature analysis can be performed by focusing on the behavior itself rather than learning the entire region. Therefore, the third stream inputs the RGB video limited to the body-ROI into the 3D-CNN. The fourth stream inputs the RGB video limited to ROIs of hand–object interactions into the 3D-CNN. Finally, because better performance is expected by combining the information of the models trained with attention to these ROIs, better recognition will be possible through late fusion of the four stream scores. The Electronics and Telecommunications Research Institute (ETRI)-Activity3D dataset was used for the experiments. This dataset contains color images, images of skeletons, and depth images of 55 daily behaviors of 50 elderly and 50 young individuals. The experimental results showed that the proposed model improved recognition by at least 4.27% and up to 20.97% compared to other behavior recognition methods

Body and Hand–Object ROI-Based Behavior Recognition Using Deep Learning

Behavior recognition has applications in automatic crime monitoring, automatic sports video analysis, and context awareness of so-called silver robots. In this study, we employ deep learning to recognize behavior based on body and hand–object interaction regions of interest (ROIs). We propose an ROI-based four-stream ensemble convolutional neural network (CNN). Behavior recognition data are mainly composed of images and skeletons. The first stream uses a pre-trained 2D-CNN by converting the 3D skeleton sequence into pose evolution images (PEIs). The second stream inputs the RGB video into the 3D-CNN to extract temporal and spatial features. The most important information in behavior recognition is identification of the person performing the action. Therefore, if the neural network is trained by removing ambient noise and placing the ROI on the person, feature analysis can be performed by focusing on the behavior itself rather than learning the entire region. Therefore, the third stream inputs the RGB video limited to the body-ROI into the 3D-CNN. The fourth stream inputs the RGB video limited to ROIs of hand–object interactions into the 3D-CNN. Finally, because better performance is expected by combining the information of the models trained with attention to these ROIs, better recognition will be possible through late fusion of the four stream scores. The Electronics and Telecommunications Research Institute (ETRI)-Activity3D dataset was used for the experiments. This dataset contains color images, images of skeletons, and depth images of 55 daily behaviors of 50 elderly and 50 young individuals. The experimental results showed that the proposed model improved recognition by at least 4.27% and up to 20.97% compared to other behavior recognition methods

An EigenECG Network Approach Based on PCANet for Personal Identification from ECG Signal

We herein propose an EigenECG Network (EECGNet) based on the principal component analysis network (PCANet) for the personal identification of electrocardiogram (ECG) from human biosignal data. The EECGNet consists of three stages. In the first stage, ECG signals are preprocessed by normalization and spike removal. The R peak points in the preprocessed ECG signals are detected. Subsequently, ECG signals are transformed into two-dimensional images to use as the input to the EECGNet. Further, we perform patch-mean removal and PCA algorithm similar to the PCANet from the transformed two-dimensional images. The second stage is almost the same as the first stage, where the mean removal and PCA process are repeatedly performed in the cascaded network. In the final stage, the binary quantization, block sliding, and histogram computation are performed. Thus, this EECGNet performs well without the use of back-propagation to obtain features from the visual content. We constructed a Chosun University (CU)-ECG database from an ECG sensor implemented by ourselves. Further, we used the well-known MIT Beth Israel Hospital (BIH) ECG database. The experimental results clearly reveal the good performance and effectiveness of the proposed method compared with conventional algorithms such as PCA, auto-encoder (AE), extreme learning machine (ELM), and ensemble extreme learning machine (EELM)

Posture Recognition Using Ensemble Deep Models under Various Home Environments

This paper is concerned with posture recognition using ensemble convolutional neural networks (CNNs) in home environments. With the increasing number of elderly people living alone at home, posture recognition is very important for helping elderly people cope with sudden danger. Traditionally, to recognize posture, it was necessary to obtain the coordinates of the body points, depth, frame information of video, and so on. In conventional machine learning, there is a limitation in recognizing posture directly using only an image. However, with advancements in the latest deep learning, it is possible to achieve good performance in posture recognition using only an image. Thus, we performed experiments based on VGGNet, ResNet, DenseNet, InceptionResNet, and Xception as pre-trained CNNs using five types of preprocessing. On the basis of these deep learning methods, we finally present the ensemble deep model combined by majority and average methods. The experiments were performed by a posture database constructed at the Electronics and Telecommunications Research Institute (ETRI), Korea. This database consists of 51,000 images with 10 postures from 51 home environments. The experimental results reveal that the ensemble system by InceptionResNetV2s with five types of preprocessing shows good performance in comparison to other combination methods and the pre-trained CNN itself