산업에서 딥러닝을 이용한 시계열 데이터 분석

학위논문 (박사)-- 서울대학교 대학원 : 자연과학대학 협동과정 계산과학전공, 2019. 2. 강명주.딥러닝은 최근 몇 년간 산업 수학에 있어서 가장 강력하고 중요시 여겨지는 방법이다. 우리는 산업 수학의 시계열 데이터에 있어서 딥러닝 모델을 분석 및 예측 등에 정의하였다. 첫 번째로, 이상 감지를 위한 새로운 딥러닝 모델을 개발하였으며 이는 다양한 길이 뿐만 아니라 노이즈, 시간 차가 있는 데이터에서도 엔지니어에게 필요한 시계열 데이터 분석을 할 수 있었다. 두 번째로, 금융 시장의 트렌드를 예측하기 위한 다양한 딥러닝 모델을 개발 및 시험해보았으며 이 중 가중치 어텐션 네트워크의 경우 높은 예측 정확도 뿐만 아니라 시각화를 통해 직관적으로 모델을 이해 및 예측한 이유를 분석할 수 있었다.Deep learning, also called as artificial neural networks, is one of the most important and powerful subjects in industrial in recent years. Deep learning starts to show a great performance from image classification and in these days it have been applied to fields including computer vision, natural language process, speech recognition and etc. The performance is better than not only previous machine learning techniques, but also human experts in some cases. For an area with time series data, recurrent neural networks is widely used algorithm of deep learning. The aim of this theseis is to apply deep learning, especially with recurrent neural networks, for an industrial such as anomaly detection and trend prediction in financial market, with time series data . Its main contributions are (1) a new model for anomaly detection in time series data even for various length inputs, (2) various neural architectures for prediction in finance, and (3) attention networks and model analysis with attention vectors. Each experimental results of applications show better performances than previous machine learning techniques.1 Introduction 2 Deep Learning Background 2.1 Neural Networks 2.2 Various Activation Functions 2.3 Error Backpropagation 2.4 Regularization 2.4.1 Dropout 2.4.2 Batch Normalization 3 Deep Learning Models 3.1 Multi Layer Perceptron 3.2 Convolutional Neural Networks 3.3 Recurrent Neural Networks 3.4 Long Short Term Memory 3.5 Attention Networks 4 Anomaly Detection 4.1 Related Works of Anomaly Detection 4.1.1 Anomaly detection 4.1.2 t-SNE 4.1.3 Clustering 4.2 Deep Correlation Mapping 4.2.1 LSTM 4.2.2 t-SNE 4.2.3 Full Model Architecture 4.2.4 Anomaly detection using Deep Correlation Mapping 4.3 Experimental Results 4.3.1 Correlation 4.3.2 Anomaly detection using DeepCorr 4.4 Conclusion 5 Trend Prediction 5.1 Related works of Trend Prediction 5.2 Trend Prediction with Deep Learning Models 5.2.1 Dataset 5.2.2 MLP 5.2.3 1D-CNN 5.2.4 LSTM 5.2.5 Attention Networks 5.2.6 Weighted Attention Networks 5.3 Experimental Results 5.3.1 Best Lookback Days 5.3.2 Results of Various Deep Learning Models 5.3.3 Visualization Attention Vectors 5.4 Conclusion 6 Conclusion and Future Works Abstract (in Korean) Acknowledgement (in Korean)Docto

Deep Learning: A Philosophical Introduction

Deep learning is currently the most prominent and widely successful method in artificial intelligence. Despite having played an active role in earlier artificial intelligence and neural network research, philosophers have been largely silent on this technology so far. This is remarkable, given that deep learning neural networks have blown past predicted upper limits on artificial intelligence performance—recognizing complex objects in natural photographs, and defeating world champions in strategy games as complex as Go and chess—yet there remains no universally-accepted explanation as to why they work so well. This article provides an introduction to these networks, as well as an opinionated guidebook on the philosophical significance of their structure and achievements. It argues that deep learning neural networks differ importantly in their structure and mathematical properties from the shallower neural networks that were the subject of so much philosophical reflection in the 1980s and 1990s. The article then explores several different explanations for their success, and ends by proposing ten areas of research that would benefit from future engagement by philosophers of mind, epistemology, science, perception, law, and ethics

Banknote Authentication and Medical Image Diagnosis Using Feature Descriptors and Deep Learning Methods

Banknote recognition and medical image analysis have been the foci of image processing and pattern recognition research. As counterfeiters have taken advantage of the innovation in print media technologies for reproducing fake monies, hence the need to design systems which can reassure and protect citizens of the authenticity of banknotes in circulation. Similarly, many physicians must interpret medical images. But image analysis by humans is susceptible to error due to wide variations across interpreters, lethargy, and human subjectivity. Computer-aided diagnosis is vital to improvements in medical analysis, as they facilitate the identification of findings that need treatment and assist the expert’s workflow. Thus, this thesis is organized around three such problems related to Banknote Authentication and Medical Image Diagnosis. In our first research problem, we proposed a new banknote recognition approach that classifies the principal components of extracted HOG features. We further experimented on computing HOG descriptors from cells created from image patch vertices of SURF points and designed a feature reduction approach based on a high correlation and low variance filter. In our second research problem, we developed a mobile app for banknote identification and counterfeit detection using the Unity 3D software and evaluated its performance based on a Cascaded Ensemble approach. The algorithm was then extended to a client-server architecture using SIFT and SURF features reduced by Bag of Words and high correlation-based HOG vectors. In our third research problem, experiments were conducted on a pre-trained mobile app for medical image diagnosis using three convolutional layers with an Ensemble Classifier comprising PCA and bagging of five base learners. Also, we implemented a Bidirectional Generative Adversarial Network to mitigate the effect of the Binary Cross Entropy loss based on a Deep Convolutional Generative Adversarial Network as the generator and encoder with Capsule Network as the discriminator while experimenting on images with random composition and translation inferences. Lastly, we proposed a variant of the Single Image Super-resolution for medical analysis by redesigning the Super Resolution Generative Adversarial Network to increase the Peak Signal to Noise Ratio during image reconstruction by incorporating a loss function based on the mean square error of pixel space and Super Resolution Convolutional Neural Network layers

Noise Reduction in EEG Signals using Convolutional Autoencoding Techniques

The presence of noise in electroencephalography (EEG) signals can significantly reduce the accuracy of the analysis of the signal. This study assesses to what extent stacked autoencoders designed using one-dimensional convolutional neural network layers can reduce noise in EEG signals. The EEG signals, obtained from 81 people, were processed by a two-layer one-dimensional convolutional autoencoder (CAE), whom performed 3 independent button pressing tasks. The signal-to-noise ratios (SNRs) of the signals before and after processing were calculated and the distributions of the SNRs were compared. The performance of the model was compared to noise reduction performance of Principal Component Analysis, with 95% explained variance, by comparing the Harrell-Davis decile differences between the SNR distributions of both methods and the raw signal SNR distribution for each task. It was found that the CAE outperformed PCA for the full dataset across all three tasks, however the CAE did not outperform PCA for the person specific datasets in any of the three tasks. The results indicate that CAEs can perform better than PCA for noise reduction in EEG signals, but performance of the model may be training size dependent

The selection and evaluation of a sensory technology for interaction in a warehouse environment

In recent years, Human-Computer Interaction (HCI) has become a significant part of modern life as it has improved human performance in the completion of daily tasks in using computerised systems. The increase in the variety of bio-sensing and wearable technologies on the market has propelled designers towards designing more efficient, effective and fully natural User-Interfaces (UI), such as the Brain-Computer Interface (BCI) and the Muscle-Computer Interface (MCI). BCI and MCI have been used for various purposes, such as controlling wheelchairs, piloting drones, providing alphanumeric inputs into a system and improving sports performance. Various challenges are experienced by workers in a warehouse environment. Because they often have to carry objects (referred to as hands-full) it is difficult to interact with traditional devices. Noise undeniably exists in some industrial environments and it is known as a major factor that causes communication problems. This has reduced the popularity of using verbal interfaces with computer applications, such as Warehouse Management Systems. Another factor that effects the performance of workers are action slips caused by a lack of concentration during, for example, routine picking activities. This can have a negative impact on job performance and allow a worker to incorrectly execute a task in a warehouse environment. This research project investigated the current challenges workers experience in a warehouse environment and the technologies utilised in this environment. The latest automation and identification systems and technologies are identified and discussed, specifically the technologies which have addressed known problems. Sensory technologies were identified that enable interaction between a human and a computerised warehouse environment. Biological and natural behaviours of humans which are applicable in the interaction with a computerised environment were described and discussed. The interactive behaviours included the visionary, auditory, speech production and physiological movement where other natural human behaviours such paying attention, action slips and the action of counting items were investigated. A number of modern sensory technologies, devices and techniques for HCI were identified with the aim of selecting and evaluating an appropriate sensory technology for MCI. iii MCI technologies enable a computer system to recognise hand and other gestures of a user, creating means of direct interaction between a user and a computer as they are able to detect specific features extracted from a specific biological or physiological activity. Thereafter, Machine Learning (ML) is applied in order to train a computer system to detect these features and convert them to a computer interface. An application of biomedical signals (bio-signals) in HCI using a MYO Armband for MCI is presented. An MCI prototype (MCIp) was developed and implemented to allow a user to provide input to an HCI, in a hands-free and hands-full situation. The MCIp was designed and developed to recognise the hand-finger gestures of a person when both hands are free or when holding an object, such a cardboard box. The MCIp applies an Artificial Neural Network (ANN) to classify features extracted from the surface Electromyography signals acquired by the MYO Armband around the forearm muscle. The MCIp provided the results of data classification for gesture recognition to an accuracy level of 34.87% with a hands-free situation. This was done by employing the ANN. The MCIp, furthermore, enabled users to provide numeric inputs to the MCIp system hands-full with an accuracy of 59.7% after a training session for each gesture of only 10 seconds. The results were obtained using eight participants. Similar experimentation with the MYO Armband has not been found to be reported in any literature at submission of this document. Based on this novel experimentation, the main contribution of this research study is a suggestion that the application of a MYO Armband, as a commercially available muscle-sensing device on the market, has the potential as an MCI to recognise the finger gestures hands-free and hands-full. An accurate MCI can increase the efficiency and effectiveness of an HCI tool when it is applied to different applications in a warehouse where noise and hands-full activities pose a challenge. Future work to improve its accuracy is proposed

Advanced Information Systems and Technologies

This book comprises the proceedings of the VI International Scientific Conference “Advanced Information Systems and Technologies, AIST-2018”. The proceeding papers cover issues related to system analysis and modeling, project management, information system engineering, intelligent data processing, computer networking and telecomunications, modern methods and information technologies of sustainable development. They will be useful for students, graduate students, researchers who interested in computer science

The Perception and Evaluation of Visual Beauty

What are the perceptual and cognitive processes that underlie our experiences of beauty? In this dissertation, I describe a series of experiments where we used functional magnetic resonance imaging (fMRI) and behavioral methods to explore the mechanisms of perception, reward representation, and decision-making during evaluations of face and place beauty. In our first study, we used fMRI to ask whether evaluative signals in frontal cortex contain category-specific information or whether these signals are encoded as a common currency across reward types. By comparing neural activity correlated with subjective ratings of face and place beauty, we showed overlapping activity in dorsal ventromedial prefrontal cortex (vmPFC), consistence with the common currency hypothesis. At the same time, our results revealed category-specific patterns of activity in ventral vmPFC and in lateral orbitofrontal cortex (latOFC), suggesting at least a partial distinction in the frontal networks recruited during the processing of different types of rewards. In a follow-up study, we used fMRI to further examine face-responsive patches of activity in latOFC by measuring response in these patches while subjects evaluated but did explicitly rate face beauty. Our results demonstrated a similar pattern of response to that observed during explicit ratings, suggesting that reward-related activity in this region is not dependent on a decision-making task. Lastly, in a series of behavioral studies, we developed a novel experimental design to measure the influence of recent trial history on current judgments of face attractiveness. We found that attractiveness judgments are simultaneously contrasted away from the attractiveness of the previous face but assimilated towards the previous numerical rating given. Our results also suggested that these influences are not specific to attractiveness judgments but may be linked to more general properties of perception and decision-making. Collectively, this work furthers our understanding of the neural mechanisms underlying evaluations of face and place beauty, and illuminates some of the specific contextual influences on these evaluations

Interpretable Tsetlin Machine For Explaining Board Games With Complex Game States

Stefan Dorra's For Sale is both a turn-based and simultaneous action zero-sum game where the objective is to become as rich as possible. The first phase of the game is a sequence of turn-based English auctions that bids for properties selected at random. The game itself is complex having a mix of multiple players, hidden information, and stochastic elements. Although auctions themselves have been thoroughly studied in literature this particular setup remains an open problem. In this thesis, we investigate the usage of the interpretable Coalesced Tsetlin Machine (CoTM) for solving these types of auction games providing both excellent play and an understanding of how to play. To this end, we first develop a self-playing reinforcement learning algorithm that achieves near optimal play. Secondly, based on this algorithm we construct a dataset with examples of optimal play. Thirdly, using CoTM we investigate various ways of understanding why particular moves are made. The CoTM is also shown to outperform popular methods such as decision trees, neural networks, and k-nearest neighbours. On average the CoTM accuracy is 84.55\% significantly outperforming the other competitors. We believe the resulting interpretability establishes that CoTM can be used for the interpretation of games that have a more complex game state than Hex and Go