Integrating Symbolic and Neural Processing in a Self-Organizing Architechture for Pattern Recognition and Prediction

British Petroleum (89A-1204); Defense Advanced Research Projects Agency (N00014-92-J-4015); National Science Foundation (IRI-90-00530); Office of Naval Research (N00014-91-J-4100); Air Force Office of Scientific Research (F49620-92-J-0225

Bayesian Modeling of Dynamic Scenes for Object Detection

Abstract—Accurate detection of moving objects is an important precursor to stable tracking or recognition. In this paper, we present an object detection scheme that has three innovations over existing approaches. First, the model of the intensities of image pixels as independent random variables is challenged and it is asserted that useful correlation exists in intensities of spatially proximal pixels. This correlation is exploited to sustain high levels of detection accuracy in the presence of dynamic backgrounds. By using a nonparametric density estimation method over a joint domain-range representation of image pixels, multimodal spatial uncertainties and complex dependencies between the domain (location) and range (color) are directly modeled. We propose a model of the background as a single probability density. Second, temporal persistence is proposed as a detection criterion. Unlike previous approaches to object detection which detect objects by building adaptive models of the background, the foreground is modeled to augment the detection of objects (without explicit tracking) since objects detected in the preceding frame contain substantial evidence for detection in the current frame. Finally, the background and foreground models are used competitively in a MAP-MRF decision framework, stressing spatial context as a condition of detecting interesting objects and the posterior function is maximized efficiently by finding the minimum cut of a capacitated graph. Experimental validation of the proposed method is performed and presented on a diverse set of dynamic scenes. Index Terms—Object detection, kernel density estimation, joint domain range, MAP-MRF estimation. æ

Quantitative Modelling of Climate Change Impact on Hydro-climatic Extremes

In recent decades, climate change has caused a more volatile climate leading to more extreme events such as severe rainstorms, heatwaves and floods which are likely to become more frequent. Aiming to reveal climate change impact on the hydroclimatic extremes in a quantitative sense, this thesis presents a comprehensive analysis from three main strands. The first strand focuses on developing a quantitative modelling framework to quantify the spatiotemporal variation of hydroclimatic extremes for the areas of concern. A spatial random sampling toolbox (SRS-GDA) is designed for randomizing the regions of interest (ROIs) with different geographic locations, sizes, shapes and orientations where the hydroclimatic extremes are parameterised by a nonstationary distribution model whose parameters are assumed to be time-varying. The parameters whose variation with respect to different spatial features of ROIs and climate change are finally quantified by various statistical models such as the generalised linear model. The framework is applied to quantify the spatiotemporal variation of rainfall extremes in Great Britain (GB) and Australia and is further used in a comparison study to quantify the bias between observed and climate projected extremes. Then the framework is extended to a multivariate framework to estimate the time-varying joint probability of more than one hydroclimatic variable in the perspective of non-stationarity. A case study for evaluating compound floods in Ho Chi Minh City, Vietnam is applied for demonstrating the application of the framework. The second strand aims to recognise, classify and track the development of hydroclimatic extremes (e.g., severe rainstorms) by developing a stable computer algorithm (i.e., the SPER toolbox). The SPER toolbox can detect the boundary of the event area, extract the spatial and physical features of the event, which can be used not only for pattern recognition but also to support AI-based training for labelling/cataloguing the pattern from the large-sized, grid-based, multi-scaled environmental datasets. Three illustrative cases are provided; and as the front-end of AI study, an example for training a convolution neural network is given for classifying the rainfall extremes in the last century of GB. The third strand turns to support decision making by building both theory-driven and data-driven decision-making models to simulate the decisions in the context of flood forecasting and early warning, using the data collected via laboratory-style experiments based on various information of probabilistic flood forecasts and consequences. The research work demonstrated in this thesis has been able to bridge the knowledge gaps in the related field and it also provides a precritical insight in managing future risks arising from hydroclimatic extremes, which makes perfect sense given the urgent situation of climate change and the related challenges our societies are facing

Review of analytical instruments for EEG analysis

Since it was first used in 1926, EEG has been one of the most useful instruments of neuroscience. In order to start using EEG data we need not only EEG apparatus, but also some analytical tools and skills to understand what our data mean. This article describes several classical analytical tools and also new one which appeared only several years ago. We hope it will be useful for those researchers who have only started working in the field of cognitive EEG

시계열 데이터 패턴 분석을 위한 종단 심층 학습망 설계 방법론

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 컴퓨터공학부, 2019. 2. 장병탁.Pattern recognition within time series data became an important avenue of research in artificial intelligence following the paradigm shift of the fourth industrial revolution. A number of studies related to this have been conducted over the past few years, and research using deep learning techniques are becoming increasingly popular. Due to the nonstationary, nonlinear and noisy nature of time series data, it is essential to design an appropriate model to extract its significant features for pattern recognition. This dissertation not only discusses the study of pattern recognition using various hand-crafted feature engineering techniques using physiological time series signals, but also suggests an end-to-end deep learning design methodology without any feature engineering. Time series signal can be classified into signals having periodic and non-periodic characteristics in the time domain. This thesis proposes two end-to-end deep learning design methodologies for pattern recognition of periodic and non-periodic signals. The first proposed deep learning design methodology is Deep ECGNet. Deep ECGNet offers a design scheme for an end-to-end deep learning model using periodic characteristics of Electrocardiogram (ECG) signals. ECG, recorded from the electrophysiologic patterns of heart muscle during heartbeat, could be a promising candidate to provide a biomarker to estimate event-based stress level. Conventionally, the beat-to-beat alternations, heart rate variability (HRV), from ECG have been utilized to monitor the mental stress status as well as the mortality of cardiac patients. These HRV parameters have the disadvantage of having a 5-minute measurement period. In this thesis, human's stress states were estimated without special hand-crafted feature engineering using only 10-second interval data with the deep learning model. The design methodology of this model incorporates the periodic characteristics of the ECG signal into the model. The main parameters of 1D CNNs and RNNs reflecting the periodic characteristics of ECG were updated corresponding to the stress states. The experimental results proved that the proposed method yielded better performance than those of the existing HRV parameter extraction methods and spectrogram methods. The second proposed methodology is an automatic end-to-end deep learning design methodology using Bayesian optimization for non-periodic signals. Electroencephalogram (EEG) is elicited from the central nervous system (CNS) to yield genuine emotional states, even at the unconscious level. Due to the low signal-to-noise ratio (SNR) of EEG signals, spectral analysis in frequency domain has been conventionally applied to EEG studies. As a general methodology, EEG signals are filtered into several frequency bands using Fourier or wavelet analyses and these band features are then fed into a classifier. This thesis proposes an end-to-end deep learning automatic design method using optimization techniques without this basic feature engineering. Bayesian optimization is a popular optimization technique for machine learning to optimize model hyperparameters. It is often used in optimization problems to evaluate expensive black box functions. In this thesis, we propose a method to perform whole model hyperparameters and structural optimization by using 1D CNNs and RNNs as basic deep learning models and Bayesian optimization. In this way, this thesis proposes the Deep EEGNet model as a method to discriminate human emotional states from EEG signals. Experimental results proved that the proposed method showed better performance than that of conventional method based on the conventional band power feature method. In conclusion, this thesis has proposed several methodologies for time series pattern recognition problems from the feature engineering-based conventional methods to the end-to-end deep learning design methodologies with only raw time series signals. Experimental results showed that the proposed methodologies can be effectively applied to pattern recognition problems using time series data.시계열 데이터의 패턴 인식 문제는 4차 산업 혁명의 패러다임 전환과 함께 매우 중요한 인공 지능의 한 분야가 되었다. 이에 따라, 지난 몇 년간 이와 관련된 많은 연구들이 이루어져 왔으며, 최근에는 심층 학습망 (deep learning networks) 모델을 이용한 연구들이 주를 이루어 왔다. 시계열 데이터는 비정상, 비선형 그리고 잡음 (nonstationary, nonlinear and noisy) 특성으로 인하여 시계열 데이터의 패턴 인식 수행을 위해선, 데이터의 주요한 특징점을 추출하기 위한 최적화된 모델의 설계가 필수적이다. 본 논문은 대표적인 시계열 데이터인 생체 신호를 사용하여 여러 특징 벡터 추출 방법 (hand-crafted feature engineering methods)을 이용한 패턴 인식 기법에 대하여 논할 뿐만 아니라, 궁극적으로는 특징 벡터 추출 과정이 없는 종단 심층 학습망 설계 방법론에 대한 연구 내용을 담고 있다. 시계열 신호는 시간 축 상에서 크게 주기적 신호와 비주기적 신호로 구분할 수 있는데, 본 연구는 이러한 두 유형의 신호들에 대한 패턴 인식을 위해 두 가지 종단 심층 학습망에 대한 설계 방법론을 제안한다. 첫 번째 제안된 방법론을 이용해 설계된 모델은 신호의 주기적 특성을 이용한 Deep ECGNet이다. 심장 근육의 전기 생리학적 패턴으로부터 기록된 심전도 (Electrocardiogram, ECG)는 이벤트 기반 스트레스 수준을 추정하기 위한 척도 (bio marker)를 제공하는 유효한 데이터가 될 수 있다. 전통적으로 심전도의 심박수 변동성 (Herat Rate Variability, HRV) 매개변수 (parameter)는 심장 질환 환자의 정신적 스트레스 상태 및 사망률을 모니터링하는 데 사용되었다. 하지만, 표준 심박수 변동성 매개 변수는 측정 주기가 5분 이상으로, 측정 시간이 길다는 단점이 있다. 본 논문에서는 심층 학습망 모델을 이용하여 10초 간격의 ECG 데이터만을 이용하여, 추가적인 특징 벡터의 추출 과정 없이 인간의 스트레스 상태를 인식할 수 있음을 보인다. 제안된 설계 기법은 ECG 신호의 주기적 특성을 모델에 반영하였는데, ECG의 은닉 특징 추출기로 사용된 1D CNNs 및 RNNs 모델의 주요 매개 변수에 주기적 특성을 반영함으로써, 한 주기 신호의 스트레스 상태에 따른 주요 특징점을 종단 학습망 내부적으로 추출할 수 있음을 보였다. 실험 결과 제안된 방법이 기존 심박수 변동성 매개변수와 spectrogram 추출 기법 기반의 패턴 인식 방법보다 좋은 성능을 나타내고 있음을 확인할 수 있었다. 두 번째 제안된 방법론은 비 주기적이며 비정상, 비선형 그리고 잡음 특성을 지닌 신호의 패턴인식을 위한 최적 종단 심층 학습망 자동 설계 방법론이다. 뇌파 신호 (Electroencephalogram, EEG)는 중추 신경계 (CNS)에서 발생되어 무의식 상태에서도 본연의 감정 상태를 나타내는데, EEG 신호의 낮은 신호 대 잡음비 (SNR)로 인해 뇌파를 이용한 감정 상태 판정을 위해서 주로 주파수 영역의 스펙트럼 분석이 뇌파 연구에 적용되어 왔다. 통상적으로 뇌파 신호는 푸리에 (Fourier) 또는 웨이블렛 (wavelet) 분석을 사용하여 여러 주파수 대역으로 필터링 된다. 이렇게 추출된 주파수 특징 벡터는 보통 얕은 학습 분류기 (shallow machine learning classifier)의 입력으로 사용되어 패턴 인식을 수행하게 된다. 본 논문에서는 이러한 기본적인 특징 벡터 추출 과정이 없는 베이지안 최적화 (Bayesian optimization) 기법을 이용한 종단 심층 학습망 자동 설계 기법을 제안한다. 베이지안 최적화 기법은 초 매개변수 (hyperparamters)를 최적화하기 위한 기계 학습 분야의 대표적인 최적화 기법인데, 최적화 과정에서 평가 시간이 많이 소요되는 목적 함수 (expensive black box function)를 갖고 있는 최적화 문제에 적합하다. 이러한 베이지안 최적화를 이용하여 기본적인 학습 모델인 1D CNNs 및 RNNs의 전체 모델의 초 매개변수 및 구조적 최적화를 수행하는 방법을 제안하였으며, 제안된 방법론을 바탕으로 Deep EEGNet이라는 인간의 감정상태를 판별할 수 있는 모델을 제안하였다. 여러 실험을 통해 제안된 모델이 기존의 주파수 특징 벡터 (band power feature) 추출 기법 기반의 전통적인 감정 패턴 인식 방법보다 좋은 성능을 나타내고 있음을 확인할 수 있었다. 결론적으로 본 논문은 시계열 데이터를 이용한 패턴 인식문제를 여러 특징 벡터 추출 기법 기반의 전통적인 방법을 통해 설계하는 방법부터, 추가적인 특징 벡터 추출 과정 없이 원본 데이터만을 이용하여 종단 심층 학습망을 설계하는 방법까지 제안하였다. 또한, 다양한 실험을 통해 제안된 방법론이 시계열 신호 데이터를 이용한 패턴 인식 문제에 효과적으로 적용될 수 있음을 보였다.Chapter 1 Introduction 1 1.1 Pattern Recognition in Time Series 1 1.2 Major Problems in Conventional Approaches 7 1.3 The Proposed Approach and its Contribution 8 1.4 Thesis Organization 10 Chapter 2 Related Works 12 2.1 Pattern Recognition in Time Series using Conventional Methods 12 2.1.1 Time Domain Features 12 2.1.2 Frequency Domain Features 14 2.1.3 Signal Processing based on Multi-variate Empirical Mode Decomposition (MEMD) 15 2.1.4 Statistical Time Series Model (ARIMA) 18 2.2 Fundamental Deep Learning Algorithms 20 2.2.1 Convolutional Neural Networks (CNNs) 20 2.2.2 Recurrent Neural Networks (RNNs) 22 2.3 Hyper Parameters and Structural Optimization Techniques 24 2.3.1 Grid and Random Search Algorithms 24 2.3.2 Bayesian Optimization 25 2.3.3 Neural Architecture Search 28 2.4 Research Trends related to Time Series Data 29 2.4.1 Generative Model of Raw Audio Waveform 30 Chapter 3 Preliminary Researches: Patten Recognition in Time Series using Various Feature Extraction Methods 31 3.1 Conventional Methods using Time and Frequency Features: Motor Imagery Brain Response Classification 31 3.1.1 Introduction 31 3.1.2 Methods 32 3.1.3 Ensemble Classification Method (Stacking & AdaBoost) 32 3.1.4 Sensitivity Analysis 33 3.1.5 Classification Results 36 3.2 Statistical Feature Extraction Methods: ARIMA Model Based Feature Extraction Methodology 38 3.2.1 Introduction 38 3.2.2 ARIMA Model 38 3.2.3 Signal Processing 39 3.2.4 ARIMA Model Conformance Test 40 3.2.5 Experimental Results 40 3.2.6 Summary 43 3.3 Application on Specific Time Series Data: Human Stress States Recognition using Ultra-Short-Term ECG Spectral Feature 44 3.3.1 Introduction 44 3.3.2 Experiments 45 3.3.3 Classification Methods 49 3.3.4 Experimental Results 49 3.3.5 Summary 56 Chapter 4 Master Framework for Pattern Recognition in Time Series 57 4.1 The Concept of the Proposed Framework for Pattern Recognition in Time Series 57 4.1.1 Optimal Basic Deep Learning Models for the Proposed Framework 57 4.2 Two Categories for Pattern Recognition in Time Series Data 59 4.2.1 The Proposed Deep Learning Framework for Periodic Time Series Signals 59 4.2.2 The Proposed Deep Learning Framework for Non-periodic Time Series Signals 61 4.3 Expanded Models of the Proposed Master Framework for Pattern Recogntion in Time Series 63 Chapter 5 Deep Learning Model Design Methodology for Periodic Signals using Prior Knowledge: Deep ECGNet 65 5.1 Introduction 65 5.2 Materials and Methods 67 5.2.1 Subjects and Data Acquisition 67 5.2.2 Conventional ECG Analysis Methods 72 5.2.3 The Initial Setup of the Deep Learning Architecture 75 5.2.4 The Deep ECGNet 78 5.3 Experimental Results 83 5.4 Summary 98 Chapter 6 Deep Learning Model Design Methodology for Non-periodic Time Series Signals using Optimization Techniques: Deep EEGNet 100 6.1 Introduction 100 6.2 Materials and Methods 104 6.2.1 Subjects and Data Acquisition 104 6.2.2 Conventional EEG Analysis Methods 106 6.2.3 Basic Deep Learning Units and Optimization Technique 108 6.2.4 Optimization for Deep EEGNet 109 6.2.5 Deep EEGNet Architectures using the EEG Channel Grouping Scheme 111 6.3 Experimental Results 113 6.4 Summary 124 Chapter 7 Concluding Remarks 126 7.1 Summary of Thesis and Contributions 126 7.2 Limitations of the Proposed Methods 128 7.3 Suggestions for Future Works 129 Bibliography 131 초 록 139Docto

A control algorithm for autonomous optimization of extracellular recordings

This paper develops a control algorithm that can autonomously position an electrode so as to find and then maintain an optimal extracellular recording position. The algorithm was developed and tested in a two-neuron computational model representative of the cells found in cerebral cortex. The algorithm is based on a stochastic optimization of a suitably defined signal quality metric and is shown capable of finding the optimal recording position along representative sampling directions, as well as maintaining the optimal signal quality in the face of modeled tissue movements. The application of the algorithm to acute neurophysiological recording experiments and its potential implications to chronic recording electrode arrays are discussed