A Magnetorheological Damper with Embedded Piezoelectric Force Sensor: Experiment and Modeling

This chapter describes configuration, fabrication, calibration and performance tests of the devised self-sensing MR damper firstly. Then, a black-box identification approach for modeling the forward and inverse dynamics of the self-sensing MR damper is presented, which is developed with the synthesis of NARX model and neural network within a Bayesian inference framework to have the ability of enhancing generalization.Department of Civil and Environmental Engineerin

Enhancing VAEs for Collaborative Filtering: Flexible Priors & Gating Mechanisms

학위논문(석사)--서울대학교 대학원 :융합과학기술대학원 융합과학부(디지털정보융합전공),2019. 8. 서봉원.Since Matrix Factorization based linear models have been dominant in the Collaborative Filtering context for a long time in the past, Neural Network based CF Models for recommendation have started to gain attention recently. One branch of research is based on using deep generative models to model user preferences and Variational Autoencoders where shown to give state-of-the-art results. However, there are some potentially problematic characteristics of the current Variational Autoencoder for CF. The first is the too simplistic prior VAEs incorporate for learning the latent representations of user preference, which may be restricting the model from learning more expressive and richer latent variables that could boost recommendation performance. The other is the models inability to learn deeper representations with more than one hidden layer. Our goal is to incorporate appropriate techniques in order to mitigate the aforementioned problems of Variational Autoencoder CF and further improve the recommendation performance of VAE based Collaborative Fil-tering. We bring the VampPrior, which successfully made improvements for image generation to tackle the restrictive prior problem. We also adopt Gat-ed Linear Units (GLUs) which were used in stacked convolutions for lan-guage modeling to control information flow in the easily deepening auto-encoder framework. We show that such simple priors (in original VAEs) may be too restric-tive to fully model user preferences and setting a more flexible prior gives significant gains. We also show that VAMP priors coupled with gating mechanisms outperform SOTA results including the Variational Autoencoder for Collaborative Filtering by meaningful margins on 4 benchmark datasets (MovieLens, Netflix, Pinterest, Melon).최근 뉴럴넷 기반 협업필터링 추천알고리즘이 주목을 받고 있다. 그 중 한 갈래의 연구는 깊은 생성모형 (Deep Generative Model)을 이용해 사용자들의 선호를 모델링하는 방법이다. 이중 Variational Autoencoder를 (VAE) 이용한 방법이 최근 state-of-the-art (SOTA) 성능을 보여주었다. 그러나 VAE를 이용한 협업필터링 알고리즘은 현재 몇 가지의 잠재적인 문제점들을 지니고 있다. 첫 번째는 사용자 선호를 압축하는 잠재변수를 학습하는 과정에서 매우 단순한 사전분포를 사용한다는 것이다. 또 다른 문제점은 모델이 현재 여러 단을 이용한 깊은 인코더와 디코더를 사용하지 못하고 있다는 것이다. 본 연구는 최신기술들을 활용하여 앞선 문제점들을 해결하고 VAE를 이용한 협업필터링 알고리즘의 추천성능을 더욱 높이는 것이 목표이다. 본 연구는 협업필터링 문제에 더 복잡한 사전분포 (Flexible Prior)를 적용한 첫 연구로서, 기존의 단순한 사전분포가 모델의 표현력을 제한할 수 있으며 더 복잡한 사전분포를 정의함으로써 모델의 성능을 더욱 높일 수 있음을 보였다. 이를 위해 이미지 생성 문제에서 좋은 결과를 보인 VampPrior를 이용해 실험을 진행하였다. 또한 VampPrior를 Gating Mechanisim과 함께 사용하였을 때 기존 SOTA를 넘어서는 성능을 보임을 추천알고리즘에서 사용되는 대표적인 데이터셋들을 통해 보여준다.1 INTRODUCTION 1 1.1 Background and Motivation 1 1.2 Research Goal 3 1.3 Enhancing VAEs for Collaborative Filtering 3 1.4 Experiments 5 1.5 Contributions 5 2 RELATED WORK 7 2.1 Collaborative Filtering 7 2.1.1 Traditional methods & Matrix-Factorization based CF 8 2.1.2 Autoencoders for CF 12 2.2 Deep Generative Models (VAE) 17 2.2.1 Variational Bayes 18 2.2.2 Variational Autoencoder 18 2.3 Variational Autoencoder for Collaborative Filtering 20 2.3.1 VAE for CF 21 2.4 Recent research in Computer Vision & Deep Learning 24 2.4.1 VampPrior 24 2.4.2 Gated CNN 25 3 METHOD 28 3.1 Flexible Prior 29 3.1.1 Motivation 29 3.1.2 VampPrior 30 3.1.3 Hierarchical Stochastic Units 31 3.2 Gating Mechanism 32 3.2.1 Motivation 32 3.2.2 Gated Linear Units 34 4 EXPERIMENT 35 4.1 Setup 35 4.1.1 Baseline Models 35 4.1.2 Proposed Models 37 4.1.3 Strong Generalization 37 4.1.4 Evaluation Metrics 38 4.2 Datasets 38 4.3 Configurations 39 4.4 Results 40 4.4.1 Model Performance 40 4.4.5 Further Analysis on the Effect of Gating 44 5 CONCLUSION 45 Bibliography 47 국문초록 51Maste

Learning with Limited Labeled Data in Biomedical Domain by Disentanglement and Semi-Supervised Learning

In this dissertation, we are interested in improving the generalization of deep neural networks for biomedical data (e.g., electrocardiogram signal, x-ray images, etc). Although deep neural networks have attained state-of-the-art performance and, thus, deployment across a variety of domains, similar performance in the clinical setting remains challenging due to its ineptness to generalize across unseen data (e.g., new patient cohort). We address this challenge of generalization in the deep neural network from two perspectives: 1) learning disentangled representations from the deep network, and 2) developing efficient semi-supervised learning (SSL) algorithms using the deep network. In the former, we are interested in designing specific architectures and objective functions to learn representations, where variations in the data are well separated, i.e., disentangled. In the latter, we are interested in designing regularizers that encourage the underlying neural function\u27s behavior toward a common inductive bias to avoid over-fitting the function to small labeled data. Our end goal is to improve the generalization of the deep network for the diagnostic model in both of these approaches. In disentangled representations, this translates to appropriately learning latent representations from the data, capturing the observed input\u27s underlying explanatory factors in an independent and interpretable way. With data\u27s expository factors well separated, such disentangled latent space can then be useful for a large variety of tasks and domains within data distribution even with a small amount of labeled data, thus improving generalization. In developing efficient semi-supervised algorithms, this translates to utilizing a large volume of the unlabelled dataset to assist the learning from the limited labeled dataset, commonly encountered situation in the biomedical domain. By drawing ideas from different areas within deep learning like representation learning (e.g., autoencoder), variational inference (e.g., variational autoencoder), Bayesian nonparametric (e.g., beta-Bernoulli process), learning theory (e.g., analytical learning theory), function smoothing (Lipschitz Smoothness), etc., we propose several leaning algorithms to improve generalization in the associated task. We test our algorithms on real-world clinical data and show that our approach yields significant improvement over existing methods. Moreover, we demonstrate the efficacy of the proposed models in the benchmark data and simulated data to understand different aspects of the proposed learning methods. We conclude by identifying some of the limitations of the proposed methods, areas of further improvement, and broader future directions for the successful adoption of AI models in the clinical environment

A neural network approach to the modeling of blast furnace

Thesis (M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1999.Includes bibliographical references (leaves 65-69).by Angela X. Ge.M.Eng