12,506 research outputs found
Lung Segmentation from Chest X-rays using Variational Data Imputation
Pulmonary opacification is the inflammation in the lungs caused by many
respiratory ailments, including the novel corona virus disease 2019 (COVID-19).
Chest X-rays (CXRs) with such opacifications render regions of lungs
imperceptible, making it difficult to perform automated image analysis on them.
In this work, we focus on segmenting lungs from such abnormal CXRs as part of a
pipeline aimed at automated risk scoring of COVID-19 from CXRs. We treat the
high opacity regions as missing data and present a modified CNN-based image
segmentation network that utilizes a deep generative model for data imputation.
We train this model on normal CXRs with extensive data augmentation and
demonstrate the usefulness of this model to extend to cases with extreme
abnormalities.Comment: Accepted to be presented at the first Workshop on the Art of Learning
with Missing Values (Artemiss) hosted by the 37th International Conference on
Machine Learning (ICML). Source code, training data and the trained models
are available here: https://github.com/raghavian/lungVAE
λ₯λ¬λ κΈ°λ° μμ± λͺ¨λΈμ μ΄μ©ν μμ°μ΄μ²λ¦¬ λ°μ΄ν° μ¦κ° κΈ°λ²
νμλ
Όλ¬Έ(λ°μ¬)--μμΈλνκ΅ λνμ :곡과λν μ»΄ν¨ν°κ³΅νλΆ,2020. 2. μ΄μꡬ.Recent advances in generation capability of deep learning models have spurred interest in utilizing deep generative models for unsupervised generative data augmentation (GDA). Generative data augmentation aims to improve the performance of a downstream machine learning model by augmenting the original dataset with samples generated from a deep latent variable model. This data augmentation approach is attractive to the natural language processing community, because (1) there is a shortage of text augmentation techniques that require little supervision and (2) resource scarcity being prevalent. In this dissertation, we explore the feasibility of exploiting deep latent variable models for data augmentation on three NLP tasks: sentence classification, spoken language understanding (SLU) and dialogue state tracking (DST), represent NLP tasks of various complexities and properties -- SLU requires multi-task learning of text classification and sequence tagging, while DST requires the understanding of hierarchical and recurrent data structures. For each of the three tasks, we propose a task-specific latent variable model based on conditional, hierarchical and sequential variational autoencoders (VAE) for multi-modal joint modeling of linguistic features and the relevant annotations. We conduct extensive experiments to statistically justify our hypothesis that deep generative data augmentation is beneficial for all subject tasks. Our experiments show that deep generative data augmentation is effective for the select tasks, supporting the idea that the technique can potentially be utilized for other range of NLP tasks. Ablation and qualitative studies reveal deeper insight into the underlying mechanisms of generative data augmentation. As a secondary contribution, we also shed light onto the recurring posterior collapse phenomenon in autoregressive VAEs and, subsequently, propose novel techniques to reduce the model risk, which is crucial for proper training of complex VAE models, enabling them to synthesize better samples for data augmentation. In summary, this work intends to demonstrate and analyze the effectiveness of unsupervised generative data augmentation in NLP. Ultimately, our approach enables standardized adoption of generative data augmentation, which can be applied orthogonally to existing regularization techniques.μ΅κ·Ό λ₯λ¬λ κΈ°λ° μμ± λͺ¨λΈμ κΈκ²©ν λ°μ μΌλ‘ μ΄λ₯Ό μ΄μ©ν μμ± κΈ°λ° λ°μ΄ν° μ¦κ° κΈ°λ²(generative data augmentation, GDA)μ μ€ν κ°λ₯μ±μ λν κΈ°λκ° μ»€μ§κ³ μλ€. μμ± κΈ°λ° λ°μ΄ν° μ¦κ° κΈ°λ²μ λ₯λ¬λ κΈ°λ° μ μ¬λ³μ λͺ¨λΈμμ μμ± λ μνμ μλ³Έ λ°μ΄ν°μ
μ μΆκ°νμ¬ μ°κ΄λ νμ€ν¬μ μ±λ₯μ ν₯μμν€λ κΈ°μ μ μλ―Ένλ€. λ°λΌμ μμ± κΈ°λ° λ°μ΄ν° μ¦κ° κΈ°λ²μ λ°μ΄ν° 곡κ°μμ μ΄λ€μ§λ μ κ·ν κΈ°μ μ ν ννλ‘ κ°μ£Όλ μ μλ€. μ΄λ¬ν λ₯λ¬λ κΈ°λ° μμ± λͺ¨λΈμ μλ‘μ΄ νμ© κ°λ₯μ±μ μμ°μ΄μ²λ¦¬ λΆμΌμμ λμ± μ€μνκ² λΆκ°λλ μ΄μ λ (1) λ²μ© κ°λ₯ν ν
μ€νΈ λ°μ΄ν° μ¦κ° κΈ°μ μ λΆμ¬μ (2) ν
μ€νΈ λ°μ΄ν°μ ν¬μμ±μ 극볡ν μ μλ λμμ΄ νμνκΈ° λλ¬Έμ΄λ€. λ¬Έμ μ 볡μ‘λμ νΉμ§μ κ³¨κ³ λ£¨ μ±μ§νκΈ° μν΄ λ³Έ λ
Όλ¬Έμμλ ν
μ€νΈ λΆλ₯(text classification), μμ°¨μ λ μ΄λΈλ§κ³Ό λ©ν°νμ€νΉ κΈ°μ μ΄ νμν λ°ν μ΄ν΄(spoken language understanding, SLU), κ³μΈ΅μ μ΄λ©° μ¬κ·μ μΈ λ°μ΄ν° ꡬ쑰μ λν κ³ λ €κ° νμν λν μν μΆμ (dialogue state tracking, DST) λ± μΈ κ°μ§ λ¬Έμ μμ λ₯λ¬λ κΈ°λ° μμ± λͺ¨λΈμ νμ©ν λ°μ΄ν° μ¦κ° κΈ°λ²μ νλΉμ±μ λν΄ λ€λ£¬λ€. λ³Έ μ°κ΅¬μμλ 쑰건λΆ, κ³μΈ΅μ λ° μμ°¨μ variational autoencoder (VAE)μ κΈ°λ°νμ¬ κ° μμ°μ΄μ²λ¦¬ λ¬Έμ μ νΉνλ ν
μ€νΈ λ° μ°κ΄ λΆμ°© μ 보λ₯Ό λμμ μμ±νλ νΉμ λ₯λ¬λ μμ± λͺ¨λΈλ€μ μ μνκ³ , λ€μν νλ₯ λͺ¨λΈκ³Ό λ°μ΄ν°μ
μ λ€λ£¨λ λ± ν λμ μ€νμ ν΅ν΄ λ₯ μμ± λͺ¨λΈ κΈ°λ° λ°μ΄ν° μ¦κ° κΈ°λ²μ ν¨κ³Όλ₯Ό ν΅κ³μ μΌλ‘ μ
μ¦νμλ€. λΆμμ μ°κ΅¬μμλ μκΈ°νκ·μ (autoregressive) VAEμμ λΉλ²ν λ°μνλ posterior collapse λ¬Έμ μ λν΄ νꡬνκ³ , ν΄λΉ λ¬Έμ λ₯Ό μνν μ μλ μ κ· λ°©μλ μ μνλ€. ν΄λΉ λ°©λ²μ μμ±μ λ°μ΄ν° μ¦κ°μ νμν 볡μ‘ν VAE λͺ¨λΈμ μ μ©νμμ λ, μμ± λͺ¨λΈμ μμ± μ§μ΄ ν₯μλμ΄ λ°μ΄ν° μ¦κ° ν¨κ³Όμλ κΈμ μ μΈ μν₯μ λ―ΈμΉ μ μμμ κ²μ¦νμλ€. λ³Έ λ
Όλ¬Έμ ν΅ν΄ μμ°μ΄μ²λ¦¬ λΆμΌμμ κΈ°μ‘΄ μ κ·ν κΈ°λ²κ³Ό λ³ν μ μ© κ°λ₯ν λΉμ§λ ννμ λ°μ΄ν° μ¦κ° κΈ°λ²μ νμ€νλ₯Ό κΈ°λν΄ λ³Ό μ μλ€.1 Introduction 1
1.1 Motivation 1
1.2 Dissertation Overview 6
2 Background and Related Work 8
2.1 Deep Latent Variable Models 8
2.1.1 Variational Autoencoder (VAE) 10
2.1.2 Deep Generative Models and Text Generation 12
2.2 Data Augmentation 12
2.2.1 General Description 13
2.2.2 Categorization of Data Augmentation 14
2.2.3 Theoretical Explanations 21
2.3 Summary 24
3 Basic Task: Text Classi cation 25
3.1 Introduction 25
3.2 Our Approach 28
3.2.1 Proposed Models 28
3.2.2 Training with I-VAE 29
3.3 Experiments 31
3.3.1 Datasets 32
3.3.2 Experimental Settings 33
3.3.3 Implementation Details 34
3.3.4 Data Augmentation Results 36
3.3.5 Ablation Studies 39
3.3.6 Qualitative Analysis 40
3.4 Summary 45
4 Multi-task Learning: Spoken Language Understanding 46
4.1 Introduction 46
4.2 Related Work 48
4.3 Model Description 48
4.3.1 Framework Formulation 48
4.3.2 Joint Generative Model 49
4.4 Experiments 56
4.4.1 Datasets 56
4.4.2 Experimental Settings 57
4.4.3 Generative Data Augmentation Results 61
4.4.4 Comparison to Other State-of-the-art Results 63
4.4.5 Ablation Studies 63
4.5 Summary 67
5 Complex Data: Dialogue State Tracking 68
5.1 Introduction 68
5.2 Background and Related Work 70
5.2.1 Task-oriented Dialogue 70
5.2.2 Dialogue State Tracking 72
5.2.3 Conversation Modeling 72
5.3 Variational Hierarchical Dialogue Autoencoder (VHDA) 73
5.3.1 Notations 73
5.3.2 Variational Hierarchical Conversational RNN 74
5.3.3 Proposed Model 75
5.3.4 Posterior Collapse 82
5.4 Experimental Results 84
5.4.1 Experimental Settings 84
5.4.2 Data Augmentation Results 90
5.4.3 Intrinsic Evaluation - Language Evaluation 94
5.4.4 Qualitative Results 95
5.5 Summary 101
6 Conclusion 103
6.1 Summary 103
6.2 Limitations 104
6.3 Future Work 105Docto
Variational Bayes Estimation of Discrete-Margined Copula Models with Application to Time Series
We propose a new variational Bayes estimator for high-dimensional copulas
with discrete, or a combination of discrete and continuous, margins. The method
is based on a variational approximation to a tractable augmented posterior, and
is faster than previous likelihood-based approaches. We use it to estimate
drawable vine copulas for univariate and multivariate Markov ordinal and mixed
time series. These have dimension , where is the number of observations
and is the number of series, and are difficult to estimate using previous
methods. The vine pair-copulas are carefully selected to allow for
heteroskedasticity, which is a feature of most ordinal time series data. When
combined with flexible margins, the resulting time series models also allow for
other common features of ordinal data, such as zero inflation, multiple modes
and under- or over-dispersion. Using six example series, we illustrate both the
flexibility of the time series copula models, and the efficacy of the
variational Bayes estimator for copulas of up to 792 dimensions and 60
parameters. This far exceeds the size and complexity of copula models for
discrete data that can be estimated using previous methods
Generative Models For Deep Learning with Very Scarce Data
The goal of this paper is to deal with a data scarcity scenario where deep
learning techniques use to fail. We compare the use of two well established
techniques, Restricted Boltzmann Machines and Variational Auto-encoders, as
generative models in order to increase the training set in a classification
framework. Essentially, we rely on Markov Chain Monte Carlo (MCMC) algorithms
for generating new samples. We show that generalization can be improved
comparing this methodology to other state-of-the-art techniques, e.g.
semi-supervised learning with ladder networks. Furthermore, we show that RBM is
better than VAE generating new samples for training a classifier with good
generalization capabilities
- β¦