Domain Conditioned Adaptation Network

Tremendous research efforts have been made to thrive deep domain adaptation (DA) by seeking domain-invariant features. Most existing deep DA models only focus on aligning feature representations of task-specific layers across domains while integrating a totally shared convolutional architecture for source and target. However, we argue that such strongly-shared convolutional layers might be harmful for domain-specific feature learning when source and target data distribution differs to a large extent. In this paper, we relax a shared-convnets assumption made by previous DA methods and propose a Domain Conditioned Adaptation Network (DCAN), which aims to excite distinct convolutional channels with a domain conditioned channel attention mechanism. As a result, the critical low-level domain-dependent knowledge could be explored appropriately. As far as we know, this is the first work to explore the domain-wise convolutional channel activation for deep DA networks. Moreover, to effectively align high-level feature distributions across two domains, we further deploy domain conditioned feature correction blocks after task-specific layers, which will explicitly correct the domain discrepancy. Extensive experiments on three cross-domain benchmarks demonstrate the proposed approach outperforms existing methods by a large margin, especially on very tough cross-domain learning tasks.Comment: Accepted by AAAI 202

다양한 질환 심각도 하에서 말초동맥 질환 위치 식별을 위한 딥러닝 기반 도메인 적응 방법 연구

학위논문(석사) -- 서울대학교대학원 : 공과대학 기계공학부, 2022.2. 윤병동.This paper's primary purpose is to develop a blood pressure waveform (BPW) based deep learning diagnosis model for identifying peripheral arterial disease (PAD) on frequent PAD occurrence arteries. Two issues make it hard to obtain a generalized PAD diagnosis model with a data-driven approach: 1) domain discrepancy resulted from the differences of disease severity and occurring location, 2) data imbalance resulted from the symptomless characteristic of mild PAD. To train a generalized PAD diagnosis model considering practical issues, we propose auxiliary tasks-assisted maximum classifier discrepancy for supervised domain adaptation. The proposed model is validated using virtual patients' BPWs generated from the transmission line model under various disease severity levels. The results show that the proposed model has a superior performance for identifying PAD locations under various disease severity levels. This finding indicates the feasibility of the proposed diagnosis model to real hospitals for identifying the PAD locations in the lower extremities under various disease severity.본 논문의 주요 목적은 말초동맥 질환 빈번 발생 동맥에서 말초 동맥 질환을 식별하기 위한 혈압 파형 기반 딥러닝 진단 모델을 개발하는 것이다. 데이터 기반 방식으로 일반화된 말초동맥 질환 진단 모델을 얻기 위해서는 2가지 문제점이 있다: 1) 질환 심각도와 발병 위치의 차이로 인한 도메인 불일치, 2) 말초동맥 질환 초기 증상이 없다는 특징으로 인한 데이터 불균형. 실제 문제를 고려하여 일반화된 말초동맥 질환 진단 모델 훈련을 위해, 최대 분류 불일치 방법에 두가지 보조 태스크를 추가한 지도 도메인 적응 방법을 제안한다. 제안된 모델은 다양한 질병 심각도 수준에서 전송 선로 모델에서 생성된 가상 환자의 혈압파형을 사용하여 검증된다. 결과는 제안된 모델이 다양한 질병 심각도 수준에서 PAD 위치를 식별하기 위한 우수한 성능을 가지고 있음을 보여준다. 이 결과는 다양한 질병 심각도에서 하지의 PAD 위치를 식별하기 위해 제안된 진단 모델을 실제 병원에 적용할 가능성을 나타낸다.Chapter 1. Introduction 1 1.1 Motivation 1 1.2 Structure of the Thesis 3 Chapter 2. Materials and Methods 4 2.1 Problem Definition of naïve data-driven approach 4 2.2 Proposed Method for Training Generalized PAD Diagnosis Model 5 2.2.1 Domain Adaptation 5 2.2.2 Maximum Classifier Discrepancy 6 2.2.3 Proposed Methods 10 2.3 Virtual PAD Patients’ BPW Data Generation 14 2.3.1 Transmission Line Model 14 2.3.2 Setting for Virtual PAD Patients 17 2.3.3 Data Description 18 2.4 Overall Procedure 20 Chapter 3. Results 22 3.1 Compared Methods 22 3.2 Results 22 Chapter 4. Discussion 28 4.1 Efficacy of Proposed Learning Method 28 4.2 Effects of Domain Adaptation 29 4.3 Potential for Practical Applicability 30 Chapter 5. Conclusions 31 5.1 Summary and Contributions 31 5.2 Suggestions for Future Research 32 References 35 Abstract (Korean) 41석

Domain Agnostic Internal Distributions for Unsupervised Model Adaptation

We develop an algorithm for sequential adaptation of a classifier that is trained for a source domain to generalize in a unannotated target domain. We consider that the model has been trained on the source domain annotated data and then it needs to be adapted using the target domain unannotated data when the source domain data is not accessible. We align the distributions of the source and the target domains in a discriminative embedding space via an intermediate internal distribution. This distribution is estimated using the source data representations in the embedding space. We provide theoretical analysis and conduct extensive experiments on several benchmarks to demonstrate the proposed method is effective

HoMM: Higher-order Moment Matching for Unsupervised Domain Adaptation

Minimizing the discrepancy of feature distributions between different domains is one of the most promising directions in unsupervised domain adaptation. From the perspective of distribution matching, most existing discrepancy-based methods are designed to match the second-order or lower statistics, which however, have limited expression of statistical characteristic for non-Gaussian distributions. In this work, we explore the benefits of using higher-order statistics (mainly refer to third-order and fourth-order statistics) for domain matching. We propose a Higher-order Moment Matching (HoMM) method, and further extend the HoMM into reproducing kernel Hilbert spaces (RKHS). In particular, our proposed HoMM can perform arbitrary-order moment tensor matching, we show that the first-order HoMM is equivalent to Maximum Mean Discrepancy (MMD) and the second-order HoMM is equivalent to Correlation Alignment (CORAL). Moreover, the third-order and the fourth-order moment tensor matching are expected to perform comprehensive domain alignment as higher-order statistics can approximate more complex, non-Gaussian distributions. Besides, we also exploit the pseudo-labeled target samples to learn discriminative representations in the target domain, which further improves the transfer performance. Extensive experiments are conducted, showing that our proposed HoMM consistently outperforms the existing moment matching methods by a large margin. Codes are available at \url{https://github.com/chenchao666/HoMM-Master}Comment: Accept by AAAI-2020, codes are available at https://github.com/chenchao666/HoMM-Maste