Library-level Distributed Processing Method for CNN Training Using GPU Clusters

학위논문(석사)--서울대학교 대학원 :공과대학 컴퓨터공학부,2020. 2. 이재진.This paper presents the performance characteristics of CNN model distributed learning using GPU clusters. Unlike the classical model study, which simply reports the total memory usage or total execution time of the network model, in this paper, the distributed learning execution time is reported per layer and the computation strength considering both the computation amount and the memory access is presented as the main performance metric of the deep learning operation. This makes research on the performance characteristics of a particular model not only limited to that model, but also becomes a generally applicable methodology. And unlike research that reports the performance of a specific deep learning framework, it reports the performance characteristics of CNN models distributed at the library level. Most frameworks manage communications and operations as separate modules, which makes each library entangled with multiple parts of the framework and a mesh of dependencies, making it difficult to distribute distributedly. This paper introduces a library-level distributed processing method and lists the advantages it has over the framework foundation. To verify the effectiveness of the proposed method, we developed a prototype lightweight library and developed a CNN model benchmark based on it.본 논문은 GPU 클러스터를 이용한 CNN 모델 분산 학습에서의 성능 특성을 제시한다. 단순히 네트워크의 메모리 사용량이나 총 실행 시간만을 보고하는 모델 연구와는 달리, 레이어 별로 분산 학습 실행 시간을 보고하며, 계산량과 메모리 접근을 함께 고려한 연산 세기를 딥 러닝 연산의 주된 성능 척도로 제시한다. 이를 통해 특정 모델의 성능 특성 연구가 해당 모델에만 국한되는 것이 아니라, 일반적으로 적용할 수 있는 방법론이 된다. 그리고 특정 딥 러닝 프레임워크의 성능을 보고하는 연구와는 달리, CNN 모델을 라이브러리 수준에서 분산 처리할 때의 성능 특성을 보고한다. 대부분의 프레임워크에서는 통신과 연산을 별도의 모듈로 따로 관리하는데, 이로 인해 각각의 라이브러리가 프레임워크의 여러 부분과 의존성의 그물로 얽혀있게 되고, 유연한 분산 처리가 어려워진다. 본 논문은 라이브러리 수준의 분산 처리 방법을 소개하고, 이것이 프레임워크 기반에 비해 갖는 장점을 열거한다. 소개한 방법이 실효성을 갖는지 확인하기위해, 프로토타입 경량 라이브러리를 개발하고 이를 기반으로 CNN 모델 벤치마크를 개발하여, 실험을 통해 이를 검증하였다.제 1 장 서론 1 제 2 장 관련 연구 3 제 3 장 CNN 학습 알고리즘 4 3.1 역전파 알고리즘 4 3.2 주요 연산 별 학습 알고리즘 7 제 4 장 CNN의 분산 학습 방법 17 4.1 프레임워크의 분산 학습 방법 17 4.2 라이브러리 수준의 분산 학습 방법 20 제 5 장 CNN의 분산 학습 성능 특성 24 5.1 레이어 수준의 성능 특성 24 5.2 블록 수준의 성능 특성 26 5.3 모델 수준의 성능 특성 29 제 6 장 실험 및 결과 분석 30 6.1 실험 환경 30 6.2 실험 결과 31 6.3 결과 분석 39 제 7 장 결론 42 참고문헌 43 Abstract 45Maste

DEVELOPMENT OF TUNABLE SUPERHYDROPHOBIC SURFACES BY DUAL-SCALE SURFACE MODIFICATION USING SIMPLE SILICON WET ETCHING AND ZnO NANORODS FORMATION

1

Study of Wetting Behaviors on Micro- and Nano-engineered Surfaces: Fabrication, Characterization and Applications

DoctorPart I: We introduce effective dual-scaled surfaces using the combination of the conventional silicon wet-etching technique (for microstructures) and a solution method for ZnO nanorod formation (for nanostructures). The microstructures were sloped to facilitate the overall deposition of a ZnO seed-layer as well as the growth of nanostructures over the entire surface area. The ZnO nanorods were formed using growth solutions of various pHs. The fabricated dual-scaled surfaces were also coated with hydrophobic self-assembled monolayers (SAMs) and compared with the surfaces without the SAM coating to examine the structural effects on both hydrophilic and hydrophobic regions. A total of 16 different samples were examined and analyzed systematically by comparing the static (i.e., contact angle) and dynamic (i.e., spreadability, and contact angle hysteresis) wettability. Part II: This paper presents the results of evaporation experiments using water droplets on aluminum sheets that were either smooth or had surface structures at the micro-scale, the nano-scale or at both micro- and nano-scale (dual-scale). The smooth surface was a polished aluminum sheetthe surface with micro-scale structures was obtained by sandblastingthe surface with nano-scale structures was obtained using conventional aluminum anodization, and the surface with dual-scale structures was prepared using sandblasting and anodization sequentially. The wetting properties and evaporation rates were measured for each surface. The evaporation rates were affected by their static and dynamic wetting properties. Evaporation on the surface with dual-scale structures was fastest and the evaporation rate was analyzed quantitatively. Part III: We report the drop impact characteristics on four hydrophobic surfaces with different well-scale structures (smooth, nano, micro, and hierarchical micro/nano) and the effects of those structures on behavior of water drops during impact. The specimens were fabricated using silicon wet etching, black silicon formation, or the combination of these methods. On the surfaces, the microstructures form obstacles to drop spreading and retracting, the nanostructures give extreme water-repellency, and the hierarchical micro/nanostructures facilitate drop fragmentation. The maximum spreading factor (D*max) differed among the structures. Based on published models of D*max, we interpret the results of our experiment, and suggest reasonable explanations for these differences. Especially, the micro/nanostructures caused instability of the interface between liquid and air at Weber number (We) > ~80, and impacting drops fragmented at We > ~150. .Part IV: We report the droplet impact characteristics on five superhydrophobic surfaces with different well-defined micro-scale structures and the effects of the roughness on the behavior of water droplets during impact, especially the threshold from rebound to fragmentation. The specimens were fabricated using silicon wet etching and black silicon formation, and the resulting nanostructures caused superhydrophobicity take an important role to decouple the roughness effect and the wettability effect during droplet impact. As results from precisely conducted experiments, specimen 1 having only nano-scale structures required relatively the higher We (Wec = ~190) to generate droplet fragmentation than those in the other specimens, and specimen 5 is the most effective for droplet fragmentation (Wec = ~110). This clearly shows the impacting droplet fragmentation can be facilitated by the surface roughness. However, otherwise known in previous works, arithmetic surface roughness (Ra) is not proper parameter for correlation with the fragmentation event. We interpret the results of our experiment and suggest alternative surface roughness parameter, Wenzel roughness (Rw) instead of Ra

티타늄 임플란트 표면에 상온초박막코팅기법으로 수산화인회석을 코팅한 HAPTITE 임플란트의 초기 안정성

학위논문(석사)아주대학교 임상치의학대학원 :임상치의학과,2013. 8Maste

마이크로/나노 사이즈 구조를 갖는 고효율 증발 표면의 개발

2

Spreading and Fragmenting Characteristics of Impacting Droplet on Micro/ Nanostructured Water-Repellent Surfaces

1

EFFECTIVE THREE-DIMENSIONAL SUPERHYDROPHOBIC CHANNEL COATING USING ORGANICALLY MODIFIED SILICA AEROGEL

1

Wettability Control of Stainless Steel Mesh by Electrochemical Etching

1

마이크로/나노 사이즈 구조를 갖는 극발수 특성의 초소수성 표면 개발

2

고속 적외선 열화상 기법을 이용한 마이크로/나노구조표면에서의 미시적 비등 인자들의 측정

2