Empirical Performance Analysis of Collective Communication for Distributed Deep Learning in a Many-Core CPU Environment

To accommodate lots of training data and complex training models, “distributed” deep learning training has become employed more and more frequently. However, communication bottlenecks between distributed systems lead to poor performance of distributed deep learning training. In this study, we proposed a new collective communication method in a Python environment by utilizing Multi-Channel Dynamic Random Access Memory (MCDRAM) in Intel Xeon Phi Knights Landing processors. Major deep learning software platforms, such as TensorFlow and PyTorch, offer Python as a main development language, so we developed an efficient communication library by adapting Memkind library, which is a C-based library to utilize high-performance memory MCDRAM. For performance evaluation, we tested the popular collective communication methods in distributed deep learning, such as Broadcast, Gather, and AllReduce. We conducted experiments to analyze the effect of high-performance memory and processor location on communication performance. In addition, we analyze performance in a Docker environment for further relevance given the recent major trend of Cloud computing. By extensive experiments in our testbed, we confirmed that the communication in our proposed method showed performance improvement by up to 487%

Communication Optimization Schemes for Accelerating Distributed Deep Learning Systems

In a distributed deep learning system, a parameter server and workers must communicate to exchange gradients and parameters, and the communication cost increases as the number of workers increases. This paper presents a communication data optimization scheme to mitigate the decrease in throughput due to communication performance bottlenecks in distributed deep learning. To optimize communication, we propose two methods. The first is a layer dropping scheme to reduce communication data. The layer dropping scheme we propose compares the representative values of each hidden layer with a threshold value. Furthermore, to guarantee the training accuracy, we store the gradients that are not transmitted to the parameter server in the worker’s local cache. When the value of gradients stored in the worker’s local cache is greater than the threshold, the gradients stored in the worker’s local cache are transmitted to the parameter server. The second is an efficient threshold selection method. Our threshold selection method computes the threshold by replacing the gradients with the L1 norm of each hidden layer. Our data optimization scheme reduces the communication time by about 81% and the total training time by about 70% in a 56 Gbit network environment

Trends in Research on Patients With COVID-19 in Korean Medical Journals

Objectives: This study was conducted to systematically summarize trends in research concerning patients with coronavirus disease 2019 (COVID-19) as reported in Korean medical journals. Methods: We performed a literature search of KoreaMed from January 2020 to September 2022. We included only primary studies of patients with COVID-19. Two reviewers screened titles and abstracts, then performed full-text screening, both independently and in duplicate. We first identified the 5 journals with the greatest numbers of eligible publications, then extracted data pertaining to the general characteristics, study population attributes, and research features of papers published in these journals. Results: Our analysis encompassed 142 primary studies. Of these, approximately 41.0% reported a funding source, while 3.5% disclosed a conflict of interest. In 2020, 42.9% of studies included fewer than 10 participants; however, by 2022, the proportion of studies with over 200 participants had increased to 40.6%. The most common design was the cohort study (48.6%), followed by case reports/series (35.2%). Only 3 randomized controlled trials were identified. Studies most frequently focused on prognosis (58.5%), followed by therapy/intervention (20.4%). Regarding the type of intervention/exposure, therapeutic clinical interventions comprised 26.1%, while studies of morbidity accounted for 13.4%. As for the outcomes measured, 50.7% of studies assessed symptoms/clinical status/improvement, and 14.1% evaluated mortality. Conclusions: Employing a systematic approach, we examined the characteristics of research involving patients with COVID-19 that was published in Korean medical journals from 2020 onward. Subsequent research should assess not only publication trends over a longer timeframe but also the quality of evidence provided

Factors associated with adherence to noninvasive positive pressure ventilation in amyotrophic lateral sclerosis.

IntroductionThis cohort study aimed to investigate the factors associated with noninvasive positive pressure ventilation adherence and assess the long-term effects of noninvasive positive pressure ventilation adherence in patients with amyotrophic lateral sclerosis (ALS).MethodsThe medical records of patients with ALS admitted to a tertiary hospital for noninvasive positive pressure ventilation initiation were retrospectively reviewed. Pulmonary function parameters, variables of blood gas analysis, the site of symptom onset, the time from onset and diagnosis to noninvasive positive pressure ventilation application, ALS Functional Rating Scale-Revised, neurophysiological index, and the length of hospital stay were evaluated. The adherence to noninvasive positive pressure ventilation was defined as the use of noninvasive positive pressure ventilation for ≥ 2 h/day or ≥ 4 h/day. The correlations between noninvasive positive pressure ventilation adherence or length of hospital stay and other clinical parameters were analyzed.ResultsFifty-one patients with ALS were included in the study. The time from onset and diagnosis to NIPPV application was reduced by 16 months in the adherent group than that in the non-adherent group; however, the parameters of blood gas analysis and pulmonary function tests did not differ significantly between the groups. Furthermore, the neurophysiological index of the abductor digiti minimi muscle was higher by 4.05 in the adherent group than that in the non-adherent group. The adherence to noninvasive positive pressure ventilation prolonged tracheostomy-free survival compared to that of non-adherence. Desaturation events, lower forced vital capacity, last pCO2, bicarbonate, and base excess, and higher differences in pCO2, were associated with an increase in the length of hospital stay.ConclusionsNoninvasive positive pressure ventilation application shortly after symptom onset and ALS diagnosis in patients with CO2 retention and reduced forced vital capacity can be considered for successful adherence. Adherence to noninvasive positive pressure ventilation may result in reduced tracheostomy conversion rates and prolonged tracheostomy-free survival