Workload-aware Automatic Parallelization for Multi-GPU DNN Training

Deep neural networks (DNNs) have emerged as successful solutions for variety of artificial intelligence applications, but their very large and deep models impose high computational requirements during training. Multi-GPU parallelization is a popular option to accelerate demanding computations in DNN training, but most state-of-the-art multi-GPU deep learning frameworks not only require users to have an in-depth understanding of the implementation of the frameworks themselves, but also apply parallelization in a straight-forward way without optimizing GPU utilization. In this work, we propose a workload-aware auto-parallelization framework (WAP) for DNN training, where the work is automatically distributed to multiple GPUs based on the workload characteristics. We evaluate WAP using TensorFlow with popular DNN benchmarks (AlexNet and VGG-16), and show competitive training throughput compared with the state-of-the-art frameworks, and also demonstrate that WAP automatically optimizes GPU assignment based on the workload's compute requirements, thereby improving energy efficiency.Comment: This paper is accepted in ICASSP201

Latent tuberculosis co-infection is associated with heightened levels of humoral, cytokine and acute phase responses in seropositive SARS-CoV-2 infection

OBJECTIVES: : Latent Tuberculosis infection (LTBI) is postulated to modulate immune responses and alter disease severity in SARS-CoV-2 co-infection. However, no data exist on the effect of LTBI on the immune responses in SARS-CoV-2 co-infected individuals. METHODS: : We examined the SARS-CoV-2 specific antibody responses, plasma cytokines, chemokines, acute phase proteins and growth factor levels in LTBI positive and negative individuals with SARS-CoV-2 infection. RESULTS: : Our results demonstrated that individuals with LTBI (LTBI+) and seropositive for SARS-CoV-2 infection were associated with elevated SARS-CoV-2 specific IgM, IgG and IgA antibodies, as well as enhanced neutralization activity compared to those negative for LTBI (LTBI-) individuals. Our results also demonstrate that LTBI+ individuals exhibited significantly higher plasma levels of IFNγ, IL-2, TNFα, IL-1α, IL-1β, IL-6, IL-12, IL-15, IL-17, IL-3, GM-CSF, IL-10, IL-25, IL-33, CCL3 and CXCL10 compared to LTBI- individuals. Finally, our results show that LTBI+ individuals exhibit significantly higher levels of C-reactive protein, alpha-2 macroglobulin, VEGF and TGFα compared to LTBI- individuals. CONCLUSIONS: : Thus, our data clearly demonstrates that LTBI+ individuals seropositive for SARS-CoV2 infection exhibit heightened levels of humoral, cytokine and acute phase responses compared to LTBI- individuals. Thus, LTBI is associated with modulation of antibody and cytokine responses as well as systemic inflammation in individuals seropositive for SARS-CoV2 infection

BCG vaccination induces enhanced frequencies of dendritic cells and altered plasma levels of type I and type III interferons in elderly individuals

OBJECTIVE: : BCG can improve the response to vaccines directed against viral infections and also BCG vaccination reduces all-cause mortality, most likely by protection against unrelated infections. However, the effect of BCG vaccination on the dendritic cells (DC) subsets is not well characterized. METHODS: : We investigated the impact of BCG vaccination on the frequencies of DC subsets and type I and III interferons (IFNs) using whole blood and plasma samples in a group of elderly individuals (age 60-80 years) at one month post vaccination as part of our clinical study to examine the effect of BCG on COVID-19. RESULTS: : Our results demonstrate that BCG vaccination induced enhanced frequencies of plasmacytoid DC (pDC) and myeloid DC (mDC). BCG vaccination also induced diminished plasma levels of type I IFNs like IFNα and IFNβ but increased levels of type III IFNs IL-28A and IL-29. CONCLUSIONS: : Thus, BCG vaccination was associated with enhanced DC subsets as well IL-29 in elderly individuals, suggesting its ability to induce non-specific innate immune responses

BCG vaccination induces enhanced frequencies of memory T cells and altered plasma levels of common γc cytokines in elderly individuals

BCG vaccination is known to induce innate immune memory, which confers protection against heterologous infections. However, the effect of BCG vaccination on the conventional adaptive immune cells subsets is not well characterized. We investigated the impact of BCG vaccination on the frequencies of T cell subsets and common gamma c (γc) cytokines in a group of healthy elderly individuals (age 60–80 years) at one month post vaccination as part of our clinical study to examine the effect of BCG on COVID-19. Our results demonstrate that BCG vaccination induced enhanced frequencies of central (p<0.0001) and effector memory (p<0.0001) CD4+ T cells and diminished frequencies of naïve (p<0.0001), transitional memory (p<0.0001), stem cell memory (p = 0.0001) CD4+ T cells and regulatory T cells. In addition, BCG vaccination induced enhanced frequencies of central (p = 0.0008), effector (p<0.0001) and terminal effector memory (p<0.0001) CD8+ T cells and diminished frequencies of naïve (p<0.0001), transitional memory (p<0.0001) and stem cell memory (p = 0.0034) CD8+T cells. BCG vaccination also induced enhanced plasma levels of IL-7 (p<0.0001) and IL-15 (p = 0.0020) but diminished levels of IL-2 (p = 0.0033) and IL-21 (p = 0.0020). Thus, BCG vaccination was associated with enhanced memory T cell subsets as well as memory enhancing γc cytokines in elderly individuals, suggesting its ability to induce non-specific adaptive immune responses

Characterization of memory T cell subsets and common γ−chain cytokines in convalescent COVID-19 individuals

T cells are thought to be an important correlates of protection against SARS‐CoV2 infection. However, the composition of T cell subsets in convalescent individuals of SARS‐CoV2 infection has not been well studied. The authors determined the lymphocyte absolute counts, the frequency of memory T cell subsets, and the plasma levels of common γ−chain in 7 groups of COVID‐19 individuals, based on days since RT‐PCR confirmation of SARS‐CoV‐2 infection. The data show that both absolute counts and frequencies of lymphocytes as well as, the frequencies of CD4(+) central and effector memory cells increased, and the frequencies of CD4(+) naïve T cells, transitional memory, stem cell memory T cells, and regulatory cells decreased from Days 15–30 to Days 61–90 and plateaued thereafter. In addition, the frequencies of CD8(+) central memory, effector, and terminal effector memory T cells increased, and the frequencies of CD8(+) naïve cells, transitional memory, and stem cell memory T cells decreased from Days 15–30 to Days 61–90 and plateaued thereafter. The plasma levels of IL‐2, IL‐7, IL‐15, and IL‐21—common γc cytokines started decreasing from Days 15–30 till Days 151–180. Severe COVID‐19 patients exhibit decreased levels of lymphocyte counts and frequencies, higher frequencies of naïve cells, regulatory T cells, lower frequencies of central memory, effector memory, and stem cell memory, and elevated plasma levels of IL‐2, IL‐7, IL‐15, and IL‐21. Finally, there was a significant correlation between memory T cell subsets and common γc cytokines. Thus, the study provides evidence of alterations in lymphocyte counts, memory T cell subset frequencies, and common γ−chain cytokines in convalescent COVID‐19 individuals