175 research outputs found
IOP-FL: Inside-Outside Personalization for Federated Medical Image Segmentation
Federated learning (FL) allows multiple medical institutions to
collaboratively learn a global model without centralizing all clients data. It
is difficult, if possible at all, for such a global model to commonly achieve
optimal performance for each individual client, due to the heterogeneity of
medical data from various scanners and patient demographics. This problem
becomes even more significant when deploying the global model to unseen clients
outside the FL with new distributions not presented during federated training.
To optimize the prediction accuracy of each individual client for critical
medical tasks, we propose a novel unified framework for both Inside and Outside
model Personalization in FL (IOP-FL). Our inside personalization is achieved by
a lightweight gradient-based approach that exploits the local adapted model for
each client, by accumulating both the global gradients for common knowledge and
local gradients for client-specific optimization. Moreover, and importantly,
the obtained local personalized models and the global model can form a diverse
and informative routing space to personalize a new model for outside FL
clients. Hence, we design a new test-time routing scheme inspired by the
consistency loss with a shape constraint to dynamically incorporate the models,
given the distribution information conveyed by the test data. Our extensive
experimental results on two medical image segmentation tasks present
significant improvements over SOTA methods on both inside and outside
personalization, demonstrating the great potential of our IOP-FL scheme for
clinical practice. Code will be released at https://github.com/med-air/IOP-FL.Comment: Submitted to IEEE TMI special issue on federated learning for medical
imagin
D-STEM: a Design led approach to STEM innovation
Advances in the Science, Technology, Engineering and Maths (STEM) disciplines offer opportunities for designers to propose and make products with advanced, enhanced and engineered properties and functionalities. In turn, these advanced characteristics are becoming increasingly necessary as resources become ever more strained through 21st century demands, such as ageing populations, connected communities, depleting raw materials, waste management and energy supply. We need to make things that are smarter, make our lives easier, better and simpler. The products of tomorrow need to do more with less. The issue is how to maximize the potential for exploiting opportunities offered by STEM developments and how best to enable designers to strengthen their position within the innovation ecosystem. As a society, we need designers able to navigate emerging developments from the STEM community to a level that enables understanding and knowledge of the new material properties, the skill set to facilitate absorption into the design âtoolboxâ and the agility to identify, manage and contextualise innovation opportunities emerging from STEM developments. This paper proposes the blueprint for a new design led approach to STEM innovation that begins to redefine studio culture for the 21st Century
Decoupled Model Schedule for Deep Learning Training
Recent years have seen an increase in the development of large deep learning
(DL) models, which makes training efficiency crucial. Common practice is
struggling with the trade-off between usability and performance. On one hand,
DL frameworks such as PyTorch use dynamic graphs to facilitate model developers
at a price of sub-optimal model training performance. On the other hand,
practitioners propose various approaches to improving the training efficiency
by sacrificing some of the flexibility, ranging from making the graph static
for more thorough optimization (e.g., XLA) to customizing optimization towards
large-scale distributed training (e.g., DeepSpeed and Megatron-LM).
In this paper, we aim to address the tension between usability and training
efficiency through separation of concerns. Inspired by DL compilers that
decouple the platform-specific optimizations of a tensor-level operator from
its arithmetic definition, this paper proposes a schedule language to decouple
model execution from definition. Specifically, the schedule works on a PyTorch
model and uses a set of schedule primitives to convert the model for common
model training optimizations such as high-performance kernels, effective 3D
parallelism, and efficient activation checkpointing. Compared to existing
optimization solutions, we optimize the model as-needed through high-level
primitives, and thus preserving programmability and debuggability for users to
a large extent. Our evaluation results show that by scheduling the existing
hand-crafted optimizations in a systematic way, we are able to improve training
throughput by up to 3.35x on a single machine with 8 NVIDIA V100 GPUs, and by
up to 1.32x on multiple machines with up to 64 GPUs, when compared to the
out-of-the-box performance of DeepSpeed and Megatron-LM
Downregulation of E-Cadherin enhances proliferation of head and neck cancer through transcriptional regulation of EGFR
<p>Abstract</p> <p>Background</p> <p>Epidermal growth factor receptor (EGFR) has been reported to downregulate E-cadherin (E-cad); however, whether the downregulation of E-cad has any effect on EGFR expression has not been elucidated. Our previous studies have found an inverse correlation between EGFR and E-cad expression in tissue specimens of squamous cell carcinoma of the head and neck (SCCHN). To understand the biological mechanisms underlying this clinical observation, we knocked down E-cad expression utilizing E-cad siRNA in four SCCHN cell lines.</p> <p>Results</p> <p>It was observed that downregulation of E-cad upregulated EGFR expression compared with control siRNA-transfected cells after 72 hours. Cellular membrane localization of EGFR was also increased. Consequently, downstream signaling molecules of the EGFR signaling pathway, p-AKT, and p-ERK, were increased at 72 hours after the transfection with E-cad siRNA. Reverse transcriptase-polymerase chain reaction (RT-PCR) showed EGFR mRNA was upregulated by E-cad siRNA as early as 24 hours. In addition, RT-PCR revealed this upregulation was due to the increase of EGFR mRNA stability, but not protein stability. Sulforhodamine B (SRB) assay indicated growth of E-cad knocked down cells was enhanced up to 2-fold more than that of control siRNA-transfected cells at 72-hours post-transfection. The effect of E-cad reduction on cell proliferation was blocked by treating the E-cad siRNA-transfected cells with 1 ÎźM of the EGFR-specific tyrosine kinase inhibitor erlotinib.</p> <p>Conclusion</p> <p>Our results suggest for the first time that reduction of E-cad results in upregulation of EGFR transcriptionally. It also suggests that loss of E-cad may induce proliferation of SCCHN by activating EGFR and its downstream signaling pathways.</p
Interfacial engineering of self-assembled monolayer modified semi-roll-to-roll planar heterojunction perovskite solar cells on flexible substrates
A self-assembled monolayer is employed to modify the HTL in inverted perovskite solar cells, which results in significant photovoltaic performance enhancement, and can be applied on roll-to-roll fabricated flexible perovskite solar cells.</p
Achieving high-performance thick-film perovskite solar cells with electron transporting Bingel fullerenes
Two Bingel fullerenes, PCP and MCM, as electron transporting materials (ETMs) have been developed for achieving thick-film perovskite solar cells (PVSCs) with efficiencies beyond 19% with a planar absorber layer over 1 micrometer. Almost no PVSCs have exhibited PCEs above 18% with a 1 micrometer planar layer before, owing to the excess perovskite defects deteriorating charge extraction and the performance of thick-film based devices. Benefiting from the nearly identical optoelectronic properties of two ETMs stemming from tailored chemical structures, the studies on them allow us to unveil the fact that subtle molecular interaction (anionâĎ and Lewis acidâbase) between ETMs and perovskites strongly affects the charge extraction at the heterointerface, which in turn influences the device hysteresis and performance. Particularly, weak Lewis baseâacid OâPb^(2+) interaction between MCM and the perovskite helps passivate the trap-states at the interface, resulting in a smooth electron extraction and reduced device hysteresis (the average hysteresis index (HI) of 0.03 Âą 0.01). However, the strong NâPb^(2+) coordination induces misalignment of the energy levels at the perovskite/PCP heterojunction, causing electron accumulation at the junction, and hence the large HI (0.17 Âą 0.05) in devices. This work provides new insights into the charge extraction at the perovskite/organic interface and the possible molecular interaction from organics to cure perovskite defects
Framework for Hyperspectral Image Processing and Quantification for Cancer Detection During Animal Tumor Surgery
Hyperspectral imaging (HSI) is an imaging modality that holds strong potential for rapid cancer detection during image-guided surgery. But the data from HSI often needs to be processed appropriately in order to extract the maximum useful information that differentiates cancer from normal tissue. We proposed a framework for hyperspectral image processing and quantification, which includes a set of steps including image preprocessing, glare removal, feature extraction, and ultimately image classification. The framework has been tested on images from mice with head and neck cancer, using spectra from 450- to 900-nm wavelength. The image analysis computed Fourier coefficients, normalized reflectance, mean, and spectral derivatives for improved accuracy. The experimental results demonstrated the feasibility of the hyperspectral image processing and quantification framework for cancer detection during animal tumor surgery, in a challenging setting where sensitivity can be low due to a modest number of features present, but potential for fast image classification can be high. This HSI approach may have potential application in tumor margin assessment during image-guided surgery, where speed of assessment may be the dominant factor
Distributed Graph Neural Network Training: A Survey
Graph neural networks (GNNs) are a type of deep learning models that are
trained on graphs and have been successfully applied in various domains.
Despite the effectiveness of GNNs, it is still challenging for GNNs to
efficiently scale to large graphs. As a remedy, distributed computing becomes a
promising solution of training large-scale GNNs, since it is able to provide
abundant computing resources. However, the dependency of graph structure
increases the difficulty of achieving high-efficiency distributed GNN training,
which suffers from the massive communication and workload imbalance. In recent
years, many efforts have been made on distributed GNN training, and an array of
training algorithms and systems have been proposed. Yet, there is a lack of
systematic review on the optimization techniques for the distributed execution
of GNN training. In this survey, we analyze three major challenges in
distributed GNN training that are massive feature communication, the loss of
model accuracy and workload imbalance. Then we introduce a new taxonomy for the
optimization techniques in distributed GNN training that address the above
challenges. The new taxonomy classifies existing techniques into four
categories that are GNN data partition, GNN batch generation, GNN execution
model, and GNN communication protocol. We carefully discuss the techniques in
each category. In the end, we summarize existing distributed GNN systems for
multi-GPUs, GPU-clusters and CPU-clusters, respectively, and give a discussion
about the future direction on distributed GNN training
Recommended from our members
Network during light-induced cotyledons opening and greening in Astragalus membranaceus
Opening and greening are main characteristics of morphogenesis of cotyledons. For revealing interrelationship between metabolism and morphogenesis, metabolic shifts were analyzed in cotyledon of A. membranaceus seedlings with different stages in light and in darkness. Light induced 69 metabolites (MA), related to cotyledon greening; additional 89 metabolites (MB), related to cotyledon opening, were identified by WGCNA. The screening of metabolites shared in both MA and MB obtained 37 specific metabolites (MC) related to both opening and greening. In this context, main changes in MC occurred during A3, the stage in which cotyledons fully opened and greened. Within MC, few sugars, including L-(-)-sorbose, mannose, glucose and its derivatives, markedly decreased, while other sugars, amino acids, and unsaturated fatty acids increased. Most isoflavones and flavonols including ononin, caycosin-7-glucosides, quercetin, genistein, and catechin increased 5.3, 5.5, 13.4, 6.4 and 1.8 times, respectively. Thus, accumulated flavonoids play an important role during this developmental stage. Š 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group
- âŚ