Learning Pruned Structure and Weights Simultaneously from Scratch: an Attention based Approach

As a deep learning model typically contains millions of trainable weights, there has been a growing demand for a more efficient network structure with reduced storage space and improved run-time efficiency. Pruning is one of the most popular network compression techniques. In this paper, we propose a novel unstructured pruning pipeline, Attention-based Simultaneous sparse structure and Weight Learning (ASWL). Unlike traditional channel-wise or weight-wise attention mechanism, ASWL proposed an efficient algorithm to calculate the pruning ratio through layer-wise attention for each layer, and both weights for the dense network and the sparse network are tracked so that the pruned structure is simultaneously learned from randomly initialized weights. Our experiments on MNIST, Cifar10, and ImageNet show that ASWL achieves superior pruning results in terms of accuracy, pruning ratio and operating efficiency when compared with state-of-the-art network pruning methods

Modality-Agnostic Learning for Medical Image Segmentation Using Multi-modality Self-distillation

Medical image segmentation of tumors and organs at risk is a time-consuming yet critical process in the clinic that utilizes multi-modality imaging (e.g, different acquisitions, data types, and sequences) to increase segmentation precision. In this paper, we propose a novel framework, Modality-Agnostic learning through Multi-modality Self-dist-illation (MAG-MS), to investigate the impact of input modalities on medical image segmentation. MAG-MS distills knowledge from the fusion of multiple modalities and applies it to enhance representation learning for individual modalities. Thus, it provides a versatile and efficient approach to handle limited modalities during testing. Our extensive experiments on benchmark datasets demonstrate the high efficiency of MAG-MS and its superior segmentation performance than current state-of-the-art methods. Furthermore, using MAG-MS, we provide valuable insight and guidance on selecting input modalities for medical image segmentation tasks

DNA methylation and regulatory elements during chicken germline stem cell differentiation

Funding for Open Access provided by the UMD Libraries' Open Access Publishing Fund.The production of germ cells in vitro would open important new avenues for stem biology and human medicine, but the mechanisms of germ cell differentiation are not well understood. The chicken, as a great model for embryology and development, was used in this study to help us explore its regulatory mechanisms. In this study, we reported a comprehensive genome-wide DNA methylation landscape in chicken germ cells, and transcriptomic dynamics was also presented. By uncovering DNA methylation patterns on individual genes, some genes accurately modulated by DNA methylation were found to be associated with cancers and virus infection, e.g., AKT1 and CTNNB1. Chicken-unique markers were also discovered for identifying male germ cells. Importantly, integrated epigenetic mechanisms were explored during male germ cell differentiation, which provides deep insight into the epigenetic processes associated with male germ cell differentiation and possibly improves treatment options to male infertility in animals and humans

Nonlinear magnetotransport shaped by Fermi surface topology and convexity in WTe2

The nature of Fermi surface defines the physical properties of conductors and many physical phenomena can be traced to its shape. Although the recent discovery of a current-dependent nonlinear magnetoresistance in spin-polarized non-magnetic materials has attracted considerable attention in spintronics, correlations between this phenomenon and the underlying fermiology remain unexplored. Here, we report the observation of nonlinear magnetoresistance at room temperature in a semimetal WTe2, with an interesting temperature-driven inversion. Theoretical calculations reproduce the nonlinear transport measurements and allow us to attribute the inversion to temperature-induced changes in Fermi surface convexity. We also report a large anisotropy of nonlinear magnetoresistance in WTe2, due to its low symmetry of Fermi surfaces. The good agreement between experiments and theoretical modeling reveals the critical role of Fermi surface topology and convexity on the nonlinear magneto-response. These results lay a new path to explore ramifications of distinct fermiology for nonlinear transport in condensed-matter

Low-Complexity Acoustic Scene Classification Using Data Augmentation and Lightweight ResNet

We present a work on low-complexity acoustic scene classification (ASC) with multiple devices, namely the subtask A of Task 1 of the DCASE2021 challenge. This subtask focuses on classifying audio samples of multiple devices with a low-complexity model, where two main difficulties need to be overcome. First, the audio samples are recorded by different devices, and there is mismatch of recording devices in audio samples. We reduce the negative impact of the mismatch of recording devices by using some effective strategies, including data augmentation (e.g., mix-up, spectrum correction, pitch shift), usages of multi-patch network structure and channel attention. Second, the model size should be smaller than a threshold (e.g., 128 KB required by the DCASE2021 challenge). To meet this condition, we adopt a ResNet with both depthwise separable convolution and channel attention as the backbone network, and perform model compression. In summary, we propose a low-complexity ASC method using data augmentation and a lightweight ResNet. Evaluated on the official development and evaluation datasets, our method obtains classification accuracy scores of 71.6% and 66.7%, respectively; and obtains Log-loss scores of 1.038 and 1.136, respectively. Our final model size is 110.3 KB which is smaller than the maximum of 128 KB.Comment: 5 pages, 5 figures, 4 tables. Accepted for publication in the 16th IEEE International Conference on Signal Processing (IEEE ICSP

InstructCoder: Empowering Language Models for Code Editing

Code editing encompasses a variety of pragmatic tasks that developers deal with daily. Despite its relevance and practical usefulness, automatic code editing remains an underexplored area in the evolution of deep learning models, partly due to data scarcity. In this work, we explore the use of large language models (LLMs) to edit code based on user instructions, covering a broad range of implicit tasks such as comment insertion, code optimization, and code refactoring. To facilitate this, we introduce InstructCoder, the first dataset designed to adapt LLMs for general-purpose code editing, containing highdiversity code-editing tasks. It consists of over 114,000 instruction-input-output triplets and covers multiple distinct code editing scenarios. The dataset is systematically expanded through an iterative process that commences with code editing data sourced from GitHub commits as seed tasks. Seed and generated tasks are used subsequently to prompt ChatGPT for more task data. Our experiments demonstrate that open-source LLMs fine-tuned on InstructCoder can edit code correctly based on users' instructions most of the time, exhibiting unprecedented code-editing performance levels. Such results suggest that proficient instruction-finetuning can lead to significant amelioration in code editing abilities. The dataset and the source code are available at https://github.com/qishenghu/CodeInstruct

Top-down effects of filter-feeding fish and bivalves moderate bottom-up effects of nutrients on phytoplankton in subtropical shallow lakes: An outdoor mesocosm study

Biomanipulation has been widely used in the ecological restoration of eutrophic lakes for decades. However, biomanipulation is prone to failure if external nutrient loads are not reduced. In order to explore the importance of filter-feeding fish and bivalves on algal control, an outdoor mesocosm experiment was conducted using different nutrient concentrations. Four treatments simulating daily loads of nutrients in Lake Taihu were studied: current, two times, and three times average daily loads of nutrients with both fish (Aristichthys nobilis) and Asian clam (Corbicula fluminea) and as a control current daily loads without fish or bivalves. Results showed that stocking of filter-feeding fish and bivalves (80 g m-3 bighead carp; 200 g cm-2 clams) at two times daily nutrient loads could effectively control water column Chl a concentrations and phytoplankton biomass. At higher nutrient concentrations (TN & GE; 260 & mu;g L-1 d-1; TP & GE; 10 & mu;g L-1 d-1), top-down control of filter-feeding fish and bivalves was less effective and bottom-up effects resulted in significant increases of Chl a concentration. Thus, as phytoplankton biomass in freshwater ecosystems is determined by both the top-down effects of predators and the bottom-up effects of nutrients, external loadings should be controlled when filter-feeding fish and bivalves are used for algal control to ensure the efficacy of biomanipulation.A combination of filter-feeding fish and clams suppressed phytoplankton, which could not be affected by low-level nutrients.Bottom-up effects at high-level nutrients on phytoplankton overcome top-down effects, indicating that nutrient levels should be controlled to optimize the effect of the intervention.imag