One-Shot Pruning for Fast-adapting Pre-trained Models on Devices

Large-scale pre-trained models have been remarkably successful in resolving downstream tasks. Nonetheless, deploying these models on low-capability devices still requires an effective approach, such as model pruning. However, pruning the model from scratch can pose a practical challenge given the limited resources of each downstream task or device. To tackle this issue, we present a scalable one-shot pruning method that leverages pruned knowledge of similar tasks to extract a sub-network from the pre-trained model for a new task. Specifically, we create a score mask using the pruned models of similar tasks to identify task-specific filters/nodes in the pre-trained model for the new task. Based on this mask, we conduct a single round of pruning to extract a suitably-sized sub-network that can quickly adapt to the new task with only a few training iterations. Our experimental analysis demonstrates the effectiveness of the proposed method on the convolutional neural networks (CNNs) and vision transformers (ViT) with various datasets. The proposed method consistently outperforms popular pruning baseline methods in terms of accuracy and efficiency when dealing with diverse downstream tasks with different memory constraints

Forecasting tourism demand with an improved mixed data sampling model

Search query data reflect users’ intentions, preferences and interests. The interest in using such data to forecast tourism demand has increased in recent years. The mixed data sampling (MIDAS) method is often used in such forecasting, but is not effective when moving average (MA) dynamics are involved. To investigate the relevance of the MA components in MIDAS models to tourism demand forecasting, an improved MIDAS model that integrates MIDAS and the seasonal autoregressive integrated moving average process is proposed. Its performance is tested by forecasting monthly tourist arrivals in Hong Kong from mainland China with daily composite indices constructed from a large number of search queries using the generalised dynamic factor model. The forecasting results suggest that this new model significantly outperforms the benchmark model. In addition, comparing the forecasts and nowcasts shows that the latter generally outperform the former

Trajectory Scheduling Methods for minimizing total tardiness in a flowshop

AbstractIn this paper, Trajectory Scheduling Methods (TSMs) are proposed for the permutation flowshop scheduling problem with total tardiness minimization criterion. TSMs belong to an iterative local search framework, in which local search is performed on an initial solution, a perturbation operator is deployed to improve diversification, and a restart point mechanism is used to select the new start point of another cycle. In terms of the insertion and swap neighborhood structures, six composite heuristics are introduced, which exploit the search space with a strong intensification effect. Based on purely insertion-based or swap-based perturbation structures, three compound perturbation structures are developed that construct a candidate restart point set rather than just a single restart point. The distance between the current best solution and each start point of the set is defined, according to which the diversification effect of TSMs can be boosted by choosing the most appropriate restart point for the next iteration. A total of 18 trajectory scheduling methods are constructed by different combinations of composite heuristics. Both the best and worst combinations are compared with three best existing sequential meta-heuristics for the considered problem on 540 benchmark instances. Experimental results show that the proposed heuristics significantly outperform the three best existing algorithms within the same computation time

Scenario Forecasting for Global Tourism

This study provides innovative forecasts of the probabilities of certain scenarios of tourism demand. The scenarios of interest are constructed in relation to tourism growth and economic growth. The probability forecasts based on these scenarios provide valuable information for destination policy makers. The time-varying parameter panel vector autoregressive (TVP-PVAR) model is adopted for scenario forecasting. Both the accuracy rate and the Brier score are used to evaluate the forecasting performance. A global set of 25 tourism destinations is empirically examined, and the results confirm that the TVP-PVAR model with a time-varying error covariance matrix is generally a promising tool for forecasting. Our study contributes to tourism forecasting literature in advocating the use of scenario forecasting to facilitate industry decision making in situations wherein forecasts are defined by two or more dimensions simultaneously. In addition, it is the first study to introduce the TVP-PVAR model to tourism demand forecasting

Does Continual Learning Equally Forget All Parameters?

Distribution shift (e.g., task or domain shift) in continual learning (CL) usually results in catastrophic forgetting of neural networks. Although it can be alleviated by repeatedly replaying buffered data, the every-step replay is time-consuming. In this paper, we study which modules in neural networks are more prone to forgetting by investigating their training dynamics during CL. Our proposed metrics show that only a few modules are more task-specific and sensitively alter between tasks, while others can be shared across tasks as common knowledge. Hence, we attribute forgetting mainly to the former and find that finetuning them only on a small buffer at the end of any CL method can bring non-trivial improvement. Due to the small number of finetuned parameters, such ``Forgetting Prioritized Finetuning (FPF)'' is efficient in computation. We further propose a more efficient and simpler method that entirely removes the every-step replay and replaces them by only $k$-times of FPF periodically triggered during CL. Surprisingly, this ``$k$-FPF'' performs comparably to FPF and outperforms the SOTA CL methods but significantly reduces their computational overhead and cost. In experiments on several benchmarks of class- and domain-incremental CL, FPF consistently improves existing CL methods by a large margin, and $k$-FPF further excels in efficiency without degrading the accuracy. We also empirically studied the impact of buffer size, epochs per task, and finetuning modules on the cost and accuracy of our methods

The co-transfer of plasmid-borne colistin-resistant genes mcr-1 and mcr-3.5, the carbapenemase gene blaNDM-5 and the 16S methylase gene rmtB from Escherichia coli

We found an unusual Escherichia coli strain with resistance to colistin, carbapenem and amikacin from sewage. We therefore characterized the strain and determined the co-transfer of the resistance determinants. Whole genome sequencing was performed using both Illumina HiSeq X10 and MinION sequencers. Short and long reads were subjected to de novo hybrid assembly. Sequence type, antimicrobial resistance genes and plasmid replicons were identified from the genome sequences. Phylogenetic analysis of all IncHI2 plasmids carrying mcr-1 available in GenBank was performed based on core genes. Conjugation experiments were performed. mcr-3.5 was cloned into E. coli DH5α. The strain belonged to ST410, a type with a global distribution. Two colistin-resistant genes, mcr-1.1 and mcr-3.5, a carbapenemase gene blaNDM-5, and a 16S methylase gene rmtB were identified on different plasmids of IncHI2(ST3)/IncN, IncP, IncX3 and IncFII, respectively. All of the four plasmids were self-transmissible and mcr-1.1, mcr-3.5, blaNDM-5 and rmtB were transferred together. mcr-1-carrying IncHI2 plasmids belonged to several sequence types with ST3 and ST4 being predominant. MIC of colistin (4 μg/ml) for DH5α containing mcr-3.5 was identical to that containing the original mcr-3 variant. In conclusion, carbapenem resistance, colistin resistance and high-level aminoglycoside resistance can be transferred together even when their encoding genes are not located on the same plasmid. The co-transfer of multiple clinically-important antimicrobial resistance represents a particular challenge for clinical treatment and infection control in healthcare settings. Isolates with resistance to both carbapenem and colistin are not restricted to a given sequence type but rather are diverse in clonal background, which warrants further surveillance. The amino acid substitutions of MCR-3.5 have not altered its activity against colistin.</p

Promoting Shewanella Bidirectional Extracellular Electron Transfer for Bioelectrocatalysis by Electropolymerized Riboflavin Interface on Carbon Electrode

The extracellular electron transfer (EET) that connects the intracellular metabolism of electroactive microorganisms to external electron donors/acceptors, is the foundation to develop diverse microbial electrochemical technologies. For a particular microbial electrochemical device, the surface chemical property of an employed electrode material plays a crucial role in the EET process owing to the direct and intimate biotic-abiotic interaction. The functional modification of an electrode surface with redox mediators has been proposed as an effectual approach to promote EET, but the underlying mechanism remains unclear. In this work, we investigated the enhancement of electrochemically polymerized riboflavin interface on the bidirectional EET of Shewanella putrefaciens CN32 for boosting bioelectrocatalytic ability. An optimal polyriboflavin functionalized carbon cloth electrode achieved about 4.3-fold output power density (∼707 mW/m2) in microbial fuel cells and 3.7-fold cathodic current density (∼0.78 A/m2) for fumarate reduction in three-electrode cells compared to the control, showing great increases in both outward and inward EET rates. Likewise, the improvement was observed for polyriboflavin-functionalized graphene electrodes. Through comparison between wild-type strain and outer-membrane cytochrome (MtrC/UndA) mutant, the significant improvements were suggested to be attributed to the fast interfacial electron exchange between the polyriboflavin interface with flexible electrochemical activity and good biocompatibility and the outer-membrane cytochromes of the Shewanella strain. This work not only provides an effective approach to boost microbial electrocatalysis for energy conversion, but also offers a new demonstration of broadening the applications of riboflavin-functionalized interface since the widespread contribution of riboflavin in various microbial EET pathways together with the facile electropolymerization approach

Mesenchymal Stem Cell in the Intervertebral Disc

Degeneration of the intervertebral disc (IVD) is a major spinal disorder that causes back pain. Nucleus pulposus (NP) in the central of IVD dehydrates and become more fibrous in the IVD degeneration. NP cells undergo apoptosis with the degeneration of extracellular matrix (ECM) components. To replenish the NP cells and core ECM, bone marrow mesenchymal stromal cells (BMSCs) have been highlighted in the regeneration of IVD degeneration. BMSCs differentiate into NP-like cells with the secretion of ECM components, which may not only replenish the number of NP cells but also stimulate NP reconstruction. This further maintains tissue homeostasis. Up to date, the disc progenitor cells (DPCs) have been identified with the characteristics of multidifferentiation and stem cell phenotype. These cells are involved in the IVD diseases and show regenerative potentials. However, the differences between the BMSCs and DPCs remain elusive, in particular, the cellular connection in vivo. As such, this chapter will discuss the findings of the two cell types and propose a novel concept in the understanding of the biology of IVD

ST273 carbapenem-resistant <i>Klebsiella pneumoniae</i> carrying <i>bla</i>_NDM-1 and <i>bla</i>_IMP-4

ABSTRACT A carbapenem-resistant Klebsiella pneumoniae isolate was recovered from human blood. Its whole-genome sequence was obtained using Illumina and long-read MinION sequencing. The strain belongs to sequence type 273 (ST273), which was found recently and caused an outbreak in Southeast Asia. It has two carbapenemase genes, bla NDM-1 (carried by an ST7 IncN self-transmissible plasmid) and bla IMP-4 (located on a self-transmissible IncHI5 plasmid). Non-KPC-producing ST237 may represent a lineage of carbapenem-resistant K. pneumoniae , which warrants further monitoring. </jats:p