Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers

Efficient time series forecasting has become critical for real-world applications, particularly with deep neural networks (DNNs). Efficiency in DNNs can be achieved through sparse connectivity and reducing the model size. However, finding the sparsity level automatically during training remains a challenging task due to the heterogeneity in the loss-sparsity tradeoffs across the datasets. In this paper, we propose \enquote{\textbf{P}runing with \textbf{A}daptive \textbf{S}parsity \textbf{L}evel} (\textbf{PALS}), to automatically seek an optimal balance between loss and sparsity, all without the need for a predefined sparsity level. PALS draws inspiration from both sparse training and during-training methods. It introduces the novel "expand" mechanism in training sparse neural networks, allowing the model to dynamically shrink, expand, or remain stable to find a proper sparsity level. In this paper, we focus on achieving efficiency in transformers known for their excellent time series forecasting performance but high computational cost. Nevertheless, PALS can be applied directly to any DNN. In the scope of these arguments, we demonstrate its effectiveness also on the DLinear model. Experimental results on six benchmark datasets and five state-of-the-art transformer variants show that PALS substantially reduces model size while maintaining comparable performance to the dense model. More interestingly, PALS even outperforms the dense model, in 12 and 14 cases out of 30 cases in terms of MSE and MAE loss, respectively, while reducing 65% parameter count and 63% FLOPs on average. Our code will be publicly available upon acceptance of the paper

Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers

Efficient time series forecasting has become critical for real-world applications, particularly with deep neural networks (DNNs). Efficiency in DNNs can be achieved through sparse connectivity and reducing the model size. However, finding the sparsity level automatically during training remains a challenging task due to the heterogeneity in the loss-sparsity tradeoffs across the datasets. In this paper, we propose \enquote{\textbf{P}runing with \textbf{A}daptive \textbf{S}parsity \textbf{L}evel} (\textbf{PALS}), to automatically seek an optimal balance between loss and sparsity, all without the need for a predefined sparsity level. PALS draws inspiration from both sparse training and during-training methods. It introduces the novel "expand" mechanism in training sparse neural networks, allowing the model to dynamically shrink, expand, or remain stable to find a proper sparsity level. In this paper, we focus on achieving efficiency in transformers known for their excellent time series forecasting performance but high computational cost. Nevertheless, PALS can be applied directly to any DNN. In the scope of these arguments, we demonstrate its effectiveness also on the DLinear model. Experimental results on six benchmark datasets and five state-of-the-art transformer variants show that PALS substantially reduces model size while maintaining comparable performance to the dense model. More interestingly, PALS even outperforms the dense model, in 12 and 14 cases out of 30 cases in terms of MSE and MAE loss, respectively, while reducing 65% parameter count and 63% FLOPs on average. Our code will be publicly available upon acceptance of the paper

Flux, Impact, and Fate of Halogenated Xenobiotic Compounds in the Gut

Humans and their associated microbiomes are exposed to numerous xenobiotics through drugs, dietary components, personal care products as well as environmental chemicals. Most of the reciprocal interactions between the microbiota and xenobiotics, such as halogenated compounds, occur within the human gut harboring diverse and dense microbial communities. Here, we provide an overview of the flux of halogenated compounds in the environment, and diverse exposure routes of human microbiota to these compounds. Subsequently, we review the impact of halogenated compounds in perturbing the structure and function of gut microbiota and host cells. In turn, cultivation-dependent and metagenomic surveys of dehalogenating genes revealed the potential of the gut microbiota to chemically alter halogenated xenobiotics and impact their fate. Finally, we provide an outlook for future research to draw attention and attract interest to study the bidirectional impact of halogenated and other xenobiotic compounds and the gut microbiota.Peer reviewe

Inter-species Metabolic Interactions in an In-vitro Minimal Human Gut Microbiome of Core Bacteria

Knowledge of the functional roles and interspecies interactions are crucial for improving our understanding of the human intestinal microbiome in health and disease. However, the complexity of the human intestinal microbiome and technical challenges in investigating it pose major challenges. In this proof-of-concept study, we rationally designed, assembled and experimentally tested a synthetic Diet-based Minimal Microbiome (Db-MM) consisting of ten core intestinal bacterial species that together are capable of efficiently converting dietary fibres into short chain fatty acids (SCFAs). Despite their genomic potential for metabolic competition, all ten bacteria coexisted during growth on a mixture of dietary fibres, including pectin, inulin, xylan, cellobiose and starch. By integrated analyses of metabolite production, community composition and metatranscriptomics-based gene expression data, we identified interspecies metabolic interactions leading to production of key SCFAs such as butyrate and propionate. While public goods, such as sugars liberated from colonic fibres, are harvested by non-degraders, some species thrive by cross-feeding on energetically challenging substrates, including the butyrogenic conversion of acetate and lactate. Using a reductionist approach in an in-vitro system combined with functional measurements, our study provides key insights into the complex interspecies metabolic interactions between core intestinal bacterial species.Peer reviewe

Трендсеттінг як ключовий фактор управління інноваційними ризиками індустрії моди

Індустрія моди нового тисячоріччя перетворилася в багатомільйонний сектор економіки, у котрому інноваційна діяльність грає ключову роль. Інновації в дизайні сучасного костюма з інструмента вдосконалювання характеристик об’єкта перетворюються в одну з основних його характеристик, тому фешн-проекти є інноваційними за своєю природою [2]