Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models

Recently Large Language Models (LLMs) have demonstrated their amazing text understanding and generation capabilities. However, even stronger LLMs may still learn incorrect knowledge from the training corpus, as well as some knowledge that is outdated over time. Direct secondary fine-tuning with data containing new knowledge may be ineffective in updating knowledge due to the conflict between old and new knowledge. In this paper, we propose a new paradigm for fine-tuning called F-Learning (Forgetting before Learning), which is based on parametric arithmetic to achieve forgetting of old knowledge and learning of new knowledge. Experimental results on two publicly available datasets demonstrate that our proposed F-Learning can obviously improve the knowledge updating performance of both full fine-tuning and LoRA fine-tuning. Moreover, we have also discovered that forgetting old knowledge by subtracting the parameters of LoRA can achieve a similar effect to subtracting the parameters of full fine-tuning, and sometimes even surpass it significantly.Comment: 8 pages, 2 figures, 2 table

Supplemental methionine selenium effects on egg yolk physicochemical, functional, and protein structure during storage

To clarify the effect of the addition of methionine selenium on the physicochemical, functional, and protein structural properties of egg yolk during storage. We analyzed the changes in the main indicators of egg yolks stored at 4°C and 25°C for 28 days. The results showed that the increase in water content and pH, and the decrease in absolute zeta potential and apparent viscosity of the selenium-rich egg yolks (Se-group) during storage were smaller than those of the control group egg yolks (C-group). In addition, the antioxidant capacity and emulsifying ability of the Se-group during storage were better than those of the C-group. Simultaneously, the hardness and chewiness of the Se-group gel during storage were lower than those of the C-group. The protein structure results showed that selenium rich treatment did not affect the secondary structure of egg yolk protein during storage but could improve the fluorescence intensity of the egg yolk protein. Therefore, the addition of methionine selenium can reduce the degree of deterioration in the physicochemical properties of egg yolk during storage and extend its shelf life

Robusno upravljanje višerobotskim formacijama korištenjem klizećeg regulatora i neizrazitog kompenzatora

To form up a multiple-robot system, a robust adaptive control scheme is addressed. The control scheme is based on the methodology of sliding mode control (SMC). The formation system is leader-follower-based, whose dynamics are subject to uncertainties. A fuzzy compensator is adopted to approximate the uncertainties. To attenuate the approximation error, a robust adaptive law of the fuzzy compensator is introduced. In the sense of Lyapunov, not only such a control scheme can asymptotically stabilize the whole formation system, but also the convergence of the approximation error can be guaranteed. Compared with the sole sliding mode controller without compensator, some numerical simulations verify the feasibility and effectiveness of the control scheme for the multiple-robot system in the presence of uncertainties.Kako bi se formirao višerobotski sustav korištena je robusna adaptivna shema upravljanja. Upravljačka shema je bazirana na metodologiji upravljanja klizećim režimom (SMC). Formacijski sustav baziran je na vođa-sljedbenik metodi čija je dinamika podložna nesigurnostima. Za aproksimiranje nesigurnosti korišten je neizraziti kompenzator. Kako bi se prigušila aproksimacijska greška razvijen je robusni adaptivni upravljački zakon. Korištenjem takvog upravljačkog zakona ostvarena je stabilnost prema Lyapunovu, te je moguće garantirati konvergenciju aproksimacijske greške. U usporedbi s regulatorom zasnovanim na klizećem režimu bez kompenzatora, neke numeričke simulacije potvrđuju izvedivost i efikasnost ovakve sheme upravljanja višerobotskim sustavom uz prisutnost nesigurnosti

DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices

Recent years have witnessed the great success of vision transformer (ViT), which has achieved state-of-the-art performance on multiple computer vision benchmarks. However, ViT models suffer from vast amounts of parameters and high computation cost, leading to difficult deployment on resource-constrained edge devices. Existing solutions mostly compress ViT models to a compact model but still cannot achieve real-time inference. To tackle this issue, we propose to explore the divisibility of transformer structure, and decompose the large ViT into multiple small models for collaborative inference at edge devices. Our objective is to achieve fast and energy-efficient collaborative inference while maintaining comparable accuracy compared with large ViTs. To this end, we first propose a collaborative inference framework termed DeViT to facilitate edge deployment by decomposing large ViTs. Subsequently, we design a decomposition-and-ensemble algorithm based on knowledge distillation, termed DEKD, to fuse multiple small decomposed models while dramatically reducing communication overheads, and handle heterogeneous models by developing a feature matching module to promote the imitations of decomposed models from the large ViT. Extensive experiments for three representative ViT backbones on four widely-used datasets demonstrate our method achieves efficient collaborative inference for ViTs and outperforms existing lightweight ViTs, striking a good trade-off between efficiency and accuracy. For example, our DeViTs improves end-to-end latency by 2.89$\times$ with only 1.65% accuracy sacrifice using CIFAR-100 compared to the large ViT, ViT-L/16, on the GPU server. DeDeiTs surpasses the recent efficient ViT, MobileViT-S, by 3.54% in accuracy on ImageNet-1K, while running 1.72$\times$ faster and requiring 55.28% lower energy consumption on the edge device.Comment: Accepted by IEEE Transactions on Mobile Computin

Mibefradil reduces blood glucose concentration in db/db mice

OBJECTIVE: Numerous recent studies suggest that abnormal intracellular calcium concentration ([Ca2+]i) is a common defect in diabetic animal models and patients. Abnormal calcium handling is an important mechanism in the defective pancreatic β-cell function in type 2 diabetes. T-type Ca2+ channel antagonists lower blood glucose in type 2 diabetes, but the mechanism remains unknown. METHODS: We examined the effect of the Ca2+ channel antagonist mibefradil on blood glucose in male db/db mice and phenotypically normal heterozygous mice by intraperitoneal injection. RESULTS: Mibefradil (15 mg/kg, i.p., b.i.d.) caused a profound reduction of fasting blood glucose from 430.92±20.46 mg/dl to 285.20±5.74 mg/dl in three days. The hypoglycemic effect of mibefradil was reproduced by NNC 55-0396, a compound structurally similar to mibefradil but more selective for T-type Ca2+ channels, but not by the specific L-type Ca2+ channel blocker nicardipine. Mibefradil did not show such hypoglycemic effects in heterozygous animals. In addition, triglycerides, basal insulin and food intake were significantly decreased by mibefradil treatment in the db/db mice but not in the controls. Western blot analysis, immunohistochemistry and immunofluorescence staining showed a significantly increased expression of T-type Ca2+ channel α-subunits Cav3.1 and Cav3.2 in liver and brain tissues from db/db mice compared to those from heterozygous animals. CONCLUSIONS: Collectively, these results suggest that T-type Ca2+ channels are potential therapeutic targets for antidiabetic drugs

Pericytes display increased CCN2 expression upon culturing

By providing a source of α-smooth muscle actin (α-SMA)-expressing myofibroblasts, microvascular pericytes contribute to the matrix remodeling that occurs during tissue repair. However, the extent to which pericytes may contribute to the fibroblast phenotype post-repair is unknown. In this report, we test whether pericytes isolated from human placenta can in principle become fibroblast-like. Pericytes were cultured in vitro for 11 passages. The Affymetrix mRNA expression profile of passage 2 and passage 11 pericytes was compared. The expression of type I collagen, thrombospondin and fibronectin mRNAs was induced by passaging pericytes in culture. This induction of a fibroblast phenotype was paralleled by induction of connective tissue growth factor (CTGF/CCN2) and type I collagen protein expression and the fibroblast marker ASO2. These results indicate that, in principle, pericytes have the capacity to become fibroblast-like and that pericytes may contribute to the population of fibroblasts in a healed wound

Backpropagation Path Search On Adversarial Transferability

Deep neural networks are vulnerable to adversarial examples, dictating the imperativeness to test the model's robustness before deployment. Transfer-based attackers craft adversarial examples against surrogate models and transfer them to victim models deployed in the black-box situation. To enhance the adversarial transferability, structure-based attackers adjust the backpropagation path to avoid the attack from overfitting the surrogate model. However, existing structure-based attackers fail to explore the convolution module in CNNs and modify the backpropagation graph heuristically, leading to limited effectiveness. In this paper, we propose backPropagation pAth Search (PAS), solving the aforementioned two problems. We first propose SkipConv to adjust the backpropagation path of convolution by structural reparameterization. To overcome the drawback of heuristically designed backpropagation paths, we further construct a DAG-based search space, utilize one-step approximation for path evaluation and employ Bayesian Optimization to search for the optimal path. We conduct comprehensive experiments in a wide range of transfer settings, showing that PAS improves the attack success rate by a huge margin for both normally trained and defense models.Comment: Accepted by ICCV202

Sparks of GPTs in Edge Intelligence for Metaverse: Caching and Inference for Mobile AIGC Services

Aiming at achieving artificial general intelligence (AGI) for Metaverse, pretrained foundation models (PFMs), e.g., generative pretrained transformers (GPTs), can effectively provide various AI services, such as autonomous driving, digital twins, and AI-generated content (AIGC) for extended reality. With the advantages of low latency and privacy-preserving, serving PFMs of mobile AI services in edge intelligence is a viable solution for caching and executing PFMs on edge servers with limited computing resources and GPU memory. However, PFMs typically consist of billions of parameters that are computation and memory-intensive for edge servers during loading and execution. In this article, we investigate edge PFM serving problems for mobile AIGC services of Metaverse. First, we introduce the fundamentals of PFMs and discuss their characteristic fine-tuning and inference methods in edge intelligence. Then, we propose a novel framework of joint model caching and inference for managing models and allocating resources to satisfy users' requests efficiently. Furthermore, considering the in-context learning ability of PFMs, we propose a new metric to evaluate the freshness and relevance between examples in demonstrations and executing tasks, namely the Age of Context (AoC). Finally, we propose a least context algorithm for managing cached models at edge servers by balancing the tradeoff among latency, energy consumption, and accuracy