Rethinking Learning Rate Tuning in the Era of Large Language Models

Large Language Models (LLMs) represent the recent success of deep learning in achieving remarkable human-like predictive performance. It has become a mainstream strategy to leverage fine-tuning to adapt LLMs for various real-world applications due to the prohibitive expenses associated with LLM training. The learning rate is one of the most important hyperparameters in LLM fine-tuning with direct impacts on both fine-tuning efficiency and fine-tuned LLM quality. Existing learning rate policies are primarily designed for training traditional deep neural networks (DNNs), which may not work well for LLM fine-tuning. We reassess the research challenges and opportunities of learning rate tuning in the coming era of Large Language Models. This paper makes three original contributions. First, we revisit existing learning rate policies to analyze the critical challenges of learning rate tuning in the era of LLMs. Second, we present LRBench++ to benchmark learning rate policies and facilitate learning rate tuning for both traditional DNNs and LLMs. Third, our experimental analysis with LRBench++ demonstrates the key differences between LLM fine-tuning and traditional DNN training and validates our analysis

Resilient Data Collection in Refinery Sensor Networks Under Large Scale Failures

Wireless sensors and measurement devices are widely deployed in oil and gas refineries to monitor the health of the pipes. These sensors are deployed along the pipes in an open area and thus are subject to large scale failures due to cyber-physical attacks and hazardous environments. In this paper, we study the resilience issues in collecting data from a dense and large scale set of sensors deployed over the physical refinery pipe network. We construct a multi-tree sensor mesh network over the refinery sensors for data collection. The reporting messages within one of the trees, while passing along the tree, are protected by a secret key shared among all sensors on the tree. Our construction aims to minimize the data collection time and ensures that the information leakage probability of the secret key is bounded. To tolerate large scale failures, we present a distributed self-healing protocol, which enables a tree node to discover a secondary path when its parent fails. The simulation result shows that the self-healing protocol tolerates large scale failures with high probability and has small overhead in data collection time.Ope

Nanotechnology in cervical cancer immunotherapy: Therapeutic vaccines and adoptive cell therapy

Immunotherapy is an emerging method for the treatment of cervical cancer and is more effective than surgery and radiotherapy, especially for recurrent cervical cancer. However, immunotherapy is limited by adverse effects in clinical practice. In recent years, nanotechnology has been widely used for tumor diagnosis, drug delivery, and targeted therapy. In the setting of cervical cancer, nanotechnology can be used to actively or passively target immunotherapeutic agents to tumor sites, thereby enhancing local drug delivery, reducing drug adverse effects, achieving immunomodulation, improving the tumor immune microenvironment, and optimizing treatment efficacy. In this review, we highlight the current status of therapeutic vaccines and adoptive cell therapy in cervical cancer immunotherapy, as well as the application of lipid carriers, polymeric nanoparticles, inorganic nanoparticles, and exosomes in this context

TikTalk: A Video-Based Dialogue Dataset for Multi-Modal Chitchat in Real World

To facilitate the research on intelligent and human-like chatbots with multi-modal context, we introduce a new video-based multi-modal dialogue dataset, called TikTalk. We collect 38K videos from a popular video-sharing platform, along with 367K conversations posted by users beneath them. Users engage in spontaneous conversations based on their multi-modal experiences from watching videos, which helps recreate real-world chitchat context. Compared to previous multi-modal dialogue datasets, the richer context types in TikTalk lead to more diverse conversations, but also increase the difficulty in capturing human interests from intricate multi-modal information to generate personalized responses. Moreover, external knowledge is more frequently evoked in our dataset. These facts reveal new challenges for multi-modal dialogue models. We quantitatively demonstrate the characteristics of TikTalk, propose a video-based multi-modal chitchat task, and evaluate several dialogue baselines. Experimental results indicate that the models incorporating large language models (LLM) can generate more diverse responses, while the model utilizing knowledge graphs to introduce external knowledge performs the best overall. Furthermore, no existing model can solve all the above challenges well. There is still a large room for future improvements, even for LLM with visual extensions. Our dataset is available at \url{https://ruc-aimind.github.io/projects/TikTalk/}.Comment: Accepted to ACM Multimedia 202

Nutritional Risk, Health Outcomes, and Hospital Costs Among Chinese Immobile Older Inpatients: A National Study

Purpose: Evidence of the impact of nutritional risk on health outcomes and hospital costs among Chinese older inpatients is limited. Relatively few studies have investigated the association between clinical and cost outcomes and nutritional risk in immobile older inpatients, particularly those with neoplasms, injury, digestive, cardiac, and respiratory conditions. Methods: This China-wide prospective observational cohort study comprised 5,386 immobile older inpatients hospitalized at 25 hospitals. All patients were screened for nutritional risk using the Nutrition Risk Screening (NRS 2002). A descriptive analysis of baseline variables was followed by multivariate analysis (Cox proportional hazards models and generalized linear model) to compare the health and economic outcomes, namely, mortality, length of hospital stay (LoS), and hospital costs associated with a positive NRS 2002 result. Results: The prevalence of a positive NRS 2002 result was 65.3% (n = 3,517). The prevalence of “at-risk” patients (NRS 2002 scores of 3+) was highest in patients with cardiac conditions (31.5%) and lowest in patients with diseases of the respiratory system (6.9%). Controlling for sex, age, education, type of insurance, smoking status, the main diagnosed disease, and Charlson comorbidity index (CCI), the multivariate analysis showed that the NRS 2002 score = 3 [hazard ratio (HR): 1.376, 95% CI: 1.031–1.836] were associated with approximately a 1.5-fold higher likelihood of death. NRS 2002 scores = 4 (HR: 1.982, 95% CI: 1.491–2.633) and NRS scores ≥ 5 (HR: 1.982, 95% CI: 1.498–2.622) were associated with a 2-fold higher likelihood of death, compared with NRS 2002 scores <3. An NRS 2002 score of 3 (percentage change: 16.4, 95% CI: 9.6–23.6), score of 4 (32.4, 95% CI: 24–41.4), and scores of ≥ 5 (36.8, 95% CI 28.3–45.8) were associated with a significantly (16.4, 32.4, and 36.8%, respectively) higher likelihood of increased LoS compared with an NRS 2002 scores <3. The NRS 2002 score = 3 group (17.8, 95% CI: 8.6–27.7) was associated with a 17.8%, the NRS 2002 score = 4 group (31.1, 95% CI: 19.8–43.5) a 31.1%, and the NRS 2002 score ≥ 5 group (44.3, 95% CI: 32.3–57.4) a 44.3%, higher likelihood of increased hospital costs compared with a NRS 2002 scores <3 group. Specifically, the most notable mortality-specific comorbidity and LoS-specific comorbidity was injury, while the most notable cost-specific comorbidity was diseases of the digestive system. Conclusions: This study demonstrated the high burden of undernutrition at the time of hospital admission on the health and hospital cost outcomes for older immobile inpatients. These findings underscore the need for nutritional risk screening in all Chinese hospitalized patients, and improved diagnosis, treatment, and nutritional support to improve immobile patient outcomes and to reduce healthcare costs

Development and validation of risk prediction model for identifying 30-day frailty in older inpatients with undernutrition: A multicenter cohort study

ObjectiveTo develop and externally validate a frailty prediction model integrating physical factors, psychological variables and routine laboratory test parameters to predict the 30-day frailty risk in older adults with undernutrition.MethodsBased on an ongoing survey of geriatrics syndrome in elder adults across China (SGSE), this prognostic study identified the putative prognostic indicators for predicting the 30-day frailty risk of older adults with undernutrition. Using multivariable logistic regression analysis with backward elimination, the predictive model was subjected to internal (bootstrap) and external validation, and its calibration was evaluated by the calibration slope and its C statistic discriminative ability. The model derivation and model validation cohorts were collected between October 2018 and February 2019 from a prospective, large-scale cohort study of hospitalized older adults in tertiary hospitals in China. The modeling derivation cohort data (n = 2,194) were based on the SGSE data comprising southwest Sichuan Province, northern Beijing municipality, northwest Qinghai Province, northeast Heilongjiang Province, and eastern Zhejiang Province, with SGSE data from Hubei Province used to externally validate the model (validation cohort, n = 648).ResultsThe incidence of frailty in the older undernutrition derivation cohort was 13.54% and 13.43% in the validation cohort. The final model developed to estimate the individual predicted risk of 30-day frailty was presented as a regression formula: predicted risk of 30-day frailty = [1/(1+e-riskscore )], where riskscore = -0.106 + 0.034 × age + 0.796 × sex -0.361 × vision dysfunction + 0.373 × hearing dysfunction + 0.408 × urination dysfunction - 0.012 × ADL + 0.064 × depression - 0.139 × nutritional status - 0.007 × hemoglobin - 0.034 × serum albumin - 0.012 × (male: ADL). Area under the curve (AUC) of 0.71 in the derivation cohort, and discrimination of the model were similar in both cohorts, with a C statistic of nearly 0.7, with excellent calibration of observed and predicted risks.ConclusionA new prediction model that quantifies the absolute risk of frailty of older patients suffering from undernutrition was developed and externally validated. Based on physical, psychological, and biological variables, the model provides an important assessment tool to provide different healthcare needs at different times for undernutrition frailty patients.Clinical trial registrationChinese Clinical Trial Registry [ChiCTR1800017682]

DEBRIS DETECTION IN HYDRAULIC OIL USING A MICROFLUIDIC INDUCTIVE PULSE DEVICE

ABSTRACT Metal wear debris is an important component of oil particle contamination and is also an essential information carrier in hydraulic oil. Based on inductive Coulter counting principle, a microfluidic device to detect wear debris in oil is presented. The proposed device has the advantage that in theory the distance between wear particle in oil and an embedded coil is 0, which can greatly improve the sensitivity of detection. The relationship between coil parameters and inductive change of a coil is analyzed through the related experimental statistics. The result indicates it can distinguish effectively ferrous and nonferrous metal particles in oil, and the size of 19μm iron particles and 40μm copper particles can be detected

Effects of low-frequency rTMS combined with antidepressants on depression in patients with post-stroke depression: a systematic review and meta-analysis

ObjectiveTo evaluate the effect of low-frequency (≤1 Hz) repetitive transcranial magnetic stimulation (low-frequency rTMS) combined with antidepressants on depression and the levels of inflammatory factors IL-6 and TNF-α in patients with post-stroke depression (PSD).DesignPubMed, Embase, Web of Science, Cochrane Library (CBM), China National Knowledge Infrastructure, Technology Periodical Database, and Wanfang Database were searched until October 2022 for randomized controlled trials.ParticipantsPatients with post-stroke depression (PSD) participated in the study.ResultsA total of 16 randomized controlled trials (RCTs) involving 1,463 patients with PSD were included. According to the Physiotherapy Evidence Database (PEDro) quality assessment, three studies received high quality (eight scores) and 13 RCTs received moderate quality (six scores) results. The meta-analysis showed that low-rTMS combined with an antidepressant significantly reduced the Hamilton Depression Scale (HAMD) score and the National Institutes of Health Stroke Scale (NIHSS) score, reduced IL-6 and TNF-α levels, and improved the MMSE score in PSD compared to an antidepressant alone.ConclusionThe results of this meta-analysis evidenced the efficacy and safety of low-rTMS combined with antidepressants in the treatment of depression in PSD patients. The combined therapy could reduce The depression state and the levels of IL-6 and TNF-α, and enhance the cognitive function of patients. In addition, low-rTMS had fewer adverse effects, proving safety. However, there are shortcomings, such as a lack of long-term follow-up, different intervention sites of low-rTMS, and different intervention frequencies (0.5 or 1 Hz). Thus, in the future, RCTs with a larger sample size and longer-term observation are required to verify the efectiveness of low-rTMS combined therapy on PSD. Meantime, a new meta-analysis could be analysized, which intervention sites and frequency are more effective in treating PSD.Systematic review registrationhttps://www.crd.york.ac.uk/prospero/, identifier: CRD42022376845