HyperTuning: Toward Adapting Large Language Models without Back-propagation

Fine-tuning large language models for different tasks can be costly and inefficient, and even methods that reduce the number of tuned parameters still require full gradient-based optimization. We propose HyperTuning, a novel approach to model adaptation that uses a hypermodel to generate task-specific parameters for a fixed downstream model. We demonstrate a simple setup for hypertuning with HyperT5, a T5-based hypermodel that produces soft prefixes or LoRA parameters for a frozen T5 model from few-shot examples. We train HyperT5 in two stages: first, hyperpretraining with a modified conditional language modeling objective that trains a hypermodel to generate parameters; second, multi-task fine-tuning (MTF) on a large number of diverse language tasks. We evaluate HyperT5 on P3, MetaICL and Super-NaturalInstructions datasets, and show that it can effectively generate parameters for unseen tasks. Moreover, we show that using hypermodel-generated parameters as initializations for further parameter-efficient fine-tuning improves performance. HyperTuning can thus be a flexible and efficient way to leverage large language models for diverse downstream applications

Research Progress on the Application of Spectroscopy in Meat Spoilage Detection

The growth and metabolism of microorganisms is the main cause of meat spoilage. The rapid and nondestructive techniques for detecting microorganisms in meat have attracted more and more attentions. Spectroscopic techniques such as Raman spectroscopy, infrared spectroscopy and spectral imaging show great advantages in rapid and non-destructive detection, but their application in meat spoilage detection has not been timely summarized. Based on an overview of the dominant spoilage organisms and microbial metabolism in meat under different storage conditions, this paper briefly describes the material basis for spectroscopic prediction of meat spoilage. Then, the application of Raman spectroscopy, infrared spectroscopy and spectral imaging technology in predicting the shelf life of meat is summarized. The efficiency of predictive modeling of meat shelf life based on total bacterial count or total volatile basic nitrogen (TVB-N) content and problems existing in this field are highlighted. We anticipate that this review will provide new ideas and theoretical guidance for the development and application of rapid and nondestructive techniques for meat spoilage identification

Adjusting CA19-9 values with clinical stage and bilirubin to better predict survival of resectable pancreatic cancer patients: 5-year-follow-up of a single center

BackgroundPancreatic cancer mortality is growing every year, and radical resection is the most essential therapy strategy. It is critical to evaluate the long-term prognosis of individuals receiving radical surgery. CA19-9 is a biomarker for patient recurrence and survival, however obstructive jaundice has a significant impact on this index. Researchers have attempted to modify the index using various modification methods, but the results have been unsatisfactory. In this study, we adjusted CA19-9 values based on clinical stage and bilirubin and found that it provided better prediction than CA19-9 alone in assessing patients.MethodsWe analyzed over 5 years follow-up records of patients who underwent radical pancreatic cancer surgery between August 2009 and May 2017 in a single center. We investigated the association of risk factors with overall survival (OS) as well as disease-free survival (DFS) after surgery. Threshold values for high-risk features associated with poor prognosis in resectable pancreatic cancer were determined. The hazard ratios of the indicators were eventually examined under the stratification of patients’ clinical stages.ResultsA total of 202 patients were involved in the study. The optimum cut-off values for CA19-9 and CA19-9/TB for predicting overall survival were 219.4 (p = 0.0075) and 18.8 (p = 0.0353), respectively. CA19-9>219.4 increased the risk of patient mortality by 1.70 times (95% CI 1.217-2.377, p = 0.002), and tumor poor differentiation raised the risk by 1.66 times (95% CI 1.083-2.553, P = 0.02). Based on clinical stage stratification, we found discrepancies in the predictive efficacy of CA19-9 and CA19-9/TB. CA19-9 was a better predictor in clinical stage 1 (HR = 2.056[CI 95%1.169-3.616], P = 0.012), whereas CA19-9/TB indications were better in stages 2 (HR = 1.650[CI 95%1.023-2.662], P = 0.040) and 3 (HR = 3.989[CI95%1.145-13.896], P = 0.030).ConclusionsCA19-9, CEA, and tumor differentiation are predictors for patients with resectable PDAC. CA19-9 values can be adjusted based on clinical stage and bilirubin levels to better predict overall survival in patients with resectable PDAC. CA19-9>219.4 predicted poor survival in individuals in clinical stage 1, whereas CA19-9/TB>18.8 predicted poor survival for individuals in stages 2 and 3

Optical bulk-boundary dichotomy in a quantum spin Hall insulator

The bulk-boundary correspondence is a key concept in topological quantum materials. For instance, a quantum spin Hall insulator features a bulk insulating gap with gapless helical boundary states protected by the underlying Z2 topology. However, the bulk-boundary dichotomy and distinction are rarely explored in optical experiments, which can provide unique information about topological charge carriers beyond transport and electronic spectroscopy techniques. Here, we utilize mid-infrared absorption micro-spectroscopy and pump-probe micro-spectroscopy to elucidate the bulk-boundary optical responses of Bi4Br4, a recently discovered room-temperature quantum spin Hall insulator. Benefiting from the low energy of infrared photons and the high spatial resolution, we unambiguously resolve a strong absorption from the boundary states while the bulk absorption is suppressed by its insulating gap. Moreover, the boundary absorption exhibits a strong polarization anisotropy, consistent with the one-dimensional nature of the topological boundary states. Our infrared pump-probe microscopy further measures a substantially increased carrier lifetime for the boundary states, which reaches one nanosecond scale. The nanosecond lifetime is about one to two orders longer than that of most topological materials and can be attributed to the linear dispersion nature of the helical boundary states. Our findings demonstrate the optical bulk-boundary dichotomy in a topological material and provide a proof-of-principal methodology for studying topological optoelectronics.Comment: 26 pages, 4 figure

Association of inpatient use of angiotensin converting enzyme inhibitors and angiotensin II receptor blockers with mortality among patients with hypertension hospitalized with COVID-19

Rationale: Use of angiotensin-converting enzyme inhibitors (ACEIs) and angiotensin II receptor blockers (ARBs) is a major concern for clinicians treating coronavirus disease 2019 (COVID-19) in patients with hypertension. Objective: To determine the association between in-hospital use of ACEI/ARB and all-cause mortality in COVID-19 patients with hypertension. Methods and Results: This retrospective, multi-center study included 1128 adult patients with hypertension diagnosed with COVID-19, including 188 taking ACEI/ARB (ACEI/ARB group; median age 64 [IQR 55-68] years; 53.2% men) and 940 without using ACEI/ARB (non-ACEI/ARB group; median age 64 [IQR 57-69]; 53.5% men), who were admitted to nine hospitals in Hubei Province, China from December 31, 2019 to February 20, 2020. Unadjusted mortality rate was lower in the ACEI/ARB group versus the non-ACEI/ARB group (3.7% vs. 9.8%; P = 0.01). In mixed-effect Cox model treating site as a random effect, after adjusting for age, gender, comorbidities, and in-hospital medications, the detected risk for all-cause mortality was lower in the ACEI/ARB group versus the non-ACEI/ARB group (adjusted HR, 0.42; 95% CI, 0.19-0.92; P =0.03). In a propensity score-matched analysis followed by adjusting imbalanced variables in mixed-effect Cox model, the results consistently demonstrated lower risk of COVID-19 mortality in patients who received ACEI/ARB versus those who did not receive ACEI/ARB (adjusted HR, 0.37; 95% CI, 0.15-0.89; P = 0.03). Further subgroup propensity score-matched analysis indicated that, compared to use of other antihypertensive drugs, ACEI/ARB was also associated with decreased mortality (adjusted HR, 0.30; 95%CI, 0.12-0.70; P = 0.01) in COVID-19 patients with hypertension. Conclusions: Among hospitalized COVID-19 patients with hypertension, inpatient use of ACEI/ARB was associated with lower risk of all-cause mortality compared with ACEI/ARB non-users. While study interpretation needs to consider the potential for residual confounders, it is unlikely that in-hospital use of ACEI/ARB was associated with an increased mortality risk