Instruction Tuning With Loss Over Instructions

Instruction tuning plays a crucial role in shaping the outputs of language models (LMs) to desired styles. In this work, we propose a simple yet effective method, INSTRUCTION MODELLING (IM), which trains LMs by applying a loss function to the instruction and prompt part rather than solely to the output part. Through experiments across 21 diverse benchmarks, we show that, in many scenarios, IM can effectively improve the LM performance on both NLP tasks (e.g., MMLU, TruthfulQA, and HumanEval) and open-ended generation benchmarks (e.g., MT-Bench and AlpacaEval). Remarkably, in the most advantageous case, IM boosts model performance on AlpacaEval 1.0 by over 100%. We identify two key factors influencing the effectiveness of IM: (1) The ratio between instruction length and output length in the training data; and (2) The number of training examples. We observe that IM is especially beneficial when trained on datasets with lengthy instructions paired with brief outputs, or under the Superficial Alignment Hypothesis (SAH) where a small amount of training examples are used for instruction tuning. Further analysis substantiates our hypothesis that our improvement can be attributed to reduced overfitting to instruction tuning datasets. It is worth noting that we are not proposing IM as a replacement for the current instruction tuning process. Instead, our work aims to provide practical guidance for instruction tuning LMs, especially in low-resource scenarios. Our code is available at https://github.com/ZhengxiangShi/InstructionModelling

Study of hadronic event-shape variables in multijet final states in pp collisions at √s=7 TeV

Peer reviewe

Reconstruction of primary vertices at the ATLAS experiment in Run 1 proton–proton collisions at the LHC

This paper presents the method and performance of primary vertex reconstruction in proton–proton collision data recorded by the ATLAS experiment during Run 1 of the LHC. The studies presented focus on data taken during 2012 at a centre-of-mass energy of √s=8 TeV. The performance has been measured as a function of the number of interactions per bunch crossing over a wide range, from one to seventy. The measurement of the position and size of the luminous region and its use as a constraint to improve the primary vertex resolution are discussed. A longitudinal vertex position resolution of about 30μm is achieved for events with high multiplicity of reconstructed tracks. The transverse position resolution is better than 20μm and is dominated by the precision on the size of the luminous region. An analytical model is proposed to describe the primary vertex reconstruction efficiency as a function of the number of interactions per bunch crossing and of the longitudinal size of the luminous region. Agreement between the data and the predictions of this model is better than 3% up to seventy interactions per bunch crossing

Constraints on parton distribution functions and extraction of the strong coupling constant from the inclusive jet cross section in pp collisions at √s=7 TeV

Peer reviewe

Molecular therapy for the treatment of hepatocellular carcinoma

Hepatocellular carcinoma (HCC) is the fifth most common cancer worldwide. Conventional cytotoxic chemotherapy has failed to show a substantial benefit for patients with HCC. Recently, a number of new drugs targeting molecular mechanisms involved in liver cell transformation have entered into clinical trials and led to encouraging results. In this review we summarise this data and point to a number of new compounds, which are currently being tested and can potentially broaden our therapeutic arsenal even further

Efficacy and tolerability of bevacizumab plus capecitabine as first-line therapy in patients with advanced hepatocellular carcinoma

Molecularly targeted agents with anti-angiogenic activity, including bevacizumab, have demonstrated clinical activity in patients with advanced /metastatic hepatocellular carcinoma (HCC). This multicentre phase II study involving patients from several Asian countries sought to evaluate the safety and efficacy of bevacizumab plus capecitabine in this population. METHODS: Histologically proven/clinically diagnosed advanced HCC patients received bevacizumab 7.5 mg kg(-1) on day 1 and capecitabine 800 mg m(-2) twice daily on days 1-14 every 3 weeks as first-line therapy. RESULTS: A total of 45 patients were enrolled; 44 (96%) had extrahepatic metastasis and/or major vessel invasion and 30( 67%) had hepatitis B. No grade 3/4 haematological toxicity occurred. Treatment-related grade 3/4 non-haematological toxicities included diarrhoea (n = 2, 4%), nausea/ vomiting ( n = 1, 2%), gastrointestinal bleeding (n = 4, 9%) and hand- foot syndrome (n = 4, 9%). The overall response rate ( RECIST) was 9% and the disease control rate was 52%. Overall , median progression-free survival (PFS) and overall survival(OS) were 2.7 and 5.9 months, respectively. Median PFS and OS were 3.6 and 8.2 months, respectively, for Cancer of the Liver Italian Programme (CLIP) score <= 3 patients, and 1.4 and 3.3 months, respectively, for CLIP score 4 patients. CONCLUSION: The bevacizumab-capecitabine combination shows good tolerability and modest anti-tumour activity in patients with advanced HCC

Waterborne Outbreak of Gastroenteritis: Effects on Sick Leaves and Cost of Lost Workdays

We examined the acute and cumulative effects of this incidence on sick leaves among public sector employees residing in the clean and contaminated areas, and the additional costs of lost workdays due to the incidence.Daily information on sick leaves of 1789 Finnish Public Sector Study participants was obtained from employers' registers. Global Positioning System-coordinates were used for linking participants to the clean and contaminated areas. Prevalence ratios (PR) for weekly sickness absences were calculated using binomial regression analysis. Calculations for the costs were based on prior studies.Among those living in the contaminated areas, the prevalence of participants on sick leave was 3.54 (95% confidence interval (CI) 2.97–4.22) times higher on the week following the incidence compared to the reference period. Those living and working in the clean area were basically not affected, the corresponding PR for sick leaves was 1.12, 95% CI 0.73–1.73. No cumulative effects on sick leaves were observed among the exposed. The estimated additional costs of lost workdays due to the incidence were 1.8–2.1 million euros.The prevalence of sickness absences among public sector employees residing in affected areas increased shortly after drinking water distribution system was contaminated, but no long-term effects were observed. The estimated costs of lost workdays were remarkable, thus, the cost-benefits of better monitoring systems for the water distribution systems should be evaluated

Safety and pharmacokinetics of motesanib in combination with gemcitabine for the treatment of patients with solid tumours

The aim of this open-label phase 1b study was to assess the safety and pharmacokinetics of motesanib in combination with gemcitabine in patients with advanced solid tumours. Eligible patients with histologically or cytologically documented solid tumours or lymphoma were enroled in three sequential, dose-escalating cohorts to receive motesanib 50 mg once daily (QD), 75 mg two times daily (BID), or 125 mg QD in combination with gemcitabine (1000 mg m−2). The primary end point was the incidence of dose-limiting toxicities (DLTs). Twenty-six patients were enroled and received motesanib and gemcitabine. No DLTs occurred. The 75 mg BID cohort was discontinued early; therefore, 125 mg QD was the maximum target dose. Sixteen patients (62%) experienced motesanib-related adverse events, most commonly lethargy (n=6), diarrhoea (n=4), fatigue (n=3), headache (n=3), and nausea (n=3). The pharmacokinetics of motesanib and of gemcitabine were not markedly affected after combination therapy. The objective response rate was 4% (1 of 26), and 27% (7 of 26) of patients achieved stable disease. In conclusion, treatment with motesanib plus gemcitabine was well tolerated, with adverse event and pharmacokinetic profiles similar to that observed in monotherapy studies