OctoPack: Instruction Tuning Code Large Language Models

Finetuning large language models (LLMs) on instructions leads to vast performance improvements on natural language tasks. We apply instruction tuning using code, leveraging the natural structure of Git commits, which pair code changes with human instructions. We compile CommitPack: 4 terabytes of Git commits across 350 programming languages. We benchmark CommitPack against other natural and synthetic code instructions (xP3x, Self-Instruct, OASST) on the 16B parameter StarCoder model, and achieve state-of-the-art performance among models not trained on OpenAI outputs, on the HumanEval Python benchmark (46.2% pass@1). We further introduce HumanEvalPack, expanding the HumanEval benchmark to a total of 3 coding tasks (Code Repair, Code Explanation, Code Synthesis) across 6 languages (Python, JavaScript, Java, Go, C++, Rust). Our models, OctoCoder and OctoGeeX, achieve the best performance across HumanEvalPack among all permissive models, demonstrating CommitPack's benefits in generalizing to a wider set of languages and natural coding tasks. Code, models and data are freely available at https://github.com/bigcode-project/octopack.Comment: 57 pages (9 main), 39 figures, 16 table

Canagliflozin and Cardiovascular and Renal Outcomes in Type 2 Diabetes Mellitus and Chronic Kidney Disease in Primary and Secondary Cardiovascular Prevention Groups

Background: Canagliflozin reduces the risk of kidney failure in patients with type 2 diabetes mellitus and chronic kidney disease, but effects on specific cardiovascular outcomes are uncertain, as are effects in people without previous cardiovascular disease (primary prevention). Methods: In CREDENCE (Canagliflozin and Renal Events in Diabetes With Established Nephropathy Clinical Evaluation), 4401 participants with type 2 diabetes mellitus and chronic kidney disease were randomly assigned to canagliflozin or placebo on a background of optimized standard of care. Results: Primary prevention participants (n=2181, 49.6%) were younger (61 versus 65 years), were more often female (37% versus 31%), and had shorter duration of diabetes mellitus (15 years versus 16 years) compared with secondary prevention participants (n=2220, 50.4%). Canagliflozin reduced the risk of major cardiovascular events overall (hazard ratio [HR], 0.80 [95% CI, 0.67-0.95]; P=0.01), with consistent reductions in both the primary (HR, 0.68 [95% CI, 0.49-0.94]) and secondary (HR, 0.85 [95% CI, 0.69-1.06]) prevention groups (P for interaction=0.25). Effects were also similar for the components of the composite including cardiovascular death (HR, 0.78 [95% CI, 0.61-1.00]), nonfatal myocardial infarction (HR, 0.81 [95% CI, 0.59-1.10]), and nonfatal stroke (HR, 0.80 [95% CI, 0.56-1.15]). The risk of the primary composite renal outcome and the composite of cardiovascular death or hospitalization for heart failure were also consistently reduced in both the primary and secondary prevention groups (P for interaction >0.5 for each outcome). Conclusions: Canagliflozin significantly reduced major cardiovascular events and kidney failure in patients with type 2 diabetes mellitus and chronic kidney disease, including in participants who did not have previous cardiovascular disease

Design and baseline characteristics of the finerenone in reducing cardiovascular mortality and morbidity in diabetic kidney disease trial

Background: Among people with diabetes, those with kidney disease have exceptionally high rates of cardiovascular (CV) morbidity and mortality and progression of their underlying kidney disease. Finerenone is a novel, nonsteroidal, selective mineralocorticoid receptor antagonist that has shown to reduce albuminuria in type 2 diabetes (T2D) patients with chronic kidney disease (CKD) while revealing only a low risk of hyperkalemia. However, the effect of finerenone on CV and renal outcomes has not yet been investigated in long-term trials. Patients and Methods: The Finerenone in Reducing CV Mortality and Morbidity in Diabetic Kidney Disease (FIGARO-DKD) trial aims to assess the efficacy and safety of finerenone compared to placebo at reducing clinically important CV and renal outcomes in T2D patients with CKD. FIGARO-DKD is a randomized, double-blind, placebo-controlled, parallel-group, event-driven trial running in 47 countries with an expected duration of approximately 6 years. FIGARO-DKD randomized 7,437 patients with an estimated glomerular filtration rate >= 25 mL/min/1.73 m(2) and albuminuria (urinary albumin-to-creatinine ratio >= 30 to <= 5,000 mg/g). The study has at least 90% power to detect a 20% reduction in the risk of the primary outcome (overall two-sided significance level alpha = 0.05), the composite of time to first occurrence of CV death, nonfatal myocardial infarction, nonfatal stroke, or hospitalization for heart failure. Conclusions: FIGARO-DKD will determine whether an optimally treated cohort of T2D patients with CKD at high risk of CV and renal events will experience cardiorenal benefits with the addition of finerenone to their treatment regimen. Trial Registration: EudraCT number: 2015-000950-39; ClinicalTrials.gov identifier: NCT02545049

Canagliflozin and renal outcomes in type 2 diabetes and nephropathy

BACKGROUND Type 2 diabetes mellitus is the leading cause of kidney failure worldwide, but few effective long-term treatments are available. In cardiovascular trials of inhibitors of sodium–glucose cotransporter 2 (SGLT2), exploratory results have suggested that such drugs may improve renal outcomes in patients with type 2 diabetes. METHODS In this double-blind, randomized trial, we assigned patients with type 2 diabetes and albuminuric chronic kidney disease to receive canagliflozin, an oral SGLT2 inhibitor, at a dose of 100 mg daily or placebo. All the patients had an estimated glomerular filtration rate (GFR) of 30 to &lt;90 ml per minute per 1.73 m2 of body-surface area and albuminuria (ratio of albumin [mg] to creatinine [g], &gt;300 to 5000) and were treated with renin–angiotensin system blockade. The primary outcome was a composite of end-stage kidney disease (dialysis, transplantation, or a sustained estimated GFR of &lt;15 ml per minute per 1.73 m2), a doubling of the serum creatinine level, or death from renal or cardiovascular causes. Prespecified secondary outcomes were tested hierarchically. RESULTS The trial was stopped early after a planned interim analysis on the recommendation of the data and safety monitoring committee. At that time, 4401 patients had undergone randomization, with a median follow-up of 2.62 years. The relative risk of the primary outcome was 30% lower in the canagliflozin group than in the placebo group, with event rates of 43.2 and 61.2 per 1000 patient-years, respectively (hazard ratio, 0.70; 95% confidence interval [CI], 0.59 to 0.82; P=0.00001). The relative risk of the renal-specific composite of end-stage kidney disease, a doubling of the creatinine level, or death from renal causes was lower by 34% (hazard ratio, 0.66; 95% CI, 0.53 to 0.81; P&lt;0.001), and the relative risk of end-stage kidney disease was lower by 32% (hazard ratio, 0.68; 95% CI, 0.54 to 0.86; P=0.002). The canagliflozin group also had a lower risk of cardiovascular death, myocardial infarction, or stroke (hazard ratio, 0.80; 95% CI, 0.67 to 0.95; P=0.01) and hospitalization for heart failure (hazard ratio, 0.61; 95% CI, 0.47 to 0.80; P&lt;0.001). There were no significant differences in rates of amputation or fracture. CONCLUSIONS In patients with type 2 diabetes and kidney disease, the risk of kidney failure and cardiovascular events was lower in the canagliflozin group than in the placebo group at a median follow-up of 2.62 years

On the distribution alignment of propagation in graph neural networks

Graph neural networks (GNNs) have been widely adopted for modeling graph-structure data. Most existing GNN studies have focused on designing different strategies to propagate information over the graph structures. After systematic investigations, we observe that the propagation step in GNNs matters, but its resultant performance improvement is insensitive to the location where we apply it. Our empirical examination further shows that the performance improvement brought by propagation mostly comes from a phenomenon of distribution alignment, i.e., propagation over graphs actually results in the alignment of the underlying distributions between the training and test sets. The findings are instrumental to understand GNNs, e.g., why decoupled GNNs can work as good as standard GNNs.11 Source code: https://github.com/THUDM/DistAlign-GNNs

Toward secure and efficient deep learning inference in dependable IoT systems

The rapid development of deep learning (DL) enables resource-constrained systems and devices [e.g., Internet of Things (IoT)] to perform sophisticated artificial intelligence (AI) applications. However, AI models, such as deep neural networks (DNNs), are known to be vulnerable to adversarial examples (AEs). Past works on defending against AEs require heavy computations in the model training or inference processes, making them impractical to be applied in IoT systems. In this article, we propose a novel method, Super-IoT, to enhance the security and efficiency of AI applications in distributed IoT systems. Specifically, Super-IoT utilizes a pixel drop operation to eliminate adversarial perturbations from the input and reduce network transmission throughput. Then, it adopts a sparse signal recovery method to reconstruct the dropped pixels and wavelet-based denoising method to reduce the artificial noise. Super-IoT is a lightweight method with negligible computation cost to IoT devices and little impact on the DNN model performance. Extensive evaluations show that it can outperform three existing AE defensive solutions against most of the AE attacks with better transmission efficiency