In-context Learning and Gradient Descent Revisited

In-context learning (ICL) has shown impressive results in few-shot learning tasks, yet its underlying mechanism is still not fully understood. A recent line of work suggests that ICL performs gradient descent (GD)-based optimization implicitly. While appealing, much of the research focuses on simplified settings, where the parameters of a shallow model are optimized. In this work, we revisit evidence for ICL-GD correspondence on realistic NLP tasks and models. We find gaps in evaluation, both in terms of problematic metrics and insufficient baselines. We show that surprisingly, even untrained models achieve comparable ICL-GD similarity scores despite not exhibiting ICL. Next, we explore a major discrepancy in the flow of information throughout the model between ICL and GD, which we term Layer Causality. We propose a simple GD-based optimization procedure that respects layer causality, and show it improves similarity scores significantly.Comment: Accepted to NAACL 2024 main conferenc

ReMark: Receptive Field based Spatial WaterMark Embedding Optimization using Deep Network

Watermarking is one of the most important copyright protection tools for digital media. The most challenging type of watermarking is the imperceptible one, which embeds identifying information in the data while retaining the latter's original quality. To fulfill its purpose, watermarks need to withstand various distortions whose goal is to damage their integrity. In this study, we investigate a novel deep learning-based architecture for embedding imperceptible watermarks. The key insight guiding our architecture design is the need to correlate the dimensions of our watermarks with the sizes of receptive fields (RF) of modules of our architecture. This adaptation makes our watermarks more robust, while also enabling us to generate them in a way that better maintains image quality. Extensive evaluations on a wide variety of distortions show that the proposed method is robust against most common distortions on watermarks including collusive distortion

The Membrane-Activated Chelator Stroke Intervention (MACSI) Trial of DP-b99 in Acute Ischemic Stroke: A Randomized, Double-Blind, Placebo-Controlled, Multinational Pivotal Phase III Study

Rationale Zinc is both a direct neurotoxin and a signaling mediator in multiple early and late detrimental processes following ischemia. DP-b99, a lipophilic moderate-affinity chelator of zinc, is a first-in-class multitargeted neuroprotective agent for ischemic stroke. DP-b99 has completed several Phase I studies and two double-blind placebo-controlled Phase II trials, which supported the safety of DP-b99 and were consistent with a beneficial effect on poststroke recuperation. Aim: Membrane-Activated Chelator Stroke Intervention is a Phase III study. The primary objective is to evaluate the safety and therapeutic effects of intravenous 1.0 mg/kg/day DP-b99, initiated within nine-hours of stroke onset in patients with moderately severe hemispheric acute ischemic stroke, through the analysis across the whole distribution of scores of the primary efficacy endpoint of the modified Rankin Scale, 90 days after the stroke. Methods The Membrane-Activated Chelator Stroke Intervention study is a randomized, double-blind, placebo-controlled, multicenter, multinational, parallel-arm trial comparing a placebo group to a group treated with intravenous DP-b99 for four consecutive days. Non-rtPA-treated acute ischemic stroke patients - with a baseline NIHSS score of 10-16 and a clinical syndrome that includes language dysfunction, visual field defect and/or neglect - will be stratified on a 1 : 1 basis to one of the two treatments. Half will be randomized within 0-4.5 h of stroke onset. Follow-up after the four treatment days will occur on days 12, 30 and 90. An interim futility analysis will be performed after primary endpoint data have been collected for 50% of 770 subjects planned to be enrolled. A data and safety monitoring board will assess safety data and will oversee the interim analysis. Conclusion This Phase III Membrane-Activated Chelator Stroke Intervention trial is based on promising data derived from previous Phase I and II DP-b99 trials and capitalizes on lessons learned from failures of past stroke studies in relation to neuroprotection, patient selection and data analysis