Feature emergence via margin maximization: case studies in algebraic tasks

Understanding the internal representations learned by neural networks is a cornerstone challenge in the science of machine learning. While there have been significant recent strides in some cases towards understanding how neural networks implement specific target functions, this paper explores a complementary question -- why do networks arrive at particular computational strategies? Our inquiry focuses on the algebraic learning tasks of modular addition, sparse parities, and finite group operations. Our primary theoretical findings analytically characterize the features learned by stylized neural networks for these algebraic tasks. Notably, our main technique demonstrates how the principle of margin maximization alone can be used to fully specify the features learned by the network. Specifically, we prove that the trained networks utilize Fourier features to perform modular addition and employ features corresponding to irreducible group-theoretic representations to perform compositions in general groups, aligning closely with the empirical observations of Nanda et al. and Chughtai et al. More generally, we hope our techniques can help to foster a deeper understanding of why neural networks adopt specific computational strategies

Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models

Watermarking generative models consists of planting a statistical signal (watermark) in a model's output so that it can be later verified that the output was generated by the given model. A strong watermarking scheme satisfies the property that a computationally bounded attacker cannot erase the watermark without causing significant quality degradation. In this paper, we study the (im)possibility of strong watermarking schemes. We prove that, under well-specified and natural assumptions, strong watermarking is impossible to achieve. This holds even in the private detection algorithm setting, where the watermark insertion and detection algorithms share a secret key, unknown to the attacker. To prove this result, we introduce a generic efficient watermark attack; the attacker is not required to know the private key of the scheme or even which scheme is used. Our attack is based on two assumptions: (1) The attacker has access to a "quality oracle" that can evaluate whether a candidate output is a high-quality response to a prompt, and (2) The attacker has access to a "perturbation oracle" which can modify an output with a nontrivial probability of maintaining quality, and which induces an efficiently mixing random walk on high-quality outputs. We argue that both assumptions can be satisfied in practice by an attacker with weaker computational capabilities than the watermarked model itself, to which the attacker has only black-box access. Furthermore, our assumptions will likely only be easier to satisfy over time as models grow in capabilities and modalities. We demonstrate the feasibility of our attack by instantiating it to attack three existing watermarking schemes for large language models: Kirchenbauer et al. (2023), Kuditipudi et al. (2023), and Zhao et al. (2023). The same attack successfully removes the watermarks planted by all three schemes, with only minor quality degradation.Comment: Blog post: https://www.harvard.edu/kempner-institute/2023/11/09/watermarking-in-the-sand

Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models

Watermarking generative models consists of planting a statistical signal (watermark) in a model’s output so that it can be later verified that the output was generated by the given model. A strong watermarking scheme satisfies the property that a computationally bounded attacker cannot erase the watermark without causing significant quality degradation. In this paper, we study the (im)possibility of strong watermarking schemes. We prove that, under well-specified and natural assumptions, strong watermarking is impossible to achieve. This holds even in the private detection algorithm setting, where the watermark insertion and detection algorithms share a secret key, unknown to the attacker. To prove this result, we introduce a generic efficient watermark attack; the attacker is not required to know the private key of the scheme or even which scheme is used. Our attack is based on two assumptions: (1) The attacker has access to a “quality oracle” that can evaluate whether a candidate output is a high-quality response to a prompt, and (2) The attacker has access to a “perturbation oracle” which can modify an output with a nontrivial probability of maintaining quality, and which induces an efficiently mixing random walk on high-quality outputs. We argue that both assumptions can be satisfied in practice by an attacker with weaker computational capabilities than the watermarked model itself, to which the attacker has only black-box access. Furthermore, our assumptions will likely only be easier to satisfy over time as models grow in capabilities and modalities. We demonstrate the feasibility of our attack by instantiating it to attack three existing watermarking schemes for large language models: Kirchenbauer et al. (2023), Kuditipudi et al. (2023), and Zhao et al. (2023). The same attack successfully removes the watermarks planted by all three schemes, with only minor quality degradation

Regulation of heparanase expression in coronary artery disease in diabetic, hyperlipidemic swine

Objective Enzymatic degradation of the extracellular matrix is known to be powerful regulator of atherosclerosis. However, little is known about the enzymatic regulation of heparan sulfate proteoglycans (HSPGs) during the formation and progression of atherosclerotic plaques. Methods and results Swine were rendered diabetic through streptozotocin injection and hyperlipidemic through a high fat diet. Arterial remodeling and local endothelial shear stress (ESS) were assessed using intravascular ultrasound, coronary angiography and computational fluid dynamics at weeks 23 and 30. Coronary arteries were harvested and 142 arterial subsegments were analyzed using histomorphologic staining, immunostaining and real time PCR. Heparanase staining and activity was increased in arterial segments with low ESS, in lesions with thin cap fibroatheroma (TCFA) morphology and in lesions with severely degraded internal elastic laminae. In addition, heparanase staining co-localized with staining for CD45 and MMP-2 within atherosclerotic plaques. Dual staining with gelatinase zymography and heparanase immunohistochemical staining demonstrated co-localization of matrix metalloprotease activity with heparanase staining. A heparanase enzymatic activity assay demonstrated increased activity in TCFA lesions, subsegments with low ESS and in macrophages treated with oxidized LDL or angiotensin II. Conclusions Taken together, our results support a critical role for heparanase in the development of vulnerable plaques and suggest a novel therapeutic target for the treatment of atherosclerosis.Novartis (Firm)Boston Scientific CorporationNational Institutes of Health (U.S.) (Grant R01 GM49039

Augmented Expression and Activity of Extracellular Matrix-Degrading Enzymes in Regions of Low Endothelial Shear Stress Colocalize With Coronary Atheromata With Thin Fibrous Caps in Pigs

Background—The molecular mechanisms that determine the localized formation of thin-capped atheromata in the coronary arteries remain unknown. This study tested the hypothesis that low endothelial shear stress augments the expression of matrix-degrading proteases and thereby promotes the formation of thin-capped atheromata. : Methods and Results—Intravascular ultrasound–based, geometrically correct 3-dimensional reconstruction of the coronary arteries of 12 swine was performed in vivo 23 weeks after initiation of diabetes mellitus and a hyperlipidemic diet. Local endothelial shear stress was calculated in plaque-free subsegments of interest (n=142) with computational fluid dynamics. At week 30, the coronary arteries (n=31) were harvested and the same subsegments were identified. The messenger RNA and protein expression and elastolytic activity of selected elastases and their endogenous inhibitors were assessed. Subsegments with low preceding endothelial shear stress at week 23 showed reduced endothelial coverage, enhanced lipid accumulation, and intense infiltration of activated inflammatory cells at week 30. These lesions showed increased expression of messenger RNAs encoding matrix metalloproteinase-2, -9, and -12, and cathepsins K and S relative to their endogenous inhibitors and increased elastolytic activity. Expression of these enzymes correlated positively with the severity of internal elastic lamina fragmentation. Thin-capped atheromata developed in regions with lower preceding endothelial shear stress and had reduced endothelial coverage, intense lipid and inflammatory cell accumulation, enhanced messenger RNA expression and elastolytic activity of MMPs and cathepsins, and severe internal elastic lamina fragmentation. : Conclusions—Low endothelial shear stress induces endothelial discontinuity and accumulation of activated inflammatory cells, thereby augmenting the expression and activity of elastases in the intima and shifting the balance with their inhibitors toward matrix breakdown. Our results provide new insight into the mechanisms of regional formation of plaques with thin fibrous caps.Novartis Pharmaceuticals CorporationBoston Scientific CorporationHellenic Heart FoundationHellenic Atherosclerosis SocietyAlexander S. Onassis Public Benefit FoundationPropondis FoundationHellenic Harvard FoundationA.G. Leventis FoundationPhilip Morris International. External Research ProgramAmerican Heart Association (Scientist Development Grant)National Institutes of Health (U.S.) (Grant NIHR01 GM49039

Multiparameter Telemetry as a Sensitive Screening Method to Detect Vaccine Reactogenicity in Mice

Refined vaccines and adjuvants are urgently needed to advance immunization against global infectious challenges such as HIV, hepatitis C, tuberculosis and malaria. Large-scale screening efforts are ongoing to identify adjuvants with improved efficacy profiles. Reactogenicity often represents a major hurdle to the clinical use of new substances. Yet, irrespective of its importance, this parameter has remained difficult to screen for, owing to a lack of sensitive small animal models with a capacity for high throughput testing. Here we report that continuous telemetric measurements of heart rate, heart rate variability, body core temperature and locomotor activity in laboratory mice readily unmasked systemic side-effects of vaccination, which went undetected by conventional observational assessment and clinical scoring. Even minor aberrations in homeostasis were readily detected, ranging from sympathetic activation over transient pyrogenic effects to reduced physical activity and apathy. Results in real-time combined with the potential of scalability and partial automation in the industrial context suggest multiparameter telemetry in laboratory mice as a first-line screen for vaccine reactogenicity. This may accelerate vaccine discovery in general and may further the success of vaccines in combating infectious disease and cancer

2nd ESMO Consensus Conference on Lung Cancer: early-stage non-small-cell lung cancer consensus on diagnosis, treatment and follow-up

This manuscript provides recommendations for the diagnosis, treatment and follow-up of early stage non-small cell lung cancer developed during the 2nd ESMO Consensus Conference on Lung Cancer in May 201