Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced Data

Modern deep neural networks have achieved impressive performance on tasks from image classification to natural language processing. Surprisingly, these complex systems with massive amounts of parameters exhibit the same structural properties in their last-layer features and classifiers across canonical datasets when training until convergence. In particular, it has been observed that the last-layer features collapse to their class-means, and those class-means are the vertices of a simplex Equiangular Tight Frame (ETF). This phenomenon is known as Neural Collapse ($\mathcal{NC}$). Recent papers have theoretically shown that $\mathcal{NC}$ emerges in the global minimizers of training problems with the simplified ``unconstrained feature model''. In this context, we take a step further and prove the $\mathcal{NC}$ occurrences in deep linear networks for the popular mean squared error (MSE) and cross entropy (CE) losses, showing that global solutions exhibit $\mathcal{NC}$ properties across the linear layers. Furthermore, we extend our study to imbalanced data for MSE loss and present the first geometric analysis of $\mathcal{NC}$ under bias-free setting. Our results demonstrate the convergence of the last-layer features and classifiers to a geometry consisting of orthogonal vectors, whose lengths depend on the amount of data in their corresponding classes. Finally, we empirically validate our theoretical analyses on synthetic and practical network architectures with both balanced and imbalanced scenarios.Comment: 93 pages, 20 figures, 4 tables. Hien Dang and Tho Tran contributed equally to this wor

Synthèse de copolymères thermosensibles par polymérisation radicalaire contrôlée RAFT : caractérisation et étude de leur interaction avec des protéines

The synthesis of well-defined azlactone-functionalized thermoresponsive copolymers was performed using the RAFT polymerization and their interaction with primary amines and proteins was studied. Three different strategies have been developed. The first strategy was based on the synthesis of a novel azlactone-functionalized chain transfer agent which was used to target well defined azlactone-functionalized thermoresponsive polymers. In the second approach, ω-azlactone-terminated thermoresponsive copolymers were prepared by a combination of RAFT polymerization and “thiol ene” Michaël’s addition. In the last strategy, RAFT copolymerization of 2-vinyl-4,4 dimethylazlactone with other monomers has been performed to target well-defined azlactone functionalized copolymers. Finally, the reactivity of such reactive thermoresponsive copolymers was successfully demonstrated by bioconjugation with a model protein (lysozyme).Ce travail de thèse porte sur la synthèse de (co)polymères thermosensibles présentant une fonctionnalité azlactone par polymérisation radicalaire contrôlée RAFT pour l’ancrage de biomolécules. Trois stratégies différentes ont été étudiées. La première stratégie a consisté en la synthèse d’un nouvel agent de transfert permettant d’obtenir des polymères thermosensibles à fonctionnalité azlactone en position . La seconde approche a permis d’introduire la fonctionnalité azlactone en position ω de copolymères thermosensibles via la combinaison de la polymérisation RAFT et de l’addition de Michaël « thiol-ène ». La dernière stratégie a conduit à des copolymères thermosensibles à fonctionnalité azlactone en position latérale par copolymérisation RAFT de la 2-vinyl-4,4-diméthylazlactone avec d’autres monomères. Enfin, la réactivité de ces copolymères thermosensibles pour l’ancrage d’une protéine modèle (lysozyme) a été mise en évidence

Synthesis of thermoresponsive copolymers by RAFT polymerization : characterization and study of their interaction with proteins

Ce travail de thèse porte sur la synthèse de (co)polymères thermosensibles présentant une fonctionnalité azlactone par polymérisation radicalaire contrôlée RAFT pour l’ancrage de biomolécules. Trois stratégies différentes ont été étudiées. La première stratégie a consisté en la synthèse d’un nouvel agent de transfert permettant d’obtenir des polymères thermosensibles à fonctionnalité azlactone en position . La seconde approche a permis d’introduire la fonctionnalité azlactone en position ω de copolymères thermosensibles via la combinaison de la polymérisation RAFT et de l’addition de Michaël « thiol-ène ». La dernière stratégie a conduit à des copolymères thermosensibles à fonctionnalité azlactone en position latérale par copolymérisation RAFT de la 2-vinyl-4,4-diméthylazlactone avec d’autres monomères. Enfin, la réactivité de ces copolymères thermosensibles pour l’ancrage d’une protéine modèle (lysozyme) a été mise en évidence.The synthesis of well-defined azlactone-functionalized thermoresponsive copolymers was performed using the RAFT polymerization and their interaction with primary amines and proteins was studied. Three different strategies have been developed. The first strategy was based on the synthesis of a novel azlactone-functionalized chain transfer agent which was used to target well defined azlactone-functionalized thermoresponsive polymers. In the second approach, ω-azlactone-terminated thermoresponsive copolymers were prepared by a combination of RAFT polymerization and “thiol ene” Michaël’s addition. In the last strategy, RAFT copolymerization of 2-vinyl-4,4 dimethylazlactone with other monomers has been performed to target well-defined azlactone functionalized copolymers. Finally, the reactivity of such reactive thermoresponsive copolymers was successfully demonstrated by bioconjugation with a model protein (lysozyme)

"Michael addition" reaction onto vinyl sulfonyl(trifluoromethylsulfonyl) imide: An easy access to sulfonyl(trifluoromethylsulfonyl)imide-based monomers and polymers

WOS:000450376300009Sulfonyl(trifluoromethylsulfonyl) imide anion (STFSI, -SO2N(-)SO2CF3) is one of the most important sulfonimide groups for organic and material synthesis. However, the synthesis of STFSI-functionalized monomers and related polymers is still a problematic issue, comprising multi-step and complex synthetic processes. To overcome this drawback, we developed a robust and versatile strategy to prepare original STFSI-based monomers using the "Michael addition" reaction conducted in the presence of vinyl sulfonyl(trifluoromethylsulfonyl)imide and either an amine or a malonate derivative. Furthermore, the synthesized monomers were used to prepare the corresponding STFSI-functionalized polymers using step-growth polymerization with potential applications as electrolytes

Introducing the Azlactone Functionality into Polymers through Controlled Radical Polymerization: Strategies and Recent Developments

International audiencePolymers containing the highly reactive azlactone group have emerged as a powerful platform useful in various application areas. This Highlight summarizes recent developments in the field of azlactone-derived polymers made in our group using controlled radical polymerizations (ATRP and RAFT) and ‘click’ chemistry methodology (thiol-Michael addition), leading to well defined reactive polymers

A post-polymerization functionalization strategy for the synthesis of sulfonyl (trifluoromethanesulfonyl)imide functionalized (co)polymers

International audienc

Phosphonate-Functionalized Polycarbonates Synthesis through Ring-Opening Polymerization and Alternative Approaches

Well-defined phosphonate-functionalized polycarbonate with low dispersity (Ð = 1.22) was synthesized using organocatalyzed ring-opening polymerization (ROP) of novel phosphonate-based cyclic monomers. Copolymerization was also performed to access different structures of phosphonate-containing polycarbonates (PC). Furthermore, phosphonate-functionalized PC was successfully synthesized using a combination of ROP and post-modification reaction

Synthesis and characterization of innovative well-defined difluorophosphonylated-(co)polymers by RAFT polymerization

International audienc

Well-defined amine-reactive polymethacrylates through organocatalyzed controlled radical polymerization

International audienceA novel alkyl iodide bearing an azlactone group was synthesized and used as a reversible complexmediated polymerization (RCMP) initiator for the organocatalyzed RCMP of methyl methacrylate (MMA) and poly(ethylene glycol) methyl ether methacrylate (mPEGMA). The efficiency of the new initiator was studied under different processes including thermal and photochemical activation. Well-defined α-azlactone-functionalized polymethacrylates (Đ < 1.30) with an absence of "free" toxic transition metals were obtained for the first-time. The amine-scavenging ability of these functional polymers was demonstrated by using benzylamine

Thermoresponsive block copolymers containing reactive azlactone groups and their bioconjugation with lysozyme

International audienceThermoresponsive block copolymers based on poly(ethylene oxide) (PEO) and poly(N-isopropyl acrylamide) (PNIPAM) containing azlactone groups along the backbone and at the chain-end of the macromolecular chain were synthesized by statistically reversible addition–fragmentation chain transfer (RAFT) copolymerization and by using a combination of RAFT polymerization and thiol–ene Michael addition. Well-defined poly(ethylene oxide)-b-poly(2-vinyl-4,4-dimethylazlactone-co-N-isopropyl acrylamide) (PEO-b-P(VDM-co-NIPAM)) block copolymers and azlactone-terminated poly(ethylene oxide)-b-poly(N-isopropyl acrylamide) (PEO-b-PNIPAM-VDM) diblock copolymers with low polydispersity indices (PDIs ≤ 1.10) were prepared and fully characterized by 1H NMR spectroscopy, FT-IR spectroscopy, and SEC. Such PEO-b-P(VDM-co-NIPAM) block copolymers and azlactone-terminated PEO-b-PNIPAM block copolymers present tunable lower critical solution temperature (LCST) depending on PEO, PNIPAM, and PVDM molar ratios. The reactivity of the PEO44-b-P(VDM20-co-NIPAM80) copolymer (Mn,NMR = 14 200 g mol−1, PDI = 1.08) and of the PEO44-b-PNIPAM101-VDM copolymer (Mn,NMR = 13 700 g mol−1, PDI = 1.08) was studied with lysozyme as a model protein. A bioconjugate with a higher apparent molecular weight was obtained with the PEO44-b-P(VDM20-co-NIPAM80) copolymer in comparison with the one obtained using the PEO44-b-PNIPAM101-VDM copolymer as shown by sodium dodecyl sulfate polyacrylamide gel electrophoresis (SDS-PAGE). The results suggest promising applications of azlactone-functionalized polymers within the field of bioconjugation