A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models

Distillation from Weak Teacher (DWT) is a method of transferring knowledge from a smaller, weaker teacher model to a larger student model to improve its performance. Previous studies have shown that DWT can be effective in the vision domain and natural language processing (NLP) pre-training stage. Specifically, DWT shows promise in practical scenarios, such as enhancing new generation or larger models using pre-trained yet older or smaller models and lacking a resource budget. However, the optimal conditions for using DWT have yet to be fully investigated in NLP pre-training. Therefore, this study examines three key factors to optimize DWT, distinct from those used in the vision domain or traditional knowledge distillation. These factors are: (i) the impact of teacher model quality on DWT effectiveness, (ii) guidelines for adjusting the weighting value for DWT loss, and (iii) the impact of parameter remapping as a student model initialization technique for DWT.Comment: Findings of ACL 202

Crystal Facet Effect in Plasmonic Catalysis

In the realm of plasmonic catalytic systems, much attention has been devoted to the plasmon-derived mechanisms, yet the influence of nanoparticles' crystal facets in this type of processes has been sparsely investigated. In this work, we study the plasmon-assisted electrocatalytic CO2 reduction reaction using three different shapes of plasmonic Au nanoparticles - nanocube (NC), rhombic dodecahedron (RD) and octahedron (OC) - with three different exposed facets: {100}, {110} and {111}, respectively. These particles were synthesized with similar sizes and LSPR wavelengths to reveal the role of the facet more than other contributions to the plasmon-assisted reaction. Upon plasmon excitation, Au OCs exhibited nearly a doubling in the Faradaic efficiency of CO (FE(CO)) and a remarkable threefold enhancement in the partial current density of CO (j(CO)) compared to the non-illuminated response, NCs also demonstrated an improved performance under illumination. In contrast, Au RDs showed nearly the same performance in dark or light conditions. Temperature-dependent experiments ruled out heat as the main factor in the enhanced response of Au OCs and NCs. Large-scale atomistic simulations of the nanoparticles' electronic structure and electromagnetic modeling revealed higher hot carrier abundance and electric field enhancement on Au OCs and NCs compared to RDs. Abundant hot carriers on edges facilitate molecular activation, leading to enhanced selectivity and activity. Thus, OCs with the highest edge/facet ratio exhibited the strongest enhancement in FE(CO) and j(CO) upon illumination. This observation is further supported by plasmon-assisted H2 evolution reaction experiments. Our findings highlight the dominance of low coordinated sites over facets in plasmonic catalytic processes, providing valuable insights for designing more efficient catalysts for solar fuels production

Synthetic lethality by targeting the RUVBL1/2-TTT complex in mTORC1-hyperactive cancer cells

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.Despite considerable efforts, mTOR inhibitors have produced limited success in the clinic. To define the vulnerabilities of mTORC1-addicted cancer cells and to find previously unknown therapeutic targets, we investigated the mechanism of piperlongumine, a small molecule identified in a chemical library screen to specifically target cancer cells with a hyperactive mTORC1 phenotype. Sensitivity to piperlongumine was dependent on its ability to suppress RUVBL1/2-TTT, a complex involved in chromatin remodeling and DNA repair. Cancer cells with high mTORC1 activity are subjected to higher levels of DNA damage stress via c-Myc and displayed an increased dependency on RUVBL1/2 for survival and counteracting genotoxic stress. Examination of clinical cancer tissues also demonstrated that high mTORC1 activity was accompanied by high RUVBL2 expression. Our findings reveal a previously unknown role for RUVBL1/2 in cell survival, where it acts as a functional chaperone to mitigate stress levels induced in the mTORC1-Myc-DNA damage axis.NIH 1RO1CA142805National Research Foundation of Korea (NRF) grant (NRF-2017R1C1B1006072

Cytoprotective Effect of Phloroglucinol on Oxidative Stress Induced Cell Damage via Catalase Activation

Abstract We investigated the cytoprotective effect of phloroglucinol, which was isolated from Ecklonia cava (brown alga), against oxidative stress induced cell damage in Chinese hamster lung fibroblast (V79-4) cells. Phloroglucinol was found to scavenge 1,1-diphenyl-2-picrylhydrazyl (DPPH) radical, hydrogen peroxide (H 2 O 2 ), hydroxy radical, intracellular reactive oxygen species (ROS), and thus prevented lipid peroxidation. As a result, phloroglucinol reduced H 2 O 2 induced apoptotic cells formation in V79-4 cells. In addition, phloroglucinol inhibited cell damage induced by serum starvation and radiation through scavenging ROS. Phloroglucinol increased the catalase activity and its protein expression. In addition, catalase inhibitor abolished the protective effect of phloroglucinol from H 2 O 2 induced cell damage. Furthermore, phloroglucinol increased phosphorylation of extracellular signal regulated kinase (ERK). Taken together, the results suggest that phloroglucinol protects V79-4 cells against oxidative damage by enhancing the cellular catalase activity and modulating ERK signal pathway

An Integrated Magneto-Electrochemical Device for the Rapid Profiling of Tumour Extracellular Vesicles from Blood Plasma

Assays for cancer diagnosis via the analysis of biomarkers on circulating extracellular vesicles (EVs) typically have lengthy sample workups, limited throughput or insufficient sensitivity, or do not use clinically validated biomarkers. Here we report the development and performance of a 96-well assay that integrates the enrichment of EVs by antibody-coated magnetic beads and the electrochemical detection, in less than one hour of total assay time, of EV-bound proteins after enzymatic amplification. By using the assay with a combination of antibodies for clinically relevant tumour biomarkers (EGFR, EpCAM, CD24 and GPA33) of colorectal cancer (CRC), we classified plasma samples from 102 patients with CRC and 40 non-CRC controls with accuracies of more than 96%, prospectively assessed a cohort of 90 patients, for whom the burden of tumour EVs was predictive of five-year disease-free survival, and longitudinally analysed plasma from 11 patients, for whom the EV burden declined after surgery and increased on relapse. Rapid assays for the detection of combinations of tumour biomarkers in plasma EVs may aid cancer detection and patient monitoring

Scintillation Arc from FRB 20220912A

We present the interstellar scintillation analysis of fast radio burst (FRB) 20220912A during its extremely active episode in 2022 using data from the Five-hundred-meter Aperture Spherical Radio Telescope (FAST). We detect a scintillation arc in the FRB's secondary spectrum, which describes the power in terms of the scattered FRB signals' time delay and Doppler shift. The arc indicates that the scintillation is caused by a highly localized region of the ionized interstellar medium (IISM). Our analysis favors a Milky Way origin for the localized scattering medium but cannot rule out a host galaxy origin. We present our method for detecting the scintillation arc, which can be applied generally to sources with irregularly spaced bursts or pulses. These methods could help shed light on the complex interstellar environment surrounding the FRBs and in our Galaxy.Comment: SCIENCE CHINA Physics, Mechanics & Astronomy , Volume 67, Issue 1: 219512 (2024

Epidemiologic Clues to SARS Origin in China

An epidemic of severe acute respiratory syndrome (SARS) began in Foshan municipality, Guangdong Province, China, in November 2002. We studied SARS case reports through April 30, 2003, including data from case investigations and a case series analysis of index cases. A total of 1,454 clinically confirmed cases (and 55 deaths) occurred; the epidemic peak was in the first week of February 2003. Healthcare workers accounted for 24% of cases. Clinical signs and symptoms differed between children (<18 years) and older persons (>65 years). Several observations support the hypothesis of a wild animal origin for SARS. Cases apparently occurred independently in at least five different municipalities; early case-patients were more likely than later patients to report living near a produce market (odds ratio undefined; lower 95% confidence interval 2.39) but not near a farm; and 9 (39%) of 23 early patients, including 6 who lived or worked in Foshan, were food handlers with probable animal contact

Low-Temperature Preparation of Superparamagnetic CoFe2O4 Microspheres with High Saturation Magnetization

Based on a low-temperature route, monodispersed CoFe2O4 microspheres (MSs) were fabricated through aggregation of primary nanoparticles. The microstructural and magnetic characteristics of the as-prepared MSs were characterized by X-ray diffraction/photoelectron spectroscopy, scanning/transmitting electron microscopy, and vibrating sample magnetometer. The results indicate that the diameters of CoFe2O4 MSs with narrow size distribution can be tuned from over 200 to ~330 nm. Magnetic measurements reveal these MSs exhibit superparamagnetic behavior at room temperature with high saturation magnetization. Furthermore, the mechanism of formation of the monodispersed CoFe2O4 MSs was discussed on the basis of time-dependent experiments, in which hydrophilic PVP plays a crucial role

Truly form-factor–free industrially scalable system integration for electronic textile architectures with multifunctional fiber devices

Funding Information: This work was supported by the European Commission (H2020, 1D-NEON, grant agreement ID: 685758). J.M.K. and L.G.O. acknowledge the support from the U.K. Research and Innovation (EPSRC, EP/P027628/1). We thank Y. Bernstein and J. Faulkner for helping with grammar check. Funding Information: Acknowledgments Funding:ThisworkwassupportedbytheEuropeanCommission(H2020,1D-NEON,grant agreementID:685758).J.M.K.andL.G.O.acknowledgethesupportfromtheU.K.Researchand Innovation(EPSRC,EP/P027628/1).W ethankY .BernsteinandJ.Faulknerforhelpingwith grammarcheck.Authorcontributions:S.L.andJ.M.K.conceivedtheproject.S.L.,L.G.O.,P .B., R.Martins,andJ.M.K.supervisedtheproject.S.L.andH.L.developedF-PD.S.L.,Y .-W .L., G.-H.A., D.-W .S., J.I.S.,andS.C.developedF-SC.C.L.F ., A.S.,R.I.,P .B., andR.Martinsdevelopedfiber transistor.S.L.,H.L.,andS.C.developedF-LED.ThefiberdeviceswereevaluatedbyS.L.,H.W .C., D.-W .S., H.L.,S.J.,S.D.H.,S.Y .B., S.Z.,W .H.-C., Y .-H.S., X.-B.F ., T .H.L., J.-W .J., andY .K. The developmentofweavingprocesswasconductedbyS.L.,H.W .C., F .M.M., P .J., andV .G.C. Thelaser interconnectionwasdevelopedbyS.L.,H.W .C., K.U.,M.E.,andM.S.Thetextiledemonstrations werecharacterizedbyS.L.,H.W .C., D.-W .S., J.Y ., S.S.,U.E.,S.N.,A.C.,A.M.,R.Momentè,J.G.,N.D., S.M.,C.-H.K.,M.L.,A.N.,D.J.,M.C.,andY .C. ThismanuscriptwaswrittenbyS.L.andJ.M.K.and reviewed by H.W .C., D.-W .S., M.C.,L.G.O., P .B., E.F ., and G.A.J.A. All authors discussed the results andcommentedonthemanuscript.Competinginterests:Theauthorsdeclarethattheyhave nocompetinginterests.Dataandmaterialsavailability:Alldataneededtoevaluatethe conclusionsinthepaperarepresentinthepaperand/ortheSupplementaryMaterials. Publisher Copyright: Copyright © 2023 The Authors, some rights reserved.An integrated textile electronic system is reported here, enabling a truly free form factor system via textile manufacturing integration of fiber-based electronic components. Intelligent and smart systems require freedom of form factor, unrestricted design, and unlimited scale. Initial attempts to develop conductive fibers and textile electronics failed to achieve reliable integration and performance required for industrial-scale manufacturing of technical textiles by standard weaving technologies. Here, we present a textile electronic system with functional one-dimensional devices, including fiber photodetectors (as an input device), fiber supercapacitors (as an energy storage device), fiber field-effect transistors (as an electronic driving device), and fiber quantum dot light-emitting diodes (as an output device). As a proof of concept applicable to smart homes, a textile electronic system composed of multiple functional fiber components is demonstrated, enabling luminance modulation and letter indication depending on sunlight intensity.publishersversionpublishe