Parameter-Efficient Tuning Makes a Good Classification Head

In recent years, pretrained models revolutionized the paradigm of natural language understanding (NLU), where we append a randomly initialized classification head after the pretrained backbone, e.g. BERT, and finetune the whole model. As the pretrained backbone makes a major contribution to the improvement, we naturally expect a good pretrained classification head can also benefit the training. However, the final-layer output of the backbone, i.e. the input of the classification head, will change greatly during finetuning, making the usual head-only pretraining (LP-FT) ineffective. In this paper, we find that parameter-efficient tuning makes a good classification head, with which we can simply replace the randomly initialized heads for a stable performance gain. Our experiments demonstrate that the classification head jointly pretrained with parameter-efficient tuning consistently improves the performance on 9 tasks in GLUE and SuperGLUE.Comment: Accepted as a long paper to EMNLP 2022 Main Conferenc

Competitive and/or cooperative interactions of graphene-family materials and benzo[a]pyrene with pulmonary surfactant: a computational and experimental study

Background Airborne nanoparticles can be inhaled and deposit in human alveoli, where pulmonary surfactant (PS) molecules lining at the alveolar air–water interface act as the first barrier against inhaled nanoparticles entering the body. Although considerable efforts have been devoted to elucidate the mechanisms underlying nanoparticle-PS interactions, our understanding on this important issue is limited due to the high complexity of the atmosphere, in which nanoparticles are believed to experience transformations that remarkably change the nanoparticles’ surface properties and states. By contrast with bare nanoparticles that have been extensively studied, relatively little is known about the interactions between PS and inhaled nanoparticles which already adsorb contaminants. In this combined experimental and computational effort, we investigate the joint interactions between PS and graphene-family materials (GFMs) with coexisting benzo[a]pyrene (BaP). Results Depending on the BaP concentration, molecular agglomeration, and graphene oxidation, different nanocomposite structures are formed via BaPs adsorption on GFMs. Upon deposition of GFMs carrying BaPs at the pulmonary surfactant (PS) layer, competition and cooperation of interactions between different components determines the interfacial processes including BaP solubilization, GFM translocation and PS perturbation. Importantly, BaPs adsorbed on GFMs are solubilized to increase BaP’s bioavailability. By contrast with graphene adhering on the PS layer to release part of adsorbed BaPs, more BaPs are released from graphene oxide, which induces a hydrophilic pore in the PS layer and shows adverse effect on the PS biophysical function. Translocation of graphene across the PS layer is facilitated by BaP adsorption through segregating it from contact with PS, while translocation of graphene oxide is suppressed by BaP adsorption due to the increase of surface hydrophobicity. Graphene extracts PS molecules from the layer, and the resultant PS depletion declines with graphene oxidation and BaP adsorption. Conclusion GFMs showed high adsorption capacity towards BaPs to form nanocomposites. Upon deposition of GFMs carrying BaPs at the alveolar air–water interface covered by a thin PS layer, the interactions of GFM-PS, GFM-BaP and BaP-PS determined the interfacial processes of BaP solubilization, GFM translocation and PS perturbation

Reflective plasmonic color filters based on lithographically patterned silver nanorod arrays

10.1039/c3nr01419cNanoscale5146243-624

Hexamethyldisiloxane Removal from Biogas Using a Fe₃O₄–Urea-Modified Three-Dimensional Graphene Aerogel

Volatile methyl siloxanes (VMS), which are considered to be the most troublesome impurities in current biogas-cleaning technologies, need to be removed. In this study, we fabricated a series of Fe3O4–urea-modified reduced graphene-oxide aerogels (Fe3O4–urea–rGOAs) by using industrial-grade graphene oxide as the raw material. A fixed-bed dynamic adsorption setup was built, and the adsorption properties of the Fe3O4–urea–rGOAs for hexamethyldisiloxane (L2, as a VMS model pollutant) were studied. The properties of the as-prepared samples were investigated by employing various characterization techniques (SEM, TEM, FTIR, XRD, Raman spectroscopy, and N2 adsorption/desorption techniques). The results showed that the Fe3O4–urea–rGOA–0.4 had a high specific surface area (188 m2 g−1), large porous texture (0.77 cm3 g−1), and the theoretical maximum adsorption capacity for L2 (146.5 mg g−1). The adsorption capacity considerably increased with a decrease in the bed temperature of the adsorbents, as well as with an increase in the inlet concentration of L2. More importantly, the spent Fe3O4–urea–rGOA adsorbent could be readily regenerated and showed an excellent adsorption performance. Thus, the proposed Fe3O4–urea–rGOAs are promising adsorbents for removing the VMS in biogas

Removal of Hexamethyldisiloxane via a Novel Hydrophobic (3–Aminopropyl)Trimethoxysilane-Modified Activated Porous Carbon

Volatile methyl siloxanes (VMS) must be removed because the formation of silica in the combustion process seriously affects the resource utilization of biogas. Herein, a series of APTMS ((3–aminopropyl)trimethoxysilane)-modified activated porous carbon (APC) adsorbents (named APTMS@APC) were prepared for VMS efficient removal. The as-prepared adsorbents were characterized using SEM, FTIR, Raman, X-ray diffraction analyses, and N2 adsorption/desorption. The results showed that the surface modification with APTMS enhanced the hydrophobicity of APC with the water contact angle increasing from 74.3° (hydrophilic) to 127.1° (hydrophobic), and meanwhile improved its texture properties with the SBET increasing from 981 to 1274 m2 g−1. The maximum breakthrough adsorption capacity of APTMS@APC for hexamethyldisiloxane (L2, model pollutant) was 360.1 mg g−1. Effects of an inlet L2 concentration (31.04–83.82 mg L−1) and a bed temperature (0–50 °C) on the removal of L2 were investigated. Meanwhile, after five adsorption–desorption cycles, the APTMS@APC demonstrated a superior cycling performance. This indicated that the hydrophobic APTMS@APC has a great significance to remove VMS