SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

Transfer learning has fundamentally changed the landscape of natural language processing (NLP) research. Many existing state-of-the-art models are first pre-trained on a large text corpus and then fine-tuned on downstream tasks. However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and forget the knowledge of the pre-trained model. To address the above issue in a more principled manner, we propose a new computational framework for robust and efficient fine-tuning for pre-trained language models. Specifically, our proposed framework contains two important ingredients: 1. Smoothness-inducing regularization, which effectively manages the capacity of the model; 2. Bregman proximal point optimization, which is a class of trust-region methods and can prevent knowledge forgetting. Our experiments demonstrate that our proposed method achieves the state-of-the-art performance on multiple NLP benchmarks.Comment: The 58th annual meeting of the Association for Computational Linguistics (ACL 2020

DoSTra: Discovering common behaviors of objects using the duration of staying on each location of trajectories

Since semantic trajectories can discover more semantic meanings of a user\u27s interests without geographic restrictions, research on semantic trajectories has attracted a lot of attentions in recent years. Most existing work discover the similar behavior of moving objects through analysis of their semantic trajectory pattern, that is, sequences of locations. However, this kind of trajectories without considering the duration of staying on a location limits wild applications. For example, Tom and Anne have a common pattern of Home→Restaurant → Company → Restaurant, but they are not similar, since Tom works at Restaurant, sends snack to someone at Company and return to Restaurant while Anne has breakfast at Restaurant, works at Company and has lunch at Restaurant. If we consider duration of staying on each location we can easily to differentiate their behaviors. In this paper, we propose a novel approach for discovering common behaviors by considering the duration of staying on each location of trajectories (DoSTra). Our approach can be used to detect the group that has similar lifestyle, habit or behavior patterns and predict the future locations of moving objects. We evaluate the experiment based on synthetic dataset, which demonstrates the high effectiveness and efficiency of the proposed method

Y2O3 nanosheets as slurry abrasives for chemical-mechanical planarization of copper

Abstract Continued reduction in feature dimension in integrated circuits demands high degree of flatness after chemical mechanical polishing. Here we report using new yttrium oxide (Y2O3) nanosheets as slurry abrasives for chemical-mechanical planarization (CMP) of copper. Results showed that the global planarization was improved by 30% using a slurry containing Y2O3 nanosheets in comparison with a standard industrial slurry. During CMP, the two-dimensional square shaped Y2O3 nanosheet is believed to induce the low friction, the better rheological performance, and the laminar flow leading to the decrease in the within-wafer-non-uniformity, surface roughness, as well as dishing. The application of the two-dimensional nanosheets as abrasive in CMP would increase the manufacturing yield of integrated circuits.</jats:p

Research progress of 3D printed poly (ether ether ketone) in the reconstruction of craniomaxillofacial bone defects

The clinical challenge of bone defects in the craniomaxillofacial region, which can lead to significant physiological dysfunction and psychological distress, persists due to the complex and unique anatomy of craniomaxillofacial bones. These critical-sized defects require the use of bone grafts or substitutes for effective reconstruction. However, current biomaterials and methods have specific limitations in meeting the clinical demands for structural reinforcement, mechanical support, exceptional biological performance, and aesthetically pleasing reconstruction of the facial structure. These drawbacks have led to a growing need for novel materials and technologies. The growing development of 3D printing can offer significant advantages to address these issues, as demonstrated by the fabrication of patient-specific bioactive constructs with controlled structural design for complex bone defects in medical applications using this technology. Poly (ether ether ketone) (PEEK), among a number of materials used, is gaining recognition as a feasible substitute for a customized structure that closely resembles natural bone. It has proven to be an excellent, conformable, and 3D-printable material with the potential to replace traditional autografts and titanium implants. However, its biological inertness poses certain limitations. Therefore, this review summarizes the distinctive features of craniomaxillofacial bones and current methods for bone reconstruction, and then focuses on the increasingly applied 3D printed PEEK constructs in this field and an update on the advanced modifications for improved mechanical properties, biological performance, and antibacterial capacity. Exploring the potential of 3D printed PEEK is expected to lead to more cost-effective, biocompatible, and personalized treatment of craniomaxillofacial bone defects in clinical applications

Prevalence, Antibiotic Susceptibility, and Molecular Characterization of Cronobacter spp. Isolated From Edible Mushrooms in China

Cronobacter spp. are foodborne pathogens that can infect and cause life-threatening diseases in all age groups, particularly in infants and immunocompromised elderly. This study aimed to investigate the prevalence, antibiotic susceptibility, and molecular characteristics of Cronobacter spp. isolates in edible mushrooms collected from 44 cities in China. In total, 668 edible mushrooms were collected from traditional retail markets and supermarkets and were analyzed by quantitative methods, PCR-based serotyping, multilocus sequence typing (MLST), and antibiotic susceptibility testing. Among the 668 samples tested, 89 (13.32%) were positive for Cronobacter spp., and the contamination levels exceeded the 110 most probable number (MPN)/g in 13.48% (12/89) of the samples. Flammulina velutipes samples had the highest contamination rate of 17.54% (37/211), whereas Hypsizygus marmoreus samples had the lowest contamination rate of 3.28% (2/61). Ten serotypes were identified among 115 isolates, of which the C. sakazakii serogroup O1 (n = 32) was the primary serotype. MLST indicated that there was quite high genetic diversity in Cronobacter spp. and 72 sequence types were identified, 17 of which were new. Notably, C. sakazakii ST148 (n = 10) was the most prevalent, followed by C. malonaticus ST7 (n = 5). Antibiotic susceptibility testing revealed that the majority of Cronobacter spp. strains were susceptible to the 16 antibiotics tested. However, a portion of isolates exhibited relatively high resistance to cephalothin, with resistance and intermediate rates of 93.91 and 6.09%, respectively. One isolate (cro300A) was multidrug-resistant, with resistance to five antibiotics. Overall, this large-scale study revealed the relatively high prevalence and high genetic diversity of Cronobacter spp. on edible mushrooms in China, indicating a potential public health concern. To our knowledge, this is the first large-scale and systematic study on the prevalence of Cronobacter spp. on edible mushrooms in China, and the findings can provide valuable information that can guide the establishment of effective measures for the control and precaution of Cronobacter spp on edible mushrooms during production processes