How to Fine-Tune BERT for Text Classification?

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets

Study on the Influence of Ultrasonic Vibration on the Specific Energy of Sawing Ceramic

AbstractThe hard as well as brittle constituents are typically difficult-to-machined materials, and this character upsurges the machining cost. Many non-traditional machining methods were developed to improve its cost-effectiveness. Ultrasonic vibration assisted grinding has been improved the processing performance of a variety of brittle materials, and achieved good results in processing application. In this study, engineering ceramic was precisely sawn using a thin diamond blade with or without ultrasonic vibration conditions. During the sawing process, the specific sawing energy was investigated with the measurement of sawing forces to explore the influence of ultrasonic vibration. The results showed that the ultrasonic vibration made a significant reduction in specific sawing energy. The specific sawing energy decreased with the increase of the maximum undeformed chip thickness in both the sawing conditions; however ultrasonic vibration changed the trend of specific sawing energy in normal cutting mode from exponentially decreasing to a good linear decreasing. Under the ultrasonic vibration assisted sawing condition, the impact of the diamond grain on the engineering ceramic caused to much more material removal in brittle fracture mode. The reducing of the plastic transformation also reduced the energy consumption during the engineering ceramic sawing process

PHTrans: Parallelly Aggregating Global and Local Representations for Medical Image Segmentation

The success of Transformer in computer vision has attracted increasing attention in the medical imaging community. Especially for medical image segmentation, many excellent hybrid architectures based on convolutional neural networks (CNNs) and Transformer have been presented and achieve impressive performance. However, most of these methods, which embed modular Transformer into CNNs, struggle to reach their full potential. In this paper, we propose a novel hybrid architecture for medical image segmentation called PHTrans, which parallelly hybridizes Transformer and CNN in main building blocks to produce hierarchical representations from global and local features and adaptively aggregate them, aiming to fully exploit their strengths to obtain better segmentation performance. Specifically, PHTrans follows the U-shaped encoder-decoder design and introduces the parallel hybird module in deep stages, where convolution blocks and the modified 3D Swin Transformer learn local features and global dependencies separately, then a sequence-to-volume operation unifies the dimensions of the outputs to achieve feature aggregation. Extensive experimental results on both Multi-Atlas Labeling Beyond the Cranial Vault and Automated Cardiac Diagnosis Challeng datasets corroborate its effectiveness, consistently outperforming state-of-the-art methods. The code is available at: https://github.com/lseventeen/PHTrans.Comment: 10 pages, 3 figure