How to Fine-Tune BERT for Text Classification?

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets

A WENO Algorithm of the Temperature and Ionization Profiles around a Point Source

We develop a numerical solver for radiative transfer problems based on the weighted essentially nonoscillatory (WENO) scheme modified with anti-diffusive flux corrections, in order to solve the temperature and ionization profiles around a point source of photons in the reionization epoch. Algorithms for such simulation must be able to handle the following two features: 1. the sharp profiles of ionization and temperature at the ionizing front (I-front) and the heating front (T-front), and 2. the fraction of neutral hydrogen within the ionized sphere is extremely small due to the stiffness of the rate equations of atom processes. The WENO scheme can properly handle these two features, as it has been shown to have high order of accuracy and good convergence in capturing discontinuities and complicated structures in fluid as well as to be significantly superior over piecewise smooth solutions containing discontinuities. With this algorithm, we show the time-dependence of the preheated shell around a UV photon source. In the first stage the I-front and T-front are coincident, and propagate with almost the speed of light. In later stage, when the frequency spectrum of UV photons is hardened, the speeds of propagation of the ionizing and heating fronts are both significantly less than the speed of light, and the heating front is always beyond the ionizing front. In the spherical shell between the I- and T-fronts, the IGM is heated, while atoms keep almost neutral. The time scale of the preheated shell evolution is dependent on the intensity of the photon source. We also find that the details of the pre-heated shell and the distribution of neutral hydrogen remained in the ionized sphere are actually sensitive to the parameters used. The WENO algorithm can provide stable and robust solutions to study these details.Comment: 24 pages, 7 figures, accepted in New Astronom

Electronic Actuation of a Motorcycle Clutch

ME450 Capstone Design and Manufacturing Experience: Fall 2015The problem that the Michigan Formula SAE Racing Team would like to solve is how to reduce tire slip in straight-line acceleration situations. Solving this will result in faster acceleration times and more points in the competition. As per the sponsor’s request, the solution will be approached through the implementation of clutch-control, a form of launch control using a closed-loop system. The main user requirements of the project include generating at least 10 ft-lbs of torque, avoiding bending moment damage with a safety factor of 1.2, disassembling the system from the vehicle within 20 minutes, and creating a high speed system that can adjust the clutch within .02 seconds. These specifications were used to choose the AmpFlow A28-150 High Performance motor along with the MAE3 Absolute Magnetic Encoder Kit for the final design. A proof-of-concept electrical circuit that is connected to a small DC motor has been built that allows the system to drive current from the power source and protect the motor from backwards flowing current by implementing an opto-isolator. The controls model in Simulink has also been created where mock signals were run through the code to determine the program’s validity. When the motor was connected to the power source, the clutch lever could be actuated through its full range of motion proving that it could generate enough torque. Welding the truss onto the mounting frame resulted in minimal deflection which validated stiffness. In addition, the disassembling/assembling the system took two minutes, well under the limit. The next steps for this project would be to create a more robust circuit involving thicker wiring for the AmpFlow motor, determining coefficient for the PID controller, and testing vehicle lap times with the system integrated into the car to validate that the system results in faster acceleration times.http://deepblue.lib.umich.edu/bitstream/2027.42/117348/1/ME450-F15-Project30-FinalReport.pd

PivotNet: Vectorized Pivot Learning for End-to-end HD Map Construction

Vectorized high-definition map online construction has garnered considerable attention in the field of autonomous driving research. Most existing approaches model changeable map elements using a fixed number of points, or predict local maps in a two-stage autoregressive manner, which may miss essential details and lead to error accumulation. Towards precise map element learning, we propose a simple yet effective architecture named PivotNet, which adopts unified pivot-based map representations and is formulated as a direct set prediction paradigm. Concretely, we first propose a novel point-to-line mask module to encode both the subordinate and geometrical point-line priors in the network. Then, a well-designed pivot dynamic matching module is proposed to model the topology in dynamic point sequences by introducing the concept of sequence matching. Furthermore, to supervise the position and topology of the vectorized point predictions, we propose a dynamic vectorized sequence loss. Extensive experiments and ablations show that PivotNet is remarkably superior to other SOTAs by 5.9 mAP at least. The code will be available soon.Comment: Accepted by ICCV202