How to Fine-Tune BERT for Text Classification?

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets

Characterization of the Differential Aroma Compounds among 10 Different Kinds of Premium Soy Sauce

Investigation of the aroma differences among different kinds of soy sauces is beneficial for controlling their flavor quality and processing improvement from the perspectives of raw materials and brewing techniques. The aroma compounds in ten premium soy sauces (CB, HT1, HT2, LH, LJJ1, LJJ2, QH, XH1, XH2, WZ) were qualitative and quantitative analyzed by solid phase extraction and solid-phase microextraction combined with gas chromatography-mass spectrometry (GC-MS). The contributions of aroma compounds to the aroma characteristics of premium soy sauce was determined by sensory evaluation, calculation of aroma activity value (OAV) and partial least squares regression analysis (PLSR). A total of 86 volatile compounds were identified in 10 premium soy sauces, 44 of them were both detected in 10 soy sauce. The 30 aroma compounds with OAV≥1 were detected, the 5-ethyl-4-hydroxy-2-methyl-3(2H)-furanone showed the highest OAV (373~4698), followed by 4-methoxy-2,5-dimethyl-3(2H)-furanone (0~1473). WZ soy sauce had a strong smoky aroma due to the highest variety of phenolic and ketone compounds. The overall aroma profile of CB soy sauce was the weakest with the lowest concentration of ethanol (25.775 μg/L), but the highest content of pyrazine compounds (182.796 μg/L), of which 2,6-dimethylpyrazine was 66.256 μg/L. XH1 soy sauce had a strong sauce aroma and alcoholic notes, due to the highest ethanol content (147.257 μg/L) and higher phenolic content, for example the concentration of 4-ethyl-2-methoxyphenol (18240.479 μg/L) was the highest. XH2 soy sauce had a strong malty aroma. The content of 2-methyl-1-propanol (51.223 μg/L) and 2,3-butanediol (57921.798 μg/L) in LH soy sauce was the highest among others. The content of 1-octen-3-ol (61.219 μg/L) in HT1 soy sauce was the highest. Combination of OAV and PLSR analysis confirmed the ethyl acetate, 3-hydroxy-2-butanone, 2,3-butanediol, 3-ethyl-2,5-dimethylpyrazine, 4-methoxy-2,5-dimethyl-3(2H)-furanone, 4-ethylguaiacol and 4-ethylphenol were the key aroma-active components that contribute to the aroma differences among 10 kinds of premium soy sauce

Absolute frequency measurements with a robust, transportable ^{40}Ca^{+} optical clock

We constructed a transportable 40Ca+ optical clock (with an estimated minimum systematic shift uncertainty of 1.3*10^(-17) and a stability of 5*10^(-15)/sqrt{tau} ) that can operate outside the laboratory. We transported it from the Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences, Wuhan to the National Institute of Metrology, Beijing. The absolute frequency of the 729 nm clock transition was measured for up to 35 days by tracing its frequency to the second of International System of Units. Some improvements were implemented in the measurement process, such as the increased effective up-time of 91.3 % of the 40Ca+ optical clock over a 35-day-period, the reduced statistical uncertainty of the comparison between the optical clock and hydrogen maser, and the use of longer measurement times to reduce the uncertainty of the frequency traceability link. The absolute frequency measurement of the 40Ca+ optical clock yielded a value of 411042129776400.26 (13) Hz with an uncertainty of 3.2*10^(-16), which is reduced by a factor of 1.7 compared with our previous results. As a result of the increase in the operating rate of the optical clock, the accuracy of 35 days of absolute frequency measurement can be comparable to the best results of different institutions in the world based on different optical frequency measurements.Comment: 15 pages, 5 figure

Sleep homeostasis during daytime food entrainment in mice

24h rhythms of physiology and behavior are driven by the environment and an internal endogenous timing system. Daily restricted feeding (RF) in nocturnal rodents during their inactive phase initiates food anticipatory activity (FAA) and a reorganisation of the typical 24h sleep-wake structure. Here, we investigate the effects of daytime feeding, where food access was restricted to 4h during the light period ZT4-8 (Zeitgeber time; ZT0 is lights on), on sleep-wake architecture and sleep homeostasis in mice. Following 10 days of RF, mice were returned to ad libitum feeding. To mimic the spontaneous wakefulness associated with FAA and daytime feeding, mice were then sleep deprived between ZT3-6. While the amount of wake increased during FAA and subsequent feeding, total wake time over 24h remained stable as the loss of sleep in the light phase was compensated for by an increase in sleep in the dark phase. Interestingly, sleep which followed spontaneous wake episodes during the dark period and the extended period of wake associated with FAA, exhibited lower levels of slow-wave activity (SWA) when compared to baseline or after sleep deprivation, despite a similar duration of waking. This suggests an evolutionary mechanism of reducing sleep drive during negative energy balance to enable greater arousal for food seeking behaviors. However, the total amount of sleep and SWA accumulated during the 24h was similar between baseline and RF. In summary, our study suggests that despite substantial changes in the daily distribution and quality of wake induced by RF, sleep homeostasis is maintained.</p

Trio-Based Deep Sequencing Reveals a Low Incidence of Off-Target Mutations in the Offspring of Genetically Edited Goats

Unintended off-target mutations induced by CRISPR/Cas9 nucleases may result in unwanted consequences, which will impede the efficient applicability of this technology for genetic improvement. We have recently edited the goat genome through CRISPR/Cas9 by targeting MSTN and FGF5, which increased muscle fiber diameter and hair fiber length, respectively. Using family trio-based sequencing that allow better discrimination of variant origins, we herein generated offspring from edited goats, and sequenced the members of four family trios (gene-edited goats and their offspring) to an average of ∼36.8× coverage. This data was to systematically examined for mutation profiles using a stringent pipeline that comprehensively analyzed the sequence data for de novo single nucleotide variants, indels, and structural variants from the genome. Our results revealed that the incidence of de novo mutations in the offspring was equivalent to normal populations. We further conducted RNA sequencing using muscle and skin tissues from the offspring and control animals, the differentially expressed genes (DEGs) were related to muscle fiber development in muscles, skin development, and immune responses in skin tissues. Furthermore, in contrast to recently reports of Cas9 triggered p53 expression alterations in cultured cells, we provide primary evidence to show that Cas9-mediated genetic modification does not induce apparent p53 expression changes in animal tissues. This work provides adequate molecular evidence to support the reliability of conducting Cas9-mediated genome editing in large animal models for biomedicine and agriculture

Pathogenic Germline Variants in 10,389 Adult Cancers

We conducted the largest investigation of predisposition variants in cancer to date, discovering 853 pathogenic or likely pathogenic variants in 8% of 10,389 cases from 33 cancer types. Twenty-one genes showed single or cross-cancer associations, including novel associations of SDHA in melanoma and PALB2 in stomach adenocarcinoma. The 659 predisposition variants and 18 additional large deletions in tumor suppressors, including ATM, BRCA1, and NF1, showed low gene expression and frequent (43%) loss of heterozygosity or biallelic two-hit events. We also discovered 33 such variants in oncogenes, including missenses in MET, RET, and PTPN11 associated with high gene expression. We nominated 47 additional predisposition variants from prioritized VUSs supported by multiple evidences involving case-control frequency, loss of heterozygosity, expression effect, and co-localization with mutations and modified residues. Our integrative approach links rare predisposition variants to functional consequences, informing future guidelines of variant classification and germline genetic testing in cancer. A pan-cancer analysis identifies hundreds of predisposing germline variants