UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining

Pretrained multilingual large language models have typically used heuristic temperature-based sampling to balance between different languages. However previous work has not systematically evaluated the efficacy of different pretraining language distributions across model scales. In this paper, we propose a new sampling method, UniMax, that delivers more uniform coverage of head languages while mitigating overfitting on tail languages by explicitly capping the number of repeats over each language's corpus. We perform an extensive series of ablations testing a range of sampling strategies on a suite of multilingual benchmarks, while varying model scale. We find that UniMax outperforms standard temperature-based sampling, and the benefits persist as scale increases. As part of our contribution, we release: (i) an improved and refreshed mC4 multilingual corpus consisting of 29 trillion characters across 107 languages, and (ii) a suite of pretrained umT5 model checkpoints trained with UniMax sampling

A STUDY ON THE GRIP FORCE DURING PUTTING STROKE

There are lots of variables to affect the control of ball movement during golf putting. Among several variables, it is believed that grip force during putting stroke is one of the important variables. However, there is not much quantitative evidence from published literature (Delay 1997, Gwyn 1993). Therefore, the purpose of this study was to quantify the grip force by comparing putts performed by elite and novice golfers and to identify the relationship between kinematic parameters and the grip force at 16 different parts of subjects’ right and left hand at each putting phase

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

We study the design decisions of publicly available instruction tuning methods, and break down the development of Flan 2022 (Chung et al., 2022). Through careful ablation studies on the Flan Collection of tasks and methods, we tease apart the effect of design decisions which enable Flan-T5 to outperform prior work by 3-17%+ across evaluation settings. We find task balancing and enrichment techniques are overlooked but critical to effective instruction tuning, and in particular, training with mixed prompt settings (zero-shot, few-shot, and chain-of-thought) actually yields stronger (2%+) performance in all settings. In further experiments, we show Flan-T5 requires less finetuning to converge higher and faster than T5 on single downstream tasks, motivating instruction-tuned models as more computationally-efficient starting checkpoints for new tasks. Finally, to accelerate research on instruction tuning, we make the Flan 2022 collection of datasets, templates, and methods publicly available at https://github.com/google-research/FLAN/tree/main/flan/v2

Electric field control of nonvolatile four-state magnetization at room temperature

We find the realization of large converse magnetoelectric (ME) effects at room temperature in a multiferroic hexaferrite Ba$_{0.52}$Sr$_{2.48}$Co$_{2}$Fe$_{24}$O$_{41}$ single crystal, in which rapid change of electric polarization in low magnetic fields (about 5 mT) is coined to a large ME susceptibility of 3200 ps/m. The modulation of magnetization then reaches up to 0.62 $\mu$$_{B}$/f.u. in an electric field of 1.14 MV/m. We find further that four ME states induced by different ME poling exhibit unique, nonvolatile magnetization versus electric field curves, which can be approximately described by an effective free energy with a distinct set of ME coefficients

Preclinical Analysis of Irreversible Electroporation on Rat Liver Tissues Using a Microfabricated Electroporator

A microfabricated electroporator (MFE) for the irreversible electroporation (IRE) of tissues has been developed by miniaturizing a clinical electroporator with a two-needle array while keeping the same electric field strength distribution. Since IRE was brought to special attention as one of the local tissue ablation techniques to treat tumors, many preclinical studies have been conducted to investigate the efficacy of IRE on animal tissues. However, some technical difficulties have been frequently encountered due to the macroscale dimension of clinical electroporators, particularly in experiments on small animal models such as the mouse or rat. Here, the MFE was proposed to solve the associated problems, resulting in time-and cost-effective experimental procedures. With the developed MFE, the effect of IRE on rat liver tissues was analyzed with time by immunohistological stainings and electrical measurement, and the experimental results were compared with those operated with the corresponding real-scale clinical electroporator.Choi YS, 2009, ANAL CHEM, V81, P3517, DOI 10.1021/ac900055rMaor E, 2009, PLOS ONE, V4, DOI 10.1371/journal.pone.0004757Pavlin M, 2008, BIOELECTROCHEMISTRY, V74, P38, DOI 10.1016/j.bioelechem.2008.04.016Sersa G, 2008, EJSO-EUR J SURG ONC, V34, P232, DOI 10.1016/j.ejso.2007.05.016Rubinsky B, 2007, TECHNOL CANCER RES T, V6, P255Onik G, 2007, TECHNOL CANCER RES T, V6, P295Al-Sakere B, 2007, TECHNOL CANCER RES T, V6, P301Maor E, 2007, TECHNOL CANCER RES T, V6, P307Garon EB, 2007, INT J CANCER, V121, P675, DOI 10.1002/ijc.22723Esser AT, 2007, TECHNOL CANCER RES T, V6, P261Lee EW, 2007, TECHNOL CANCER RES T, V6, P287Kimelman N, 2007, TISSUE ENG, V13, P1135, DOI 10.1089/ten.2007.0096Lavee J, 2007, HEART SURG FORUM, V10, pE162, DOI 10.1532/HSF98.20061202Rubinsky B, 2007, TECHNOL CANCER RES T, V6, P37Liu L, 2006, CANCER RES, V66, P11851, DOI 10.1158/0008-5472.CAN-06-1377Marty M, 2006, EJC SUPPL, V4, P3, DOI 10.1016/j.ejcsup.2006.08.002Sersa G, 2006, EJC SUPPL, V4, P52, DOI 10.1016/j.ejcsup.2006.08.007Edd JF, 2006, IEEE T BIO-MED ENG, V53, P1409, DOI [10.1109/TBME.2006.873745, 10.1109/TMBE.2006.873745]Miller L, 2005, TECHNOL CANCER RES T, V4, P699Sel D, 2005, IEEE T BIO-MED ENG, V52, P816, DOI 10.1109/TBME.2005.845212Davalos RV, 2005, ANN BIOMED ENG, V33, P223, DOI 10.1007/s10439-005-8981-8Pliquett U, 2004, BIOELECTROCHEMISTRY, V62, P83, DOI 10.1016/j.biolechem.2003.11.001Davalos RV, 2003, BIOELECTROCHEMISTRY, V61, P99, DOI 10.1016/j.bioelechem.2003.07.001Weaver JC, 2003, IEEE T DIELECT EL IN, V10, P754, DOI 10.1109/TDEI.2003.1237325Gothelf A, 2003, CANCER TREAT REV, V29, P371, DOI 10.1016/S0305-7372(03)00073-2Gehl J, 2003, ACTA PHYSIOL SCAND, V177, P437Leu JI, 2003, J CLIN INVEST, V111, P129, DOI 10.1172/JCI200316712Deng ZS, 2001, PHYSICA A, V300, P521Ryttsen F, 2000, BIOPHYS J, V79, P1993Dev SB, 2000, IEEE T PLASMA SCI, V28, P206, DOI 10.1109/27.842905Duffy DC, 1998, ANAL CHEM, V70, P4974Boone K, 1997, J MED ENG TECHNOL, V21, P201Weaver JC, 1996, BIOELECTROCH BIOENER, V41, P135*I LAB AN RES NAT, 1996, GUID CAR US LAB ANABIDOR IG, 1993, BIOCHIM BIOPHYS ACTA, V1152, P207DILLER KR, 1992, MODELING BIOHEAT TRAMIR LM, 1991, CR ACAD SCI III-VIE, V313, P613DUCK FA, 1990, PHYS PROPERTIES ISSUKINOSITA K, 1979, BIOCHIM BIOPHYS ACTA, V554, P479PENNES HH, 1948, J APPL PHYSIOL, V1, P93

Effects of mealworm (Tenebrio molitor) larvae hydrolysate on nutrient ileal digestibility in growing pigs compared to those of defatted mealworm larvae meal, fermented poultry by-product, and hydrolyzed fish soluble

Objective: To investigate effect of mealworm (Tenebrio molitor) larvae hydrolysate on nutrient ileal digestibility compared to those of dried mealworm larvae meal, fermented poultry by-product, and hydrolyzed fish soluble in growing pigs. Methods: A total of 12 crossbred ([LandracexYorkshire]xDuroc) growing pigs with average body weight of 28.70 +/- 0.32 kg were surgically equipped with simple T-cannulas. A total of 12 pigs were assigned to individual metabolic crates and allotted to one of four treatments with 3 replicates in a fully randomized design. Results: Apparent ileal digestibility (AID) of dry matter (DM) was the highest in pigs fed HML diet. AIDs of crude protein (CP) were higher in pigs fed HML and DMLM diets than those in pigs fed the other two diets. AID of total amino acid was higher (p = 0.06) in pigs fed HML diet. AIDs of lysine (Lys), methionine (Met), and threonine (Thr) were similar in pigs fed DMLM and HML diets, but were higher (p = 0.05, p<0.05, and p = 0.05, respectively) than those in pigs fed FPBM or HFS diet. Pigs fed HML diet had higher standardized ileal digestibilities (SIDs) of DM and CP (p<0.05 and p<0.05, respectively) compared to pigs fed the other FPBM and HFS diets. SIDs of total amino acid were not different (p = 0.06) between treatments. For SIDs of Lys, Met, and Thr, pigs fed HML and DMLM diets showed higher SIDs (p = 0.05, p<0.05, and p<0.05, respectively) than pigs fed FPBM and HFS diets. SIDs of non-essential amino acids (aspartic acid, glycine, and alanine) were higher (p<0.05, p< 0.05, and p<0.05, respectively) in pigs fed HML, FPBM, and DMLM diets than those in pigs fed the HFS diet. AID and SID of glutamic acid were higher in pigs fed HML and FPBM diets. Conclusion: In conclusion, dietary supplementation of mealworm larvae hydrolysate had higher digestibility in DM, CP, Lys, Met, and Thr compared to dietary supplementation with fermented poultry by-product and hydrolyzed fish soluble.Y

Extrasinonasal infiltrative process associated with a sinonasal fungus ball: does it mean invasive fungal sinusitis?

PURPOSE:Invasive fungal sinusitis (IFS) has rarely been reported to develop from non-IFS. The purpose of this study was to disclose the nature of the extrasinonasal infiltrative process in the presence of a sinonasal fungus ball (FB).METHODS:We retrospectively reviewed the medical records, computed tomography, magnetic resonance images of 13 patients with sinonasal FB and the extrasinonasal infiltrative process. Based on histology and clinical course, we divided the extrasinonasal infiltrative process into IFS and the nonfungal inflammatory/infectious process (NFIP). The images were analyzed with particular attention to the presence of cervicofacial tissue infarction (CFTI).RESULTS:Of the 13 patients, IFS was confirmed in only one, while the remaining 12 were diagnosed to have presumed NFIP. One patient with IFS died shortly after diagnosis. In contrast, all 12 patients with presumed NFIP, except one, survived during a mean follow-up of 17 months. FB was located in the maxillary sinus (n=4), sphenoid sinus (n=8), and both sinuses (n=1). Bone defect was found in five patients, of whom four had a defect in the sphenoid sinus. Various sites were involved in the extrasinonasal infiltrative process, including the orbit (n=10), intracranial cavity (n=9), and soft tissues of the face and neck (n=7). CFTI was recognized only in one patient with IFS.CONCLUSION:In most cases, the extrasinonasal infiltrative process in the presence of sinonasal FB did not seem to be caused by IFS but probably by NFIP. In our study, there were more cases of invasive changes with the sphenoid than with the maxillary FB