ASP: Automatic Selection of Proxy dataset for efficient AutoML

Deep neural networks have gained great success due to the increasing amounts of data, and diverse effective neural network designs. However, it also brings a heavy computing burden as the amount of training data is proportional to the training time. In addition, a well-behaved model requires repeated trials of different structure designs and hyper-parameters, which may take a large amount of time even with state-of-the-art (SOTA) hyper-parameter optimization (HPO) algorithms and neural architecture search (NAS) algorithms. In this paper, we propose an Automatic Selection of Proxy dataset framework (ASP) aimed to dynamically find the informative proxy subsets of training data at each epoch, reducing the training data size as well as saving the AutoML processing time. We verify the effectiveness and generalization of ASP on CIFAR10, CIFAR100, ImageNet16-120, and ImageNet-1k, across various public model benchmarks. The experiment results show that ASP can obtain better results than other data selection methods at all selection ratios. ASP can also enable much more efficient AutoML processing with a speedup of 2x-20x while obtaining better architectures and better hyper-parameters compared to utilizing the entire dataset.Comment: This paper was actually finished in 202

KwaiYiiMath: Technical Report

Recent advancements in large language models (LLMs) have demonstrated remarkable abilities in handling a variety of natural language processing (NLP) downstream tasks, even on mathematical tasks requiring multi-step reasoning. In this report, we introduce the KwaiYiiMath which enhances the mathematical reasoning abilities of KwaiYiiBase1, by applying Supervised Fine-Tuning (SFT) and Reinforced Learning from Human Feedback (RLHF), including on both English and Chinese mathematical tasks. Meanwhile, we also constructed a small-scale Chinese primary school mathematics test set (named KMath), consisting of 188 examples to evaluate the correctness of the problem-solving process generated by the models. Empirical studies demonstrate that KwaiYiiMath can achieve state-of-the-art (SOTA) performance on GSM8k, CMath, and KMath compared with the similar size models, respectively.Comment: technical report. arXiv admin note: text overlap with arXiv:2306.16636 by other author

Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization

Recently, the remarkable advance of the Large Language Model (LLM) has inspired researchers to transfer its extraordinary reasoning capability to both vision and language data. However, the prevailing approaches primarily regard the visual input as a prompt and focus exclusively on optimizing the text generation process conditioned upon vision content by a frozen LLM. Such an inequitable treatment of vision and language heavily constrains the model's potential. In this paper, we break through this limitation by representing both vision and language in a unified form. Specifically, we introduce a well-designed visual tokenizer to translate the non-linguistic image into a sequence of discrete tokens like a foreign language that LLM can read. The resulting visual tokens encompass high-level semantics worthy of a word and also support dynamic sequence length varying from the image. Coped with this tokenizer, the presented foundation model called LaVIT can handle both image and text indiscriminately under the same generative learning paradigm. This unification empowers LaVIT to serve as an impressive generalist interface to understand and generate multi-modal content simultaneously. Extensive experiments further showcase that it outperforms the existing models by a large margin on massive vision-language tasks. Our code and models will be available at https://github.com/jy0205/LaVIT

Knowledge, attitudes and practices relating to influenza A(H7N9) risk among live poultry traders in Guangzhou City, China

published_or_final_versio

Possible Luttinger liquid behavior of edge transport in monolayer transition metal dichalcogenide crystals.

In atomically-thin two-dimensional (2D) semiconductors, the nonuniformity in current flow due to its edge states may alter and even dictate the charge transport properties of the entire device. However, the influence of the edge states on electrical transport in 2D materials has not been sufficiently explored to date. Here, we systematically quantify the edge state contribution to electrical transport in monolayer MoS2/WSe2 field-effect transistors, revealing that the charge transport at low temperature is dominated by the edge conduction with the nonlinear behavior. The metallic edge states are revealed by scanning probe microscopy, scanning Kelvin probe force microscopy and first-principle calculations. Further analyses demonstrate that the edge-state dominated nonlinear transport shows a universal power-law scaling relationship with both temperature and bias voltage, which can be well explained by the 1D Luttinger liquid theory. These findings demonstrate the Luttinger liquid behavior in 2D materials and offer important insights into designing 2D electronics

Control of magnetic anisotropy by orbital hybridization in (La0.67Sr0.33MnO3)n/(SrTiO3)n superlattice

The asymmetry of chemical nature at the hetero-structural interface offers an unique opportunity to design desirable electronic structure by controlling charge transfer and orbital hybridization across the interface. However, the control of hetero-interface remains a daunting task. Here, we report the modulation of interfacial coupling of (La0.67Sr0.33MnO3)n/(SrTiO3)n superlattices by manipulating the periodic thickness with n unit cells of SrTiO3 and n unit cells La0.67Sr0.33MnO3. The easy axis of magnetic anisotropy rotates from in-plane (n = 10) to out-of-plane (n = 2) orientation at 150 K. Transmission electron microscopy reveals enlarged tetragonal ratio > 1 with breaking of volume conservation around the (La0.67Sr0.33MnO3)n/(SrTiO3)n interface, and electronic charge transfer from Mn to Ti 3d orbitals across the interface. Orbital hybridization accompanying the charge transfer results in preferred occupancy of 3d3z2-r2 orbital at the interface, which induces a stronger electronic hopping integral along the out-of-plane direction and corresponding out-of-plane magnetic easy axis for n = 2. We demonstrate that interfacial orbital hybridization in superlattices of strongly correlated oxides may be a promising approach to tailor electronic and magnetic properties in device applications

Spatio-Temporal Characteristics of Global Warming in the Tibetan Plateau during the Last 50 Years Based on a Generalised Temperature Zone - Elevation Model

Temperature is one of the primary factors influencing the climate and ecosystem, and examining its change and fluctuation could elucidate the formation of novel climate patterns and trends. In this study, we constructed a generalised temperature zone elevation model (GTEM) to assess the trends of climate change and temporal-spatial differences in the Tibetan Plateau (TP) using the annual and monthly mean temperatures from 1961-2010 at 144 meteorological stations in and near the TP. The results showed the following: (1) The TP has undergone robust warming over the study period, and the warming rate was 0.318°C/decade. The warming has accelerated during recent decades, especially in the last 20 years, and the warming has been most significant in the winter months, followed by the spring, autumn and summer seasons. (2) Spatially, the zones that became significantly smaller were the temperature zones of -6°C and -4°C, and these have decreased 499.44 and 454.26 thousand sq km from 1961 to 2010 at average rates of 25.1% and 11.7%, respectively, over every 5-year interval. These quickly shrinking zones were located in the northwestern and central TP. (3) The elevation dependency of climate warming existed in the TP during 1961-2010, but this tendency has gradually been weakening due to more rapid warming at lower elevations than in the middle and upper elevations of the TP during 1991-2010. The higher regions and some low altitude valleys of the TP were the most significantly warming regions under the same categorizing criteria. Experimental evidence shows that the GTEM is an effective method to analyse climate changes in high altitude mountainous regions

Protection Evaluation of a Five-Gene-Deleted African Swine Fever Virus Vaccine Candidate Against Homologous Challenge

African swine fever virus (ASFV) represents a serious threat to the global swine industry, and there are no safe or commercially available vaccines. Previous studies have demonstrated that inactivated vaccines do not provide sufficient protection against ASFV and that attenuated vaccines are effective, but raise safety concerns. Here, we first constructed a deletion mutant in which EP153R and EP402R gene clusters were knocked out. Based on the deletion mutant, a further deletion from the MGF_360-12L, MGF_360-13L to MGF_360-14L genes was obtained. The five-genes knockout virus was designated as ASFV-ΔECM3. To investigate the efficacy and safety of the ASFV-ΔECM3 virus as a vaccine candidate, the evaluation of the virus was subsequently carried out in pigs. The results showed that the ASFV-ΔECM3 virus could induce homologous protection against the parental isolate, and no significant clinical signs or viremia were observed. These results show that the contiguous deletion mutant, ASFV-ΔECM3 encompassing the EP153R/EP402R and MGF_360-12L/13L/14L genes, could be a potential live-attenuated vaccine candidate for the prevention of ASFV infection