7,433 research outputs found
ASP: Automatic Selection of Proxy dataset for efficient AutoML
Deep neural networks have gained great success due to the increasing amounts
of data, and diverse effective neural network designs. However, it also brings
a heavy computing burden as the amount of training data is proportional to the
training time. In addition, a well-behaved model requires repeated trials of
different structure designs and hyper-parameters, which may take a large amount
of time even with state-of-the-art (SOTA) hyper-parameter optimization (HPO)
algorithms and neural architecture search (NAS) algorithms. In this paper, we
propose an Automatic Selection of Proxy dataset framework (ASP) aimed to
dynamically find the informative proxy subsets of training data at each epoch,
reducing the training data size as well as saving the AutoML processing time.
We verify the effectiveness and generalization of ASP on CIFAR10, CIFAR100,
ImageNet16-120, and ImageNet-1k, across various public model benchmarks. The
experiment results show that ASP can obtain better results than other data
selection methods at all selection ratios. ASP can also enable much more
efficient AutoML processing with a speedup of 2x-20x while obtaining better
architectures and better hyper-parameters compared to utilizing the entire
dataset.Comment: This paper was actually finished in 202
KwaiYiiMath: Technical Report
Recent advancements in large language models (LLMs) have demonstrated
remarkable abilities in handling a variety of natural language processing (NLP)
downstream tasks, even on mathematical tasks requiring multi-step reasoning. In
this report, we introduce the KwaiYiiMath which enhances the mathematical
reasoning abilities of KwaiYiiBase1, by applying Supervised Fine-Tuning (SFT)
and Reinforced Learning from Human Feedback (RLHF), including on both English
and Chinese mathematical tasks. Meanwhile, we also constructed a small-scale
Chinese primary school mathematics test set (named KMath), consisting of 188
examples to evaluate the correctness of the problem-solving process generated
by the models. Empirical studies demonstrate that KwaiYiiMath can achieve
state-of-the-art (SOTA) performance on GSM8k, CMath, and KMath compared with
the similar size models, respectively.Comment: technical report. arXiv admin note: text overlap with
arXiv:2306.16636 by other author
Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization
Recently, the remarkable advance of the Large Language Model (LLM) has
inspired researchers to transfer its extraordinary reasoning capability to both
vision and language data. However, the prevailing approaches primarily regard
the visual input as a prompt and focus exclusively on optimizing the text
generation process conditioned upon vision content by a frozen LLM. Such an
inequitable treatment of vision and language heavily constrains the model's
potential. In this paper, we break through this limitation by representing both
vision and language in a unified form. Specifically, we introduce a
well-designed visual tokenizer to translate the non-linguistic image into a
sequence of discrete tokens like a foreign language that LLM can read. The
resulting visual tokens encompass high-level semantics worthy of a word and
also support dynamic sequence length varying from the image. Coped with this
tokenizer, the presented foundation model called LaVIT can handle both image
and text indiscriminately under the same generative learning paradigm. This
unification empowers LaVIT to serve as an impressive generalist interface to
understand and generate multi-modal content simultaneously. Extensive
experiments further showcase that it outperforms the existing models by a large
margin on massive vision-language tasks. Our code and models will be available
at https://github.com/jy0205/LaVIT
Knowledge, attitudes and practices relating to influenza A(H7N9) risk among live poultry traders in Guangzhou City, China
published_or_final_versio
Recommended from our members
Possible Luttinger liquid behavior of edge transport in monolayer transition metal dichalcogenide crystals.
In atomically-thin two-dimensional (2D) semiconductors, the nonuniformity in current flow due to its edge states may alter and even dictate the charge transport properties of the entire device. However, the influence of the edge states on electrical transport in 2D materials has not been sufficiently explored to date. Here, we systematically quantify the edge state contribution to electrical transport in monolayer MoS2/WSe2 field-effect transistors, revealing that the charge transport at low temperature is dominated by the edge conduction with the nonlinear behavior. The metallic edge states are revealed by scanning probe microscopy, scanning Kelvin probe force microscopy and first-principle calculations. Further analyses demonstrate that the edge-state dominated nonlinear transport shows a universal power-law scaling relationship with both temperature and bias voltage, which can be well explained by the 1D Luttinger liquid theory. These findings demonstrate the Luttinger liquid behavior in 2D materials and offer important insights into designing 2D electronics
Exploring the bi-directional relationship and shared genes between depression and stroke via NHANES and bioinformatic analysis
Background: Stroke and depression are the two most common causes of disability worldwide. Growing evidence suggests a bi-directional relationship between stroke and depression, whereas the molecular mechanisms underlying stroke and depression are not well understood. The objectives of this study were to identify hub genes and biological pathways related to the pathogenesis of ischemic stroke (IS) and major depressive disorder (MDD) and to evaluate the infiltration of immune cells in both disorders.Methods: Participants from the United States National Health and Nutritional Examination Survey (NHANES) 2005–2018 were included to evaluate the association between stroke and MDD. Two differentially expressed genes (DEGs) sets extracted from GSE98793 and GSE16561 datasets were intersected to generate common DEGs, which were further screened out in cytoHubba to identify hub genes. GO, KEGG, Metascape, GeneMANIA, NetworkAnalyst, and DGIdb were used for functional enrichment, pathway analysis, regulatory network analysis, and candidate drugs analysis. ssGSEA algorithm was used to analyze the immune infiltration.Results: Among the 29706 participants from NHANES 2005–2018, stroke was significantly associated with MDD (OR = 2.79,95% CI:2.26–3.43, p < 0.0001). A total of 41 common upregulated genes and eight common downregulated genes were finally identified between IS and MDD. Enrichment analysis revealed that the shared genes were mainly involved in immune response and immune-related pathways. A protein-protein interaction (PPI) was constructed, from which ten (CD163, AEG1, IRAK3, S100A12, HP, PGLYRP1, CEACAM8, MPO, LCN2, and DEFA4) were screened. In addition, gene-miRNAs, transcription factor-gene interactions, and protein-drug interactions coregulatory networks with hub genes were also identified. Finally, we observed that the innate immunity was activated while acquired immunity was suppressed in both disorders.Conclusion: We successfully identified the ten hub shared genes linking the IS and MDD and constructed the regulatory networks for them that could serve as novel targeted therapy for the comorbidities
Control of magnetic anisotropy by orbital hybridization in (La0.67Sr0.33MnO3)n/(SrTiO3)n superlattice
The asymmetry of chemical nature at the hetero-structural interface offers an
unique opportunity to design desirable electronic structure by controlling
charge transfer and orbital hybridization across the interface. However, the
control of hetero-interface remains a daunting task. Here, we report the
modulation of interfacial coupling of (La0.67Sr0.33MnO3)n/(SrTiO3)n
superlattices by manipulating the periodic thickness with n unit cells of
SrTiO3 and n unit cells La0.67Sr0.33MnO3. The easy axis of magnetic anisotropy
rotates from in-plane (n = 10) to out-of-plane (n = 2) orientation at 150 K.
Transmission electron microscopy reveals enlarged tetragonal ratio > 1 with
breaking of volume conservation around the (La0.67Sr0.33MnO3)n/(SrTiO3)n
interface, and electronic charge transfer from Mn to Ti 3d orbitals across the
interface. Orbital hybridization accompanying the charge transfer results in
preferred occupancy of 3d3z2-r2 orbital at the interface, which induces a
stronger electronic hopping integral along the out-of-plane direction and
corresponding out-of-plane magnetic easy axis for n = 2. We demonstrate that
interfacial orbital hybridization in superlattices of strongly correlated
oxides may be a promising approach to tailor electronic and magnetic properties
in device applications
Spatio-Temporal Characteristics of Global Warming in the Tibetan Plateau during the Last 50 Years Based on a Generalised Temperature Zone - Elevation Model
Temperature is one of the primary factors influencing the climate and ecosystem, and examining its change and fluctuation could elucidate the formation of novel climate patterns and trends. In this study, we constructed a generalised temperature zone elevation model (GTEM) to assess the trends of climate change and temporal-spatial differences in the Tibetan Plateau (TP) using the annual and monthly mean temperatures from 1961-2010 at 144 meteorological stations in and near the TP. The results showed the following: (1) The TP has undergone robust warming over the study period, and the warming rate was 0.318°C/decade. The warming has accelerated during recent decades, especially in the last 20 years, and the warming has been most significant in the winter months, followed by the spring, autumn and summer seasons. (2) Spatially, the zones that became significantly smaller were the temperature zones of -6°C and -4°C, and these have decreased 499.44 and 454.26 thousand sq km from 1961 to 2010 at average rates of 25.1% and 11.7%, respectively, over every 5-year interval. These quickly shrinking zones were located in the northwestern and central TP. (3) The elevation dependency of climate warming existed in the TP during 1961-2010, but this tendency has gradually been weakening due to more rapid warming at lower elevations than in the middle and upper elevations of the TP during 1991-2010. The higher regions and some low altitude valleys of the TP were the most significantly warming regions under the same categorizing criteria. Experimental evidence shows that the GTEM is an effective method to analyse climate changes in high altitude mountainous regions
- …