TransNet: Transaction Networks for tabular data

The present disclosure relates to a technique for implementing data loading in hierarchical pipeline for maximum utilization of computer resources (like GPU, etc.) and to obtain effective prediction performance from the numerical and categorical features. The technique involves dividing the data into chunks of data, batching them to form tensors and loading the batched data to the framework to be executed on the GPU. The framework generates embeddings for the numerical and categorical features identified from the batched data and determines the interactions between the generated embeddings to provide a predictive output

AMSP: Reducing Communication Overhead of ZeRO for Efficient LLM Training

Training large language models (LLMs) encounters challenges in GPU memory consumption due to the high memory requirements of model states. The widely used Zero Redundancy Optimizer (ZeRO) addresses this issue through strategic sharding but introduces communication challenges at scale. To tackle this problem, we propose AMSP, a system designed to optimize ZeRO for scalable LLM training. AMSP incorporates three flexible sharding strategies: Full-Replica, Full-Sharding, and Partial-Sharding, and allows each component within the model states (Parameters, Gradients, Optimizer States) to independently choose a sharding strategy as well as the device mesh. We conduct a thorough analysis of communication costs, formulating an optimization problem to discover the optimal sharding strategy. Additionally, AMSP optimizes distributed LLM training by efficiently overlapping communication with computation. Evaluations demonstrate up to 52\% Model FLOPs Utilization (MFU) when training the LLaMA-based model on 1024 GPUs, resulting in a 1.56 times improvement in training throughput compared to newly proposed systems like MiCS and ZeRO++

InternEvo: Efficient Long-sequence Large Language Model Training via Hybrid Parallelism and Redundant Sharding

Large language models (LLMs) with long sequences begin to power more and more fundamentally new applications we use every day. Existing methods for long-sequence LLM training are neither efficient nor compatible with commonly-used training algorithms such as FlashAttention. We design InternEvo to address these issues. InternEvo decouples all of the sharding dimensions into a new hierarchical space, and systematically analyzes the memory and communication cost of LLM training. Then, it generates an effective hybrid parallelism strategy. We design a new selective overlap mechanism to mitigate the communication overhead introduced by the hybrid parallelism. We also implement memory management techniques to reduce GPU memory fragmentation. Evaluation results show that InternEvo generates parallelization strategies that match or outperform existing methods in model FLOPs utilization

Rheological properties and structural features of coconut milk emulsions stabilized with maize kernels and starch

peer-reviewedIn this study, maize kernels and starch with different amylose contents at the same concentration were added to coconut milk. The nonionic composite surfactants were used to prepare various types of coconut milk beverages with optimal stability, and their fluid properties were studied. The steady and dynamic rheological property tests show that the loss modulus (G″) of coconut milk is larger than the storage modulus (G′), which is suitable for the pseudoplastic fluid model and has a shear thinning effect. As the droplet size of the coconut milk fluid changed by the addition of maize kernels and starch, the color intensity, ζ-potential, interfacial tension and stability of the sample significantly improved. The addition of the maize kernels significantly reduced the size of the droplets (p < 0.05). The potential values of zeta (ζ) and the surface tension of the coconut milk increased. Based on the differential scanning calorimetry (DSC) measurement, the addition of maize kernels leads to an increase in the transition temperature, especially in samples with a high amylose content. The higher transition temperature can be attributed to the formation of some starches and lipids and the partial denaturation of proteins in coconut milk, but phase separation occurs. These results may be helpful for determining the properties of maize kernels in food-containing emulsions (such as sauces, condiments, and beverages) that achieve the goal of physical stability

CIDO, a community-based ontology for coronavirus disease knowledge and data integration, sharing, and analysis

Ontologies, as the term is used in informatics, are structured vocabularies comprised of human- and computer-interpretable terms and relations that represent entities and relationships. Within informatics fields, ontologies play an important role in knowledge and data standardization, representation, integra- tion, sharing and analysis. They have also become a foundation of artificial intelligence (AI) research. In what follows, we outline the Coronavirus Infectious Disease Ontology (CIDO), which covers multiple areas in the domain of coronavirus diseases, including etiology, transmission, epidemiology, pathogenesis, diagnosis, prevention, and treatment. We emphasize CIDO development relevant to COVID-19

Classification of temporal lobe epilepsy based on neuropsychological tests and exploration of its underlying neurobiology

ObjectiveTo assist improving long-term postoperative seizure-free rate, we aimed to use machine learning algorithms based on neuropsychological data to differentiate temporal lobe epilepsy (TLE) from extratemporal lobe epilepsy (extraTLE), as well as explore the relationship between magnetic resonance imaging (MRI) and neuropsychological tests.MethodsTwenty-three patients with TLE and 23 patients with extraTLE underwent neuropsychological tests and MRI scans before surgery. The least absolute shrinkage and selection operator were firstly employed for feature selection, and a machine learning approach with neuropsychological tests was employed to classify TLE using leave-one-out cross-validation. A generalized linear model was used to analyze the relationship between brain alterations and neuropsychological tests.ResultsWe found that logistic regression with the selected neuropsychological tests generated classification accuracies of 87.0%, with an area under the receiver operating characteristic curve (AUC) of 0.89. Three neuropsychological tests were acquired as significant neuropsychological signatures for the diagnosis of TLE. We also found that the Right-Left Orientation Test difference was related to the superior temporal and the banks of the superior temporal sulcus (bankssts). The Conditional Association Learning Test (CALT) was associated with the cortical thickness difference in the lateral orbitofrontal area between the two groups, and the Component Verbal Fluency Test was associated with the cortical thickness difference in the lateral occipital cortex between the two groups.ConclusionThese results showed that machine learning-based classification with the selected neuropsychological data can successfully classify TLE with high accuracy compared to previous studies, which could provide kind of warning sign for surgery candidate of TLE patients. In addition, understanding the mechanism of cognitive behavior by neuroimaging information could assist doctors in the presurgical evaluation of TLE

Association between genetically proxied glucosamine and risk of cancer and non-neoplastic disease: A Mendelian randomization study

IntroductionObservational investigations have examined the impact of glucosamine use on the risk of cancer and non-neoplastic diseases. However, the findings from these studies face limitations arising from confounding variables, reverse causation, and conflicting reports. Consequently, the establishment of a causal relationship between habitual glucosamine consumption and the risk of cancer and non-neoplastic diseases necessitates further investigation.MethodsFor Mendelian randomization (MR) investigation, we opted to employ single-nucleotide polymorphisms (SNPs) as instruments that exhibit robust associations with habitual glucosamine consumption. We obtained the corresponding effect estimates of these SNPs on the risk of cancer and non-neoplastic diseases by extracting summary data for genetic instruments linked to 49 varied cancer types amounting to 378,284 cases and 533,969 controls, as well as 20 non-neoplastic diseases encompassing 292,270 cases and 842,829 controls. Apart from the primary analysis utilizing inverse-variance weighted MR, we conducted two supplementary approaches to account for potential pleiotropy (MR-Egger and weighted median) and assessed their respective MR estimates. Furthermore, the results of the leave-one-out analysis revealed that there were no outlying instruments.ResultsOur results suggest divergence from accepted biological understanding, suggesting that genetically predicted glucosamine utilization may be linked to an increased vulnerability to specific illnesses, as evidenced by increased odds ratios and confidence intervals (95% CI) for diseases, such as malignant neoplasm of the eye and adnexa (2.47 [1.34–4.55]), benign neoplasm of the liver/bile ducts (2.12 [1.32–3.43]), benign neoplasm of the larynx (2.01 [1.36–2.96]), melanoma (1.74 [1.17–2.59]), follicular lymphoma (1.50 [1.06–2.11]), autoimmune thyroiditis (2.47 [1.49–4.08]), and autoimmune hyperthyroidism (1.93 [1.17–3.18]). In contrast to prior observational research, our genetic investigations demonstrate a positive correlation between habitual glucosamine consumption and an elevated risk of sigmoid colon cancer, lung adenocarcinoma, and benign neoplasm of the thyroid gland.ConclusionCasting doubt on the purported purely beneficial association between glucosamine ingestion and prevention of neoplastic and non-neoplastic diseases, habitual glucosamine ingestion exhibits dichotomous effects on disease outcomes. Endorsing the habitual consumption of glucosamine as a preventative measure against neoplastic and non-neoplastic diseases cannot be supported