Graph-based Multi-ODE Neural Networks for Spatio-Temporal Traffic Forecasting

There is a recent surge in the development of spatio-temporal forecasting models in the transportation domain. Long-range traffic forecasting, however, remains a challenging task due to the intricate and extensive spatio-temporal correlations observed in traffic networks. Current works primarily rely on road networks with graph structures and learn representations using graph neural networks (GNNs), but this approach suffers from over-smoothing problem in deep architectures. To tackle this problem, recent methods introduced the combination of GNNs with residual connections or neural ordinary differential equations (ODE). However, current graph ODE models face two key limitations in feature extraction: (1) they lean towards global temporal patterns, overlooking local patterns that are important for unexpected events; and (2) they lack dynamic semantic edges in their architectural design. In this paper, we propose a novel architecture called Graph-based Multi-ODE Neural Networks (GRAM-ODE) which is designed with multiple connective ODE-GNN modules to learn better representations by capturing different views of complex local and global dynamic spatio-temporal dependencies. We also add some techniques like shared weights and divergence constraints into the intermediate layers of distinct ODE-GNN modules to further improve their communication towards the forecasting task. Our extensive set of experiments conducted on six real-world datasets demonstrate the superior performance of GRAM-ODE compared with state-of-the-art baselines as well as the contribution of different components to the overall performance. The code is available at https://github.com/zbliu98/GRAM-ODEComment: Published at Transactions on Machine Learning Research, 202

Execution-based Code Generation using Deep Reinforcement Learning

The utilization of programming language (PL) models, pretrained on large-scale code corpora, as a means of automating software engineering processes has demonstrated considerable potential in streamlining various code generation tasks such as code completion, code translation, and program synthesis. However, current approaches mainly rely on supervised fine-tuning objectives borrowed from text generation, neglecting specific sequence-level features of code, including but not limited to compilability as well as syntactic and functional correctness. To address this limitation, we propose PPOCoder, a new framework for code generation that combines pretrained PL models with Proximal Policy Optimization (PPO) deep reinforcement learning and employs execution feedback as the external source of knowledge into the model optimization. PPOCoder is transferable across different code generation tasks and PLs. Extensive experiments on three code generation tasks demonstrate the effectiveness of our proposed approach compared to SOTA methods, improving the success rate of compilation and functional correctness over different PLs. Our code can be found at https://github.com/reddy-lab-code-research/PPOCoder

Transformer-based Planning for Symbolic Regression

Symbolic regression (SR) is a challenging task in machine learning that involves finding a mathematical expression for a function based on its values. Recent advancements in SR have demonstrated the effectiveness of pretrained transformer-based models in generating equations as sequences, leveraging large-scale pretraining on synthetic datasets and offering notable advantages in terms of inference time over GP-based methods. However, these models primarily rely on supervised pretraining goals borrowed from text generation and overlook equation-specific objectives like accuracy and complexity. To address this, we propose TPSR, a Transformer-based Planning strategy for Symbolic Regression that incorporates Monte Carlo Tree Search into the transformer decoding process. Unlike conventional decoding strategies, TPSR enables the integration of non-differentiable feedback, such as fitting accuracy and complexity, as external sources of knowledge into the transformer-based equation generation process. Extensive experiments on various datasets show that our approach outperforms state-of-the-art methods, enhancing the model's fitting-complexity trade-off, extrapolation abilities, and robustness to noiseComment: Parshin Shojaee and Kazem Meidani contributed equally to this wor

SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training

In an era where symbolic mathematical equations are indispensable for modeling complex natural phenomena, scientific inquiry often involves collecting observations and translating them into mathematical expressions. Recently, deep learning has emerged as a powerful tool for extracting insights from data. However, existing models typically specialize in either numeric or symbolic domains, and are usually trained in a supervised manner tailored to specific tasks. This approach neglects the substantial benefits that could arise from a task-agnostic unified understanding between symbolic equations and their numeric counterparts. To bridge the gap, we introduce SNIP, a Symbolic-Numeric Integrated Pre-training, which employs joint contrastive learning between symbolic and numeric domains, enhancing their mutual similarities in the pre-trained embeddings. By performing latent space analysis, we observe that SNIP provides cross-domain insights into the representations, revealing that symbolic supervision enhances the embeddings of numeric data and vice versa. We evaluate SNIP across diverse tasks, including symbolic-to-numeric mathematical property prediction and numeric-to-symbolic equation discovery, commonly known as symbolic regression. Results show that SNIP effectively transfers to various tasks, consistently outperforming fully supervised baselines and competing strongly with established task-specific methods, especially in few-shot learning scenarios where available data is limited

HIV infection significantly reduces lipoprotein lipase which remains low after 6 months of antiretroviral therapy

Purpose of the study Fractional clearance rate of apolipoprotein B100-containing lipoproteins is reduced in HIV infection before and after antiretroviral (ARV) treatment [1]. We compared lipoprotein lipase (LPL) activity and gene expression in HIV-positive subjects before and 6 months after ARV with HIV-negative controls. Methods Fasting blood post heparin total and hepatic lipase activity,adiponectin, leptin, insulin, glucose, and lipid measurementswere made in 32 HIV-infected and 15 HIVnegative controls. LPL was estimated by subtractinghepatic lipase from total lipase. Adiponectin, LPL andhormone sensitive lipase (HSL) gene expression weremeasured from iliac crest subcutaneous fat biopsies.Patients were tested before, and 6 months after randomisation to AZT/3TC (n = 15) or TDF/FTC (n = 17) with EFV.Between-group comparison was by Mann-Whitney andpaired samples by the Wilcoxon signed rank tests. Summary of results There were no differences in gender, ethnicity, baseline BMI, regional fat distribution (whole body DEXA) and visceral (VAT) and subcutaneous fat (SAT) measured by abdominal CT scans between controls and patients. Trunk fat/BMI ratio, VAT and VAT:SAT ratio significantly increased after 6-month ARV therapy (p = 0.01). There were no differences between groups in serum NEFA,HOMA and leptin levels. Selected other results are shown in Table 1. Conclusion Post heparin lipoprotein lipase activity is reduced in HIV and does not return to control levels after 6 months of ARV therapy. AZT-containing regimens are associated with a greater increase in LPL, LPL gene expression and plasma adiponectin than TDF

Optimization of Tube Hydroforming Process Using Simulated Annealing Algorithm

AbstractIn this paper, forming parameters of tube hydroforming (THF) process are investigated and optimized using Simulated Annealing optimization algorithm linked with a finite element commercial code. The goal of this research is to obtain the maximum formability of two dimensional (2D) axisymmetric tubes under a failure criteria based on material's forming limit diagram (FLD). The initial approximated pressure loading path is determined by proved theoretical equations. Then the Simulated Annealing algorithm written in Matlab software is combined with a nonlinear structural finite element code ANSYS/ LS-DYNA in order to optimize internal hydraulic pressure. The results are compared by experimental observations and a good agreement was observed between them

Evaluation of the drug solubility and rush ageing on drug release performance of various model drugs from the modified release polyethylene oxide matrix tablets

Hydrophilic matrix systems are currently some of the most widely used drug delivery systems for controlled-release oral dosage forms. Amongst a variety of polymers, polyethylene oxide (PEO) is considered an important material used in pharmaceutical formulations. As PEO is sensitive to thermal oxidation, it is susceptible to free radical oxidative attack. The aim of this study was to investigate the stability of PEO based formulations containing different model drugs with different water solubility, namely propranolol HCl, theophylline and zonisamide. Both polyox matrices 750 and 303 grade were used as model carriers for the manufacture of tablets stored at 40 °C. The results of the present study suggest that the drug release from the matrix was affected by the length of storage conditions, solubility of drugs and the molecular weight of the polymers. Generally, increased drug release rates were prevalent in soluble drug formulations (propranolol) when stored at the elevated temperature (40 °C). In contrast, it was not observed with semi soluble (theophylline) and poorly soluble (zonisamide) drugs especially when formulated with PEO 303 polymer. This indicates that the main parameters controlling the drug release from fresh polyox matrices are the solubility of the drug in the dissolution medium and the molecular weight of the polymer. DSC traces indicated that that there was a big difference in the enthalpy and melting points of fresh and aged PEO samples containing propranolol, whereas the melting point of the aged polyox samples containing theophylline and zonisamide was unaffected

Insulin Detemir Reduces Weight Gain as a Result of Reduced Food Intake in Patients With Type 1 Diabetes

Insulin detemir lacks the usual propensity for insulin to cause weight gain. We investigated whether this effect was a result of reduced energy intake and/or increased energy expenditure