Wardrop Equilibrium Can Be Boundedly Rational: A New Behavioral Theory of Route Choice

As one of the most fundamental concepts in transportation science, Wardrop equilibrium (WE) has always had a relatively weak behavioral underpinning. To strengthen this foundation, one must reckon with bounded rationality in human decision-making processes, such as the lack of accurate information, limited computing power, and sub-optimal choices. This retreat from behavioral perfectionism in the literature, however, was typically accompanied by a conceptual modification of WE. Here we show that giving up perfect rationality need not force a departure from WE. On the contrary, WE can be reached with global stability in a routing game played by boundedly rational travelers. We achieve this result by developing a day-to-day (DTD) dynamical model that mimics how travelers gradually adjust their route valuations, hence choice probabilities, based on past experiences. Our model, called cumulative logit (CULO), resembles the classical DTD models but makes a crucial change: whereas the classical models assume routes are valued based on the cost averaged over historical data, ours values the routes based on the cost accumulated. To describe route choice behaviors, the CULO model only uses two parameters, one accounting for the rate at which the future route cost is discounted in the valuation relative to the past ones and the other describing the sensitivity of route choice probabilities to valuation differences. We prove that the CULO model always converges to WE, regardless of the initial point, as long as the behavioral parameters satisfy certain mild conditions. Our theory thus upholds WE's role as a benchmark in transportation systems analysis. It also resolves the theoretical challenge posed by Harsanyi's instability problem by explaining why equally good routes at WE are selected with different probabilities

Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization

Most offline reinforcement learning (RL) methods suffer from the trade-off between improving the policy to surpass the behavior policy and constraining the policy to limit the deviation from the behavior policy as computing $Q$-values using out-of-distribution (OOD) actions will suffer from errors due to distributional shift. The recently proposed \textit{In-sample Learning} paradigm (i.e., IQL), which improves the policy by quantile regression using only data samples, shows great promise because it learns an optimal policy without querying the value function of any unseen actions. However, it remains unclear how this type of method handles the distributional shift in learning the value function. In this work, we make a key finding that the in-sample learning paradigm arises under the \textit{Implicit Value Regularization} (IVR) framework. This gives a deeper understanding of why the in-sample learning paradigm works, i.e., it applies implicit value regularization to the policy. Based on the IVR framework, we further propose two practical algorithms, Sparse $Q$-learning (SQL) and Exponential $Q$-learning (EQL), which adopt the same value regularization used in existing works, but in a complete in-sample manner. Compared with IQL, we find that our algorithms introduce sparsity in learning the value function, making them more robust in noisy data regimes. We also verify the effectiveness of SQL and EQL on D4RL benchmark datasets and show the benefits of in-sample learning by comparing them with CQL in small data regimes.Comment: ICLR 2023 notable top 5

Epilepsy Associated With Mitochondrial Encephalomyopathy, Lactic Acidosis, and Stroke-Like Episodes

Objectives: The present study explored the clinical characteristics and prognostic factors of epilepsy in patients with mitochondrial encephalomyopathy, lactic acidosis, and stroke-like episodes (MELAS).Methods: Thirty-four MELAS patients were included in the present study. They were diagnosed by clinical characteristics, genetic testing, muscle biopsy, and retrospective analysis of other clinical data. The patients were divided into three groups according to the effects of treatment after at least 2 years of follow-up.Results: Epilepsy was more common in male MELAS patients than in females (20/14). The age of onset ranged from 0.5 to 57 years, with an average of 22.6 years. Patients with epilepsy and MELAS had various forms of seizures. Focal seizures were the most common type affecting 58.82% of patients, and some patients had multiple types of seizures. The abnormal EEG waves were mainly concentrated in the occipital (69.57%), frontal (65.22%) and temporal lobes (47.83%). Overall, the prognosis of patients with epilepsy and MELAS was poor. Poor prognosis was associated with brain atrophy (P = 0.026), status epilepticus (P < 0.001), and use of anti-seizure medications with high mitochondrial toxicity (P = 0.015).Interpretation: Avoiding the application of anti-seizure medications with high mitochondrial toxicity, controlling seizures more actively and effectively, and delaying the occurrence and progression of brain atrophy as much as possible are particularly important to improve the prognosis of patients with MELAS and epilepsy

Effects of Exercise on AMPK Signaling and Downstream Components to PI3K in Rat with Type 2 Diabetes

Exercise can increase skeletal muscle sensitivity to insulin, improve insulin resistance and regulate glucose homeostasis in rat models of type 2 diabetes. However, the potential mechanism remains poorly understood. In this study, we established a male Sprague-Dawley rat model of type 2 diabetes, with insulin resistance and β cell dysfunction, which was induced by a high-fat diet and low-dose streptozotocin to replicate the pathogenesis and metabolic characteristics of type 2 diabetes in humans. We also investigated the possible mechanism by which chronic and acute exercise improves metabolism, and the phosphorylation and expression of components of AMP-activated protein kinase (AMPK) and downstream components of phosphatidylinositol 3-kinase (PI3K) signaling pathways in the soleus. As a result, blood glucose, triglyceride, total cholesterol, and free fatty acid were significantly increased, whereas insulin level progressively declined in diabetic rats. Interestingly, chronic and acute exercise reduced blood glucose, increased phosphorylation and expression of AMPKα1/2 and the isoforms AMPKα1 and AMPKα2, and decreased phosphorylation and expression of AMPK substrate, acetyl CoA carboxylase (ACC). Chronic exercise upregulated phosphorylation and expression of AMPK upstream kinase, LKB1. But acute exercise only increased LKB1 expression. In particular, exercise reversed the changes in protein kinase C (PKC)ζ/λ phosphorylation, and PKCζ phosphorylation and expression. Additionally, exercise also increased protein kinase B (PKB)/Akt1, Akt2 and GLUT4 expression, but AS160 protein expression was unchanged. Chronic exercise elevated Akt (Thr(308)) and (Ser(473)) and AS160 phosphorylation. Finally, we found that exercise increased peroxisome proliferator-activated receptor-γ coactivator 1 (PGC1) mRNA expression in the soleus of diabetic rats. These results indicate that both chronic and acute exercise influence the phosphorylation and expression of components of the AMPK and downstream to PIK3 (aPKC, Akt), and improve GLUT4 trafficking in skeletal muscle. These data help explain the mechanism how exercise regulates glucose homeostasis in diabetic rats