協調的エッジクラウドコンピューティングのためのマルチエージェント強化学習

京都大学新制・課程博士博士(情報学)甲第24261号情博第805号京都大学大学院情報学研究科社会情報学専攻(主査)教授 伊藤 孝行, 教授 吉川 正俊, 教授 神田 崇行, 特定准教授 LIN Donghui学位規則第4条第1項該当Doctor of InformaticsKyoto UniversityDFA

Self-Agreement: A Framework for Fine-tuning Language Models to Find Agreement among Diverse Opinions

Finding an agreement among diverse opinions is a challenging topic in multiagent systems. Recently, large language models (LLMs) have shown great potential in addressing this challenge due to their remarkable capabilities in comprehending human opinions and generating human-like text. However, they typically rely on extensive human-annotated data. In this paper, we propose Self-Agreement, a novel framework for fine-tuning LLMs to autonomously find agreement using data generated by LLM itself. Specifically, our approach employs the generative pre-trained transformer-3 (GPT-3) to generate multiple opinions for each question in a question dataset and create several agreement candidates among these opinions. Then, a bidirectional encoder representations from transformers (BERT)-based model evaluates the agreement score of each agreement candidate and selects the one with the highest agreement score. This process yields a dataset of question-opinion-agreements, which we use to fine-tune a pre-trained LLM for discovering agreements among diverse opinions. Remarkably, a pre-trained LLM fine-tuned by our Self-Agreement framework achieves comparable performance to GPT-3 with only 1/25 of its parameters, showcasing its ability to identify agreement among various opinions without the need for human-annotated data

Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARL

The necessity for cooperation among intelligent machines has popularised cooperative multi-agent reinforcement learning (MARL) in the artificial intelligence (AI) research community. However, many research endeavors have been focused on developing practical MARL algorithms whose effectiveness has been studied only empirically, thereby lacking theoretical guarantees. As recent studies have revealed, MARL methods often achieve performance that is unstable in terms of reward monotonicity or suboptimal at convergence. To resolve these issues, in this paper, we introduce a novel framework named Heterogeneous-Agent Mirror Learning (HAML) that provides a general template for MARL algorithmic designs. We prove that algorithms derived from the HAML template satisfy the desired properties of the monotonic improvement of the joint reward and the convergence to Nash equilibrium. We verify the practicality of HAML by proving that the current state-of-the-art cooperative MARL algorithms, HATRPO and HAPPO, are in fact HAML instances. Next, as a natural outcome of our theory, we propose HAML extensions of two well-known RL algorithms, HAA2C (for A2C) and HADDPG (for DDPG), and demonstrate their effectiveness against strong baselines on StarCraftII and Multi-Agent MuJoCo tasks

Relationships among gut microbes, the interleukin family, and hypertension: a mediation Mendelian randomization study

PurposeObservational studies have increasingly recognized the influence of gut microbes on blood pressure modulation. Despite these findings, a direct causal link between gut flora and hypertension remains unestablished due to inherent confounders and the challenges of reverse causality in observational research. In this study, we sought to elucidate the causal relationship between specific gut flora and hypertension and its intermediary mediators.MethodsWe employed a two-sample Mendelian randomization (MR) and mediation MR analysis, analyzing 211 species of gut bacteria, with a focus on the interleukin family as potential mediators and hypertension as the primary outcome. The central methodological technique was inverse variance-weighted estimation, supplemented by various other estimators.ResultsOur findings revealed that two bacterial species positively correlated with hypertension risk, while five exhibited a negative association. Further validation was conducted using sensitivity analyses. Notably, our mediation MR results suggest interleukin-1 receptor type 2 (IL-1R2) as a mediator for the effect of the genus Clostridium innocuum group on hypertension, accounting for a mediation proportion of 14.07% [mediation effect: (b = 0.0007, 95%CI: 0.0002–0.0011); proportion mediation = 14.07% (4.26–23.40%)].ConclusionOur research confirms a genetic causal relationship between specific gut microbes and hypertension, emphasizing the potential mediating role of interleukin-1 receptor type 2 (IL-1R2) and offering insights for clinical hypertension interventions

A study of the relationship between social anxiety and mask-wearing intention among college students in the post-COVID-19 era: mediating effects of self-identity, impression management, and avoidance

IntroductionDuring the 2019 coronavirus (COVID-19) pandemic, wearing masks not only prevented transmission of the virus but also reduced social anxiety to some extent. With the end of the epidemic, the intention to wear masks to prevent transmission declined, but the effect of social anxiety on the intention to wear masks is unclear. The current study investigated the effects of social anxiety and fear of COVID-19 on mask-wearing intentions in the post-epidemic era, using self-identity, impression management and avoidance as mediating variables.MethodsIn total, 223 college students participated in the current study, and the related variables were measured using the social anxiety scale, the social behavior questionnaire, the self-identity questionnaire, and the mask-wearing intention questionnaire.ResultsThe results showed that social anxiety was significantly positively correlated with avoidance, impression management, and intention to wear masks, and significantly negatively correlated with self-identity. The fear of COVID-19, avoidance, and impression management were significantly positively correlated with mask-wearing intentions, while self-identity was significantly negatively correlated with mask-wearing intentions. Social anxiety affected college students’ intention to wear masks through three main pathways: the mediating role of avoidance, impression management, and the chain mediating role of self-identity and avoidance. The fear of COVID-19 directly and positively affected mask-wearing intentions.DiscussionThe current study reveals the differential pathways of the effects of COVID-19 fear and social anxiety on mask-wearing intentions in the post-COVID-19 era, and the findings have some practical implications for social anxiety interventions

Phase boundary engineering of metal-organic-framework-derived carbonaceous nickel selenides for sodium-ion batteries

Abstract: Sodium-ion batteries (SIBs) are promising power sources due to the low cost and abundance of battery-grade sodium resources, while practical SIBs suffer from intrinsically sluggish diffusion kinetics and severe volume changes of electrode materials. Metal-organic framework (MOFs) derived carbonaceous metal compound offer promising applications in electrode materials due to their tailorable composition, nanostructure, chemical and physical properties. Here, we fabricated hierarchical MOF-derived carbonaceous nickel selenides with bi-phase composition for enhanced sodium storage capability. As MOF formation time increases, the pyrolyzed and selenized products gradually transform from a single-phase Ni3Se4 into bi-phase NiSex then single-phase NiSe2, with concomitant morphological evolution from solid spheres into hierarchical urchin-like yolk-shell structures. As SIBs anodes, bi-phase NiSex@C/CNT-10h (10 h of hydrothermal synthesis time) exhibits a high specific capacity of 387.1 mAh/g at 0.1 A/g, long cycling stability of 306.3 mAh/g at a moderately high current density of 1 A/g after 2,000 cycles. Computational simulation further proves the lattice mismatch at the phase boundary facilitates more interstitial space for sodium storage. Our understanding of the phase boundary engineering of transformed MOFs and their morphological evolution is conducive to fabricate novel composites/hybrids for applications in batteries, catalysis, sensors, and environmental remediation

Lack of Association Between DJ-1 Gene Promoter Polymorphism and the Risk of Parkinson’s Disease

Low DJ-1 protein level caused by DJ-1 gene mutation leads to autosomal recessive Parkinson’s disease (PD) due to impaired antioxidative activity. In sporadic PD patients, although mutations were rarely found, lower DJ-1 protein level was also reported. Dysregulation of DJ-1 gene expression might contribute to low DJ-1 protein level. Since the promoter is the most important element to initiate gene expression, whether polymorphisms in the DJ-1 promoter result in the dysregulation of gene expression, thus leading to low protein level and causing PD, is worth exploring. The DJ-1 promoter region was sequenced in a Chinese cohort to evaluate possible links between DJ-1 promoter polymorphisms, PD risk and clinical phenotypes. Dual-luciferase reporter assay was conducted to evaluate the influence of promoter polymorphisms on DJ-1 transcriptional activity. Related information in an existing genome-wide association studies (GWAS) database were looked up, meta-analysis of the present study and other previous reports was conducted, and expression quantitative trait loci (eQTL) analysis was performed to further explore the association. Three single nucleotide polymorphisms (SNPs) (rs17523802, rs226249, and rs35675666) and one 18 bp deletion (rs200968609) were observed in our cohort. However, there was no significant association between the four detected genetic variations and the risk of PD either in allelic or genotype model, in single-point analysis or haplotype analysis. This was supported by the meta-analysis of this study and previous reports as well as that of GWAS database PDGene. Dual luciferase reporter assay suggested these promoter polymorphisms had no influence on DJ-1 transcriptive activity, which is consistent with the eQTL analysis results using the data from GTEx database. Thus, DJ-1 promoter polymorphisms may play little role in the dysregulation of DJ-1 expression and PD susceptibility in sporadic PD

Pricing Mechanism of Charging Pile Power Supply Market——Based on RTP Theory and Price Discrimination Model

Based on the data of monopoly enterprises in China’s new energy charging pile power retail market, this paper explores the application of RTP differential pricing in new areas. First of all, from the perspective of business, this paper constructs the incentive cost model of low period which can minimize the supply pressure of power sales enterprises. Then, from the perspective of charging consumers, based on the assumption of user’s conversion cost, an improved demand response model is established according to the price elasticity. The paper is to consider the premise of maximizing social welfare, in the supply and demand of both sides to improve the pressure of electricity measurement, to minimize the operation and maintenance costs in peak and trough period