Closed-Loop Solvability of Linear Quadratic Mean-Field Type Stackelberg Stochastic Differential Games

This paper is devoted to a Stackelberg stochastic differential game for a linear mean-field type stochastic differential system with a mean-field type quadratic cost functional in finite horizon. The coefficients in the state equation and weighting matrices in the cost functional are all deterministic. Closed-loop Stackelberg equilibrium strategies are introduced which require to be independent of initial states. Follower's problem is solved firstly, which is a stochastic linear quadratic optimal control problem. By converting the original problem into a new one whose optimal control is known, the closed-loop optimal strategy of the follower is characterized by two coupled Riccati equations as well as a linear mean-field type backward stochastic differential equation. Then the leader turns to solve a stochastic linear quadratic optimal control problem for a mean-field type forward-backward stochastic differential equation. Necessary conditions for the existence of closed-loop optimal strategies for the leader is given by the existence of two coupled Riccati equations with a linear mean-field type backward stochastic differential equation. The solvability of Riccati equations of leader's optimization problem is discussed in the case where the diffusion term of the state equation does not contain the control process of the follower. Moreover, leader's value function is expressed via two backward stochastic differential equations and two Lyapunov equations.Comment: 44 page

Federated Reinforcement Learning for Real-Time Electric Vehicle Charging and Discharging Control

With the recent advances in mobile energy storage technologies, electric vehicles (EVs) have become a crucial part of smart grids. When EVs participate in the demand response program, the charging cost can be significantly reduced by taking full advantage of the real-time pricing signals. However, many stochastic factors exist in the dynamic environment, bringing significant challenges to design an optimal charging/discharging control strategy. This paper develops an optimal EV charging/discharging control strategy for different EV users under dynamic environments to maximize EV users' benefits. We first formulate this problem as a Markov decision process (MDP). Then we consider EV users with different behaviors as agents in different environments. Furthermore, a horizontal federated reinforcement learning (HFRL)-based method is proposed to fit various users' behaviors and dynamic environments. This approach can learn an optimal charging/discharging control strategy without sharing users' profiles. Simulation results illustrate that the proposed real-time EV charging/discharging control strategy can perform well among various stochastic factors

Multi-stage collaborative efficiency measurement of scitech finance: network-DEA analysis and spatial impact research

Sci-tech and finance plays an increasingly important role and have become an important driving force in economic development. In China, the problem of insufficient financial support for sci-tech innovation is important to enterprises. According to the internal relationship between different stages of Sci-tech and the finance system, this paper is aimed at exploring the efficiency measurement method between sci-tech and finance systems. Firstly the multi-stage collaborative structure of sci-tech finance is built, where the system of sci-tech is divided into three stages including the R&D stage, transformation stage of sci-tech achievements and industrialization stage, and the financing channel is the input of the finance system into the sci-tech system at different stages. The measurement method of the multi-stage collaborative efficiency between sci-tech and finance systems is put forward by the framework of network DEA. Then, taking China as an example, we collect the information of 30 provinces and cities from 2009 to 2016 and measure the efficiency of each system and the collaborative efficiency of the both. The efficiency’s spatial correlation is tested by means of Moran index. Finally, the influencing factors of the collaborative efficiency are analyzed based on the spatial econometric regression model, which considers the financing channels and human capital. To sum up, there are significant differences in the sci-tech finance collaborative efficiency among regions in China. Among them, the collaborative efficiency of Beijing, Shanghai and Jiangsu ranks in the top three. Comparing the different stages of the sci-tech system, the commercialization stage is a weak link in many regions of China. Human capital and financing channels of sci-tech finance have different degrees of positive impact on the sci-tech finance collaborative efficiency. Among them, human capital plays a greater role in promoting the sci-tech finance collaborative development

THE ISOKINETIC MUSCLE ASYMMETRY OF THE THIGH AT 1 YEAR AFTER ANTERIOR CRUCIATE LIGAMENT RECONSTRUCTION WAS SIGNIFICANTLY ASSOCIATED WITH GAIT ASYMMETRY

Objective: To study the correlation between muscle strength asymmetry and gait asymmetry in 1 year after （Anterior Cruciate Ligament Reconstruction， ACLR）. Methods: Twenty-five ACLR patients were enrolled in the Department of Sports Medicine, Peking University Third Hospital. Data of isokinetic muscle strength test one year after ACLR were collected. The concentric and eccentric strength of extensor and flexor muscles at 60°/s, 180°/s and 300°/s on the uninjured side and the injured side were measured respectively, and the peak value of muscle strength was analyzed. The three dimensional motion information and ground reaction force during gait were collected, and the peaks of three dimensional joint angle and moments during gait stance phase were calculated by inverse dynamics analysis. The paired-samples T test was used to analyze the difference of gait parameters and isokinetic muscle strength peaks. Spearman correlation analysis was used to study the correlation between bilateral asymmetry index of isokinetic muscle strength and gait asymmetry index. Results: One year after ACLR, the isokinetic muscle strength peaks of the flexor and extensor muscles on the injured side were significantly lower than those on the uninjured side【60°/s extensor concentric, the injured side: (1.22 ± 0.4)Nm·kg-1, uninjured side: (1.73 ± 0.42)Nm·kg-1, bilateral difference: (-0.5 ± 0.39)Nm·kg-1, P \u3c 0.01; 60°/s flexor concentric, injured side: (0.84 ± 0.19)Nm·kg-1, uninjured side: (1.05 ± 0.23)Nm·kg-1, bilateral difference: (-0.21 ± 0.14)Nm·kg-1, P \u3c 0.01】. Compared with the uninjured side, the injured side showed insufficient knee extension at the time of maximum knee extension during stance phase 【injured side: (5.25 ± 4.17) °, uninjured side: (2.24 ± 3.11) °, bilateral difference: (3.01 ± 2.44) °, P \u3c 0.01】, and the peak extension moment decreased significantly 【injured side: (0.1 ± 0.09) Nm·kg-1·m-1, (0.15 ± 0.07) Nm·kg-1·m-1, (-0.05 ± 0.06) Nm·kg-1·m-1, P \u3c 0.01】. One year after ACLR, the asymmetry of 180°/s isokinetic extensor concentric strength was significantly correlated with the asymmetry of peak flexion moment (R = 0.449, P = 0.024). The asymmetry of 60°/s isokinetic extensor concentric strength was significantly correlated with the asymmetry of peak internal rotation moment (R = 0.421, P = 0.036). One year after ACLR, asymmetries of 180°/s, 300°/s isokinetic extensor concentric strength and 60°/s isokinetic flexor eccentric strength were significantly correlated with peak asymmetries during stance phase. Conclusion: There is a significant correlation between isokinetic muscle strength asymmetry of knee and gait asymmetry. This study suggests that ACLR patients still need regular rehabilitation training to improve muscle strength and motor function 1 year after ACLR, so as to reduce the risk of reinjury and secondary injury

OSP: Boosting Distributed Model Training with 2-stage Synchronization

Distributed deep learning (DDL) is a promising research area, which aims to increase the efficiency of training deep learning tasks with large size of datasets and models. As the computation capability of DDL nodes continues to increase, the network connection between nodes is becoming a major bottleneck. Various methods of gradient compression and improved model synchronization have been proposed to address this bottleneck in Parameter-Server-based DDL. However, these two types of methods can result in accuracy loss due to discarded gradients and have limited enhancement on the throughput of model synchronization, respectively. To address these challenges, we propose a new model synchronization method named Overlapped Synchronization Parallel (OSP), which achieves efficient communication with a 2-stage synchronization approach and uses Local-Gradient-based Parameter correction (LGP) to avoid accuracy loss caused by stale parameters. The prototype of OSP has been implemented using PyTorch and evaluated on commonly used deep learning models and datasets with a 9-node testbed. Evaluation results show that OSP can achieve up to 50\% improvement in throughput without accuracy loss compared to popular synchronization models.Comment: Copyright Owner/Author | ACM 2023. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record will be published in ICPP 202

Boosting Distributed Machine Learning Training Through Loss-tolerant Transmission Protocol

Distributed Machine Learning (DML) systems are utilized to enhance the speed of model training in data centers (DCs) and edge nodes. The Parameter Server (PS) communication architecture is commonly employed, but it faces severe long-tail latency caused by many-to-one "incast" traffic patterns, negatively impacting training throughput. To address this challenge, we design the \textbf{L}oss-tolerant \textbf{T}ransmission \textbf{P}rotocol (LTP), which permits partial loss of gradients during synchronization to avoid unneeded retransmission and contributes to faster synchronization per iteration. LTP implements loss-tolerant transmission through \textit{out-of-order transmission} and \textit{out-of-order Acknowledges (ACKs)}. LTP employs \textit{Early Close} to adjust the loss-tolerant threshold based on network conditions and \textit{Bubble Filling} for data correction to maintain training accuracy. LTP is implemented by C++ and integrated into PyTorch. Evaluations on a testbed of 8 worker nodes and one PS node demonstrate that LTP can significantly improve DML training task throughput by up to 30x compared to traditional TCP congestion controls, with no sacrifice to final accuracy.Comment: This paper will be published on IWQoS 2023. Preview version onl

Surface oxygen nonstoichiometry depends non-monotonically on biaxial strain in ultrathin ceria films

Strain engineering provides a ‘dopant-free’ alternative to tailor the electronic, defect chemical and catalytic properties of mixed ion and electron conducting (MIEC) oxides. By perturbing the crystallographic symmetry of an oxide, strain can dramatically alter both the position and degeneracy of electronic energy levels, which directly impact the defect chemistry of the oxide. In this work, we employed ambient pressure X-ray photoelectron spectroscopy (APXPS) to probe the valence and core levels of ultra thin films of cerium oxide, a prototypical MIEC, subject to an elastic strain. Coherently strained CeO2 films were grown on atomically flat (001) Y0.16Zr0.84O1.92 (5.5 % compressive strain) and SrTiO3 (2.1 % tensile strain) single crystal substrates. Aberration-corrected transmission electron microscopy revealed an CeO2/substrate interface free of cation diffusion and devoid of misfit dislocations. While equilibrium theory predicts that a coherently ceria/YSZ interface is unlikely because of the large lattice mismatch, we demonstrate a stable, redox active coherent ceria film up to 3 nm in thickness on YSZ. Surface sensitive APXPS performed at 450°C and 550°C under various oxygen partial pressures revealed that the surfaces of the strained ultrathin oxide films, both compressive and tensile, exhibited higher surface polaron concentration (and by extension, oxygen vacancies) compared to the bulk-like, unstrained films. This remarkable result is at odds with the conventional view that the reduction enthalpy decreases monotonically with strain. We systematically performed depth resolved XPS measurements on films of different thicknesses to deconvolve strain effect on the surface redox capacity from that of substrate induced chemical and electrostatic effects. We hypothesize that elastic biaxial strain has a two-pronged effect on the redox capacity. By its coupling with oxygen chemical potential through chemical expansion, tensile and compressive strain will have a monotonic effect on the oxygen nonstoichiometry. However, the symmetry breaking induced by the tetragonal distortion – irrespective of compression or tension - predisposes the oxide away from cubic symmetry. In turn, this favors the reduced oxide (3+ oxidation state of Ce) in its hexagonal sesquioxide environment. The latter effect leads to a non-monotonic dependence of redox properties on biaxial strain. This work expands our understanding of the behavior of highly strained MIEC oxide films under catalytically relevant conditions. The knowledge of non-monotonic oxygen nonstoichiometry dependence could be extremely useful for strain-engineered heterolayers for memristive applications and surface coatings for pseudocapacitive energy storage

Rethink Baseline of Integrated Gradients from the Perspective of Shapley Value

Numerous approaches have attempted to interpret deep neural networks (DNNs) by attributing the prediction of DNN to its input features. One of the well-studied attribution methods is Integrated Gradients (IG). Specifically, the choice of baselines for IG is a critical consideration for generating meaningful and unbiased explanations for model predictions in different scenarios. However, current practice of exploiting a single baseline fails to fulfill this ambition, thus demanding multiple baselines. Fortunately, the inherent connection between IG and Aumann-Shapley Value forms a unique perspective to rethink the design of baselines. Under certain hypothesis, we theoretically analyse that a set of baseline aligns with the coalitions in Shapley Value. Thus, we propose a novel baseline construction method called Shapley Integrated Gradients (SIG) that searches for a set of baselines by proportional sampling to partly simulate the computation path of Shapley Value. Simulations on GridWorld show that SIG approximates the proportion of Shapley Values. Furthermore, experiments conducted on various image tasks demonstrate that compared to IG using other baseline methods, SIG exhibits an improved estimation of feature's contribution, offers more consistent explanations across diverse applications, and is generic to distinct data types or instances with insignificant computational overhead.Comment: 12 page

Impacts of urban microclimate on summertime sensible and latent energy demand for cooling in residential buildings of Hong Kong

The urban heat island (UHI) and urban moisture island (UMI) effect can be significant in Hong Kong due to its high-density land utilization, and this can strongly affect building energy performance. While the UHI’ energy impact has been rather intensively studied recently, the UMI effect on latent energy is still underexplored, especially for humid subtropical climate like Hong Kong. This study investigated the intensity of UHI and UMI in Hong Kong, and its impacts on the sensible and latent cooling demand of residential buildings in summer. Firstly, a ten-year weather dataset from 2004 to 2013 for the six stations selected based on the local climate zone (LCZ) scheme was analysed. The results show that the urban area of Hong Kong appears as both a heat and moisture island during summer nights but as cooling and dry islands during daytime, and the nocturnal UHI and UMI intensity vary significantly with different LCZs. Furthermore, the energy performance of a typical residential building in Hong Kong was simulated with measured weather data for the selected stations as an input. The urban building shows a higher sensible cooling demand, approximately twice that of the comparative rural one, and the latent cooling demand could be up to 96% higher. Both sensible and latent cooling energy demand decrease with increasing LCZ grades. Our study highlights that both UHI and UMI effect should be considered in the estimation of building energy in Hong Kong due to their significant impacts on the cooling energy demand