The computation of Wavelet-Galerkin approximation on a bounded interval

International audienceThis paper describes exact evaluations of various finite integrals whose integrands involve products of Daubechies' compactly supported wavelets and their derivatives and/or integrals. These finite integrals play an essential role in the wavelet-Galerkin approximation of differential or integral equations on a bounded interval

Accelerated Policy Gradient: On the Nesterov Momentum for Reinforcement Learning

Policy gradient methods have recently been shown to enjoy global convergence at a $\Theta(1/t)$ rate in the non-regularized tabular softmax setting. Accordingly, one important research question is whether this convergence rate can be further improved, with only first-order updates. In this paper, we answer the above question from the perspective of momentum by adapting the celebrated Nesterov's accelerated gradient (NAG) method to reinforcement learning (RL), termed \textit{Accelerated Policy Gradient} (APG). To demonstrate the potential of APG in achieving faster global convergence, we formally show that with the true gradient, APG with softmax policy parametrization converges to an optimal policy at a $\tilde{O}(1/t^2)$ rate. To the best of our knowledge, this is the first characterization of the global convergence rate of NAG in the context of RL. Notably, our analysis relies on one interesting finding: Regardless of the initialization, APG could end up reaching a locally nearly-concave regime, where APG could benefit significantly from the momentum, within finite iterations. By means of numerical validation, we confirm that APG exhibits $\tilde{O}(1/t^2)$ rate as well as show that APG could significantly improve the convergence behavior over the standard policy gradient.Comment: 51 pages, 8 figure

A Self-Adaptive Cooperative Routing Protocol for Underwater Acoustic Sensor Networks

[[abstract]]Design an effective routing protocol in underwater acoustic sensor networks (UASNs) is an important issue. Long propagation time and low DATA rate which are two major concerns for routing protocol design in UASNs will lead to the long end-to-end transmission time. This paper proposes a SelfAdaptive Cooperative Routing Protocol (SACRP) to effectively route collecting DATA to the sink in UASNs. Cooperative transmission in SACRP not only can enhance the link quality (Signalto-Noise Ratio (SNR)) to improve the network throughput but also can increase the transmission range of a node to reduce the end-to-end transmission time. Some mathematical analyses about cooperative transmission scheme are done to support SACRP protocol in the different DATA size and transmission range as well. Based on the network simulations, the proposed protocol, SACRP, has a significant performance against the related work in average end-to-end delay and packet delivery ratio.[[notice]]補正完

Vertebral osteomyelitis caused by vancomycin-tolerant methicillin-resistant Staphylococcus aureus bacteremia: Experience with teicoplanin plus fosfomycin combination therapy

An 85-year-old female presented with fever and consciousness disturbance for 3 days. The patient's blood culture subsequently revealed persistent methicillin-resistant Staphylococcus aureus (MRSA) bacteremia despite the administration of vancomycin or teicoplanin monotherapy. Gallium inflammation scan and magnetic resonance image of the spine disclosed osteomyelitis and discitis at the level of L4–5. Surgical debridement was not feasible in this debilitated patient. Because of the creeping minimal inhibitory concentration of vancomycin of the causative isolate (1.5 μg/mL) and clinical failure with glycopeptide monotherapy, we changed the antibiotic therapy to a fosfomycin and teicoplanin combination therapy. The patient showed improved clinical response in terms of her enhanced consciousness as well as subsidence of persisted bacteremia. Despite the potential side effects of fosfomycin (such as diarrhea and hypernatremia), it combined with a glycopeptide may be an alternative therapy for invasive refractory MRSA infections

Coordinate Ascent for Off-Policy RL with Global Convergence Guarantees

We revisit the domain of off-policy policy optimization in RL from the perspective of coordinate ascent. One commonly-used approach is to leverage the off-policy policy gradient to optimize a surrogate objective -- the total discounted in expectation return of the target policy with respect to the state distribution of the behavior policy. However, this approach has been shown to suffer from the distribution mismatch issue, and therefore significant efforts are needed for correcting this mismatch either via state distribution correction or a counterfactual method. In this paper, we rethink off-policy learning via Coordinate Ascent Policy Optimization (CAPO), an off-policy actor-critic algorithm that decouples policy improvement from the state distribution of the behavior policy without using the policy gradient. This design obviates the need for distribution correction or importance sampling in the policy improvement step of off-policy policy gradient. We establish the global convergence of CAPO with general coordinate selection and then further quantify the convergence rates of several instances of CAPO with popular coordinate selection rules, including the cyclic and the randomized variants of CAPO. We then extend CAPO to neural policies for a more practical implementation. Through experiments, we demonstrate that CAPO provides a competitive approach to RL in practice.Comment: 47 pages, 4 figure

Thoracoscopic plication for a huge thoracic meningocele in a patient with Neurofibromatosis

Intrathoracic meningoceles associated with neurofibromatosis type I are rare, and the optimal treatment is still unknown. Herein, we present the case of a 48-year-old Asian female with a huge thoracic meningocele associated with cutaneous neurofibromatosis type I and kyphoscoliosis of the thoracic spine. The large thoracic meningocele was successfully treated through thoracoscopic plication