Actor-Critic Reinforcement Learning for Control with Stability Guarantee

Reinforcement Learning (RL) and its integration with deep learning have achieved impressive performance in various robotic control tasks, ranging from motion planning and navigation to end-to-end visual manipulation. However, stability is not guaranteed in model-free RL by solely using data. From a control-theoretic perspective, stability is the most important property for any control system, since it is closely related to safety, robustness, and reliability of robotic systems. In this paper, we propose an actor-critic RL framework for control which can guarantee closed-loop stability by employing the classic Lyapunov's method in control theory. First of all, a data-based stability theorem is proposed for stochastic nonlinear systems modeled by Markov decision process. Then we show that the stability condition could be exploited as the critic in the actor-critic RL to learn a controller/policy. At last, the effectiveness of our approach is evaluated on several well-known 3-dimensional robot control tasks and a synthetic biology gene network tracking task in three different popular physics simulation platforms. As an empirical evaluation on the advantage of stability, we show that the learned policies can enable the systems to recover to the equilibrium or way-points when interfered by uncertainties such as system parametric variations and external disturbances to a certain extent.Comment: IEEE RA-L + IROS 202

Application of Time-Fractional Order Bloch Equation in Magnetic Resonance Fingerprinting

Magnetic resonance fingerprinting (MRF) is one novel fast quantitative imaging framework for simultaneous quantification of multiple parameters with pseudo-randomized acquisition patterns. The accuracy of the resulting multi-parameters is very important for clinical applications. In this paper, we derived signal evolutions from the anomalous relaxation using a fractional calculus. More specifically, we utilized time-fractional order extension of the Bloch equations to generate dictionary to provide more complex system descriptions for MRF applications. The representative results of phantom experiments demonstrated the good accuracy performance when applying the time-fractional order Bloch equations to generate dictionary entries in the MRF framework. The utility of the proposed method is also validated by in-vivo study.Comment: Accepted at 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019

Quantifying the coupling and degeneracy of OAM modes in high-index-contrast ring core fiber

We study orbital angular momentum (OAM) mode coupling in ring-core fibers (RCFs) due to elliptical shape deformation. We introduce a coupling model based on numerical mode solver outputs of perturbation. We show improved predictions in calculating coupling strength compared to the classical modeling approach. Our model captures and quantifies the disparate behaviors of coupling in lower and higher order degenerate OAM modes. The ideal orthogonality of modes is undermined by fiber imperfections. Our model predicts the OAM order at which the orthogonality within OAM mode pair is maintained despite elliptical deformation. We use our coupling model to simulate propagation effects and compare the performance of two fibers (thin and thick RCF) designed under the same constraints. Our numerical propagation results show different performance for the two fibers under the same level of elliptical deformation. This model uncovers distinct digital signal processing requirements for these two types of fiber, and predicts their signal-to-noise ratio penalty. For each fiber, we examine the large number of supported modes and find the optimal subset of mode groups, i.e., the groups with the lowest penalty. We show that this optimal subset is different from that predicted during the fiber design optimization

Mode loss measurement in few-mode fibers with a microwave interferometric technique

We measure the mode dependent loss (MDL) of a few-mode fiber (FMF) using an improved microwave interferometric technique. A frequency-swept microwave signal modulates a filtered optical incoherent source that is injected into the FMF under test. During propagation, the microwave signal carried by the various modes experiences different losses and delays before interfering at the photodetector. The fiber MDL, between mode groups, is computed from the interference pattern by measuring two different fiber lengths under the same excitation condition. This technique relies on the dominant excitation of the fundamental mode and therefore can measure high values of MDL

Highly-elliptical-core fiber with stress-induced birefringence for mode multiplexing

We report the polarization–maintaining properties of a highly–elliptical–core fiber surrounded by a trench that was designed to optimize the modal effective indices and bending loss for a total of five spatial modes with twofold polarization degeneracy (ten channels). In addition to the asymmetric core structure, the birefringence of the fiber is increased by the thermal stress introduced during the fabrication. We examine this effect and compare the calculated modal effective index differences to experimentally measured values. The results show a modal birefringence larger than 10-4 for all guided spatial modes. The fiber has a propagation loss, averaged over all mode groups, of 0.45 dB/km. The mode stability to bending is evaluated by selectively exciting/detecting each spatial mode while perturbing the fiber. This few–mode polarization–maintaining fiber is of interest for MIMO–free mode division multiplexing transmission systems

The impact of modal interactions on receiver complexity in OAM fibers

We experimentally study the modal interactions in mode division multiplexing (MDM) links supporting orbital angular momentum (OAM) modes of order zero and one. We use time of flight and channel impulse response measurements to characterize our OAM-MDM link and quantify modal impairments. We examine two OAM fibers with different index profiles and differential mode group delays (DMGD) between supported vector modes. Data transmission experiments probe the impact of modal impairments on digital signal processing complexity and achievable bit error rate for OAM-MDM link. We discuss in particular memory depth requirements for equalizers in separate mode detection schemes, and how memory depth varies with DMGD metrics as well as crosstalk level