Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags; (2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.Comment: 20 page

Impingement-free Hip Motion: The ‘Normal' Angle Alpha after Osteochondroplasty

Femoroacetabular impingement is considered a cause of hip osteoarthrosis. In cam impingement, an aspherical head-neck junction is squeezed into the joint and causes acetabular cartilage damage. The anterior offset angle α, observed on a lateral crosstable radiograph, reflects the location where the femoral head becomes aspheric. Previous studies reported a mean angle α of 42° in asymptomatic patients. Currently, it is believed an angle α of 50° to 55° is normal. The aim of this study was to identify that angle α which allows impingement-free motion. In 45 patients who underwent surgical treatment for femoroacetabular impingement, we measured the angle α preoperatively, immediately postoperatively, and 1year postoperatively. All hips underwent femoral correction and, if necessary, acetabular correction. The correction was considered sufficient when, in 90° hip flexion, an internal rotation of 20° to 25° was possible. The angle α was corrected from a preoperative mean of 66° (range, 45°-79°) to 43° (range, 34°-60°) postoperatively. Because the acetabulum is corrected to normal first, the femoral correction is tested against a normal acetabulum. We therefore concluded an angle α of 43° achieved surgically and with impingement-free motion, represents the normal angle α, an angle lower than that currently considered sufficien

The genome sequence of the wisent (Bison bonasus)

This work was supported by the Youth Science and Technology Innovation Team of Sichuan Province (2014TD003), Shenzhen Industrial Designation Services Cloud Platform (GGJS20150429172906635), International Collaboration 111 Projects of China, Fundamental Research Funds for the Central Universities, 985 and 211 Projects of Sichuan University.The wisent, also known as the European bison, was rescued from extinction approximately 80 years ago through the conservation of 12 individuals. Here, we present the draft genome sequence of a male wisent individual descended from this founding stock. A total of 366 billion base pairs (Gb) of raw reads from whole-genome sequencing of this wisent were generated using the Illumina HiSeq2000 platform. The final genome assembly (2.58 Gb) is composed of 29,074 scaffolds with an N50 of 4.7 Mb. 47.3% of the genome is composed of repetitive elements. We identified 21,542 genes and 58,385 non-coding RNAs. A phylogenetic tree based on nuclear genomes indicated sister relationships between bison and wisent and between the wisent-bison clade and yak. For 75 genes we obtained evidence of positive evolution in the wisent lineage. We provide the first genome sequence and gene annotation for the wisent. The availability of these resources will be of value for the future conservation of this endangered large mammal and for reconstructing the evolutionary history of the Bovini tribe.Publisher PDFPeer reviewe

Identifiability of interaction kernels in mean-field equations of interacting particles

We study the identifiability of the interaction kernels in mean-field equations for intreacting particle systems. The key is to identify function spaces on which a probabilistic loss functional has a unique minimizer. We prove that identifiability holds on any subspace of two reproducing kernel Hilbert spaces (RKHS), whose reproducing kernels are intrinsic to the system and are data-adaptive. Furthermore, identifiability holds on two ambient L2 spaces if and only if the integral operators associated with the reproducing kernels are strictly positive. Thus, the inverse problem is ill-posed in general. We also discuss the implications of identifiability in computational practice

Learning Memory Kernels in Generalized Langevin Equations

We introduce a novel approach for learning memory kernels in Generalized Langevin Equations. This approach initially utilizes a regularized Prony method to estimate correlation functions from trajectory data, followed by regression over a Sobolev norm-based loss function with RKHS regularization. Our method guarantees improved performance within an exponentially weighted L^2 space, with the kernel estimation error controlled by the error in estimated correlation functions. We demonstrate the superiority of our estimator compared to other regression estimators that rely on L^2 loss functions and also an estimator derived from the inverse Laplace transform, using numerical examples that highlight its consistent advantage across various weight parameter selections. Additionally, we provide examples that include the application of force and drift terms in the equation

Drive network to a desired orbit by pinning control

summary:The primary objective of the present paper is to develop an approach for analyzing pinning synchronization stability in a complex delayed dynamical network with directed coupling. Some simple yet generic criteria for pinning such coupled network are derived analytically. Compared with some existing works, the primary contribution is that the synchronization manifold could be chosen as a weighted average of all the nodes states in the network for the sake of practical control tactics, which displays the different influences and contributions of the various nodes in synchronization seeking processes of the dynamical network. Furthermore, it is shown that in order to drive a complex network to a desired synchronization state, the coupling strength should vary according to the controller. In addition, the theoretical results about the time-invariant network is extended to the time-varying network, and the result on synchronization problem can also be extended to the consensus problem of networked multi-agent systems. Subsequently, the theoretic results are illustrated by a typical scale-free (SF) neuronal network. Numerical simulations with three kinds of the homogenous solutions, including an equilibrium point, a periodic orbit, and a chaotic attractor, are finally given to demonstrate the effectiveness of the proposed control methodology

Ancient polymorphisms and divergence hitchhiking contribute to genomic islands of divergence within a poplar species complex

How genome divergence eventually leads to speciation is a topic of prime evolutionary interest. Genomic islands of elevated divergence are frequently reported between diverging lineages, and their size is expected to increase with time and gene flow under the speciation-with-gene-flow model. However, such islands can also result from divergent sorting of ancient polymorphisms, recent ecological selection regardless of gene flow, and/or recurrent background selection and selective sweeps in low-recombination regions. It is challenging to disentangle these nonexclusive alternatives, but here we attempt to do this in an analysis of what drove genomic divergence between four lineages comprising a species complex of desert poplar trees. Within this complex we found that two morphologically delimited species, Populus euphratica and Populus pruinosa, were paraphyletic while the four lineages exhibited contrasting levels of gene flow and divergence times, providing a good system for testing hypotheses on the origin of divergence islands. We show that the size and number of genomic islands that distinguish lineages are not associated with either rate of recent gene flow or time of divergence. Instead, they are most likely derived from divergent sorting of ancient polymorphisms and divergence hitchhiking. We found that highly diverged genes under lineage-specific selection and putatively involved in ecological and morphological divergence occur both within and outside these islands. Our results highlight the need to incorporate demography, absolute divergence measurement, and gene flow rate to explain the formation of genomic islands and to identify potential genomic regions involved in speciation.Publisher PDFPeer reviewe