Evolution and biogeography of frogs and salamanders, inferred from fossils, morphology and molecules

Classified in the Lissamphibia, modern amphibians are the only non-amniote tetrapods living today. They consist of three morphologically distinct groups: the tailless frogs and toads (Anura), the limbless caecilians (Gymnophiona), and the tailed salamanders and newts (Urodela). With 205 species, the caecilians are highly specialized worm-like forms that live a fossorial lifestyle, with a relatively narrow distribution in the tropic rainforests of South America, Africa and Asia (Duellman and Trueb, 1994; Amphibiaweb, 2015). Salamanders, with 683 species, are widely distributed in the North America, Asia and Europe, with a few plethodontids extending to Central and South America (Duellman and Trueb, 1994; Amphibiaweb, 2015). Frogs are the most diverse amphibian groups, with 6644 species distributed over all continents except Antarctica (Duellman and Trueb, 1994; Amphibiaweb, 2015). Both frogs and salamanders develop a wide array of lifestyles, ranging from terrestrial, aquatic, fossorial to aboreal lifestyles (Duellman and Trueb, 1994). During ontogeny, amphibian larvae usually undergo a drastic post-embryonic shift into an adult form, a term known as metamorphosis. In salamanders, another developmental pathway – neoteny – also occurs, in which the larval morphology is retained in sexually mature adults (Duellman and Trueb, 1994; Rose, 2003). Because of the diverse lifestyles and developmental pathways, frogs and salamanders are often used as model systems in many fields of biology (e.g., evo-devo). Over a century, but especially in the past two decades, a wealth of frog and salamander fossils has been discovered from the Mesozoic and Cenozoic of East Asia (e.g., Noble, 1924; Young, 1936; Borsuk-Bialynicka, 1978; Gao, 1986; Dong and Wang, 1998; Gao and Shubin, 2001, 2003, 2012; Gao and Wang, 2001; Gao and Chen, 2004; Wang and Rose, 2005; Wang and Evans, 2006b; Zhang et al., 2009; Chen et al., 2016; this study). Some of these fossils represent the earliest members of many crown clades, including the earliest crown salamanders from the Middle Jurassic (~165 Ma, Gao and Shubin, 2003), the earliest salamandroid from the Late Jurassic, the earliest sirenid from the Late Jurassic (this study), and the earliest spadefoot toads from the late Paleocence (Chen et al., 2016). Other fossils also bear important anatomical, temporal and geographical information in understanding their evolution. Unfortunately, the importance of many of these fossils remains obscure in a phylogenetic context. For example, an early-middle Oligocene Mongolian spadefoot toad Macropelobates osborni (Noble, 1924) was discovered outside the current distribution of spadefoot toads, yet its phylogenetic position and its implication on spadefoot toad biogeography remain not well understood. A major reason for the poor understanding of these fossils can be attributed to a trend of dichotomy between morphological and molecular phylogenies on amphibians. Whereas morphologists and paleontologists sometimes use a relatively small morphological dataset to reconstruct relationships (e.g., Gao and Shubin, 2012; Henrici, 2013), large-scale phylogenies are almost always conducted with molecular data with only living taxa (e.g., Roelants and Bossuyt, 2005; Pyron and Wiens, 2011). Very few studies on amphibian phylogeny have combined morphological and molecular data together, and even fewer also combined fossils. Because of this, the positions of many important fossils remains unclear, and the evolutionary scenarios inferred from only living species can sometimes be inconsistent with fossil evidence. In this thesis, I adopt a total-evidence approach to understand the evolution of amphibians, especially frogs and salamanders. I will incorporate information from fossils, morphology and molecules together to reconstruct the relationships. Compared with studies with each individual datasets, this approach incorporates all available data in a single analysis, with a goal to reach robust and congruent results that allow further discussions on character evolution and biogeographic reconstruction. The inclusion of fossils directly into the combined analysis provides the time dimension that is independent from molecular data (Norell, 1992). The anatomical combination of fossils can represent intermediate forms that help to solve the “long branch” problems caused by highly specialized modern taxa. The morphological dataset, despite its much smaller size with molecular data, is the only link between fossils and modern taxa. The inclusion of key morphological characters in both reconstructing phylogenetic hypotheses and examining character evolution provide consistent results that allow discussion on the homology/homoplasy of a certain character without ambiguity. The molecular sequence data provides overwhelmingly large data on modern taxa for phylogenetic reconstructions compared with morphological data, which helps to reach a robust hypothesis. Although fossils contain no molecular data, the inclusion of molecular sequence data into the combined analysis does have an effect on the positions of fossil taxa. By altering the relationship “framework” of modern taxa, the character optimization of fossils and other taxa of a combined analysis also varies compared with results of morphology-only analysis, thus changing the positions of fossils. In the following five chapters, I will describe a number of fossil amphibian species, reconstruct three combined phylogenies, and use the results for discussions on character evolution and biogeography. In Chapter 1 and Chapter 2, I focus on a frog clade called spadefoot toads (Anura: Pelobatoidea). In Chapter 1, I provide descriptions on three important fossil spadefoot toads from the Cenozoic of East Asia and North America: Macropelobates osborni from the early-middle Oligocene of Mongolia, Prospea holoserisca from the latest Paleocene of Mongolia, and Scaphiopus skinneri from the middle Oligocene of the United States. In Chapter 2, I conduct a combined phylogenetic analysis of archaeobatrachian frogs, and discuss the evolution of the bony spade and the historical biogeography of spadefoot toads based on the results of the phylogeny. In Chapter 3, I describe a new fossil frog from the Early Cretaceous of Inner Mongolia, China. The unique morphology of the new fossil is distinct from previous Early Cretaceous frogs from the Jehol Biota of China. Results of the combined analysis show that the new frog represents a basal member of the Pipanura. Comparisons between the Early Cretaceous frogs from China, Spain and Brazil show a high diversity of species coupled with a high degree of endemism during the Early Cretaceous. I discuss in the phylogenetic context how early frogs gradually reach their postcranial body plan with a shortened vertebral column, loss of ribs, and specialized pelvic regions. In Chapter 4, I provide a brief review of Mesozoic fossil salamanders from northern China, and describe a new fossil from the Late Jurassic of Liaoning Province, China. I conduct a combined phylogeny of higher-level relationships of salamanders. The new fossil, despite its general-looking appearance, represents a basal member of the highly specialized eel-like neotenic family Sirenidae on the cladogram. I discuss character evolutions in the Sirenidae, and how the neotenic developmental pathway evolved in early salamanders. In Chapter 5, I conduct a combined phylogenetic analysis of the salamander suborder Cryptobranchoidea, consisting of the neotenic giant salamanders (Cryptobranchidae) and the metamorphic Asiatic salamanders (Hynobiidae). The new morphological matrix includes new characters that were previously less sampled in the hynobranchial region. The monophyly of the Hynobiidae are confirmed by the new analysis, and four unequivocal synapomorphies are found for the clade. An S-DIVA biogeographic reconstruction is conducted to disscuss the distributional patterns of the Hynobiidae

MGHRL: Meta Goal-generation for Hierarchical Reinforcement Learning

Most meta reinforcement learning (meta-RL) methods learn to adapt to new tasks by directly optimizing the parameters of policies over primitive action space. Such algorithms work well in tasks with relatively slight difference. However, when the task distribution becomes wider, it would be quite inefficient to directly learn such a meta-policy. In this paper, we propose a new meta-RL algorithm called Meta Goal-generation for Hierarchical RL (MGHRL). Instead of directly generating policies over primitive action space for new tasks, MGHRL learns to generate high-level meta strategies over subgoals given past experience and leaves the rest of how to achieve subgoals as independent RL subtasks. Our empirical results on several challenging simulated robotics environments show that our method enables more efficient and generalized meta-learning from past experience.Comment: Accepted to the ICLR 2020 workshop: Beyond tabula rasa in RL (BeTR-RL

Addressing Action Oscillations through Learning Policy Inertia

Deep reinforcement learning (DRL) algorithms have been demonstrated to be effective in a wide range of challenging decision making and control tasks. However, these methods typically suffer from severe action oscillations in particular in discrete action setting, which means that agents select different actions within consecutive steps even though states only slightly differ. This issue is often neglected since the policy is usually evaluated by its cumulative rewards only. Action oscillation strongly affects the user experience and can even cause serious potential security menace especially in real-world domains with the main concern of safety, such as autonomous driving. To this end, we introduce Policy Inertia Controller (PIC) which serves as a generic plug-in framework to off-the-shelf DRL algorithms, to enables adaptive trade-off between the optimality and smoothness of the learned policy in a formal way. We propose Nested Policy Iteration as a general training algorithm for PIC-augmented policy which ensures monotonically non-decreasing updates under some mild conditions. Further, we derive a practical DRL algorithm, namely Nested Soft Actor-Critic. Experiments on a collection of autonomous driving tasks and several Atari games suggest that our approach demonstrates substantial oscillation reduction in comparison to a range of commonly adopted baselines with almost no performance degradation.Comment: Accepted paper on AAAI 202

How Reliable Are Reliability-Based Multiple Factor Code Formats?

Hazard

Joint embedding in Hierarchical distance and semantic representation learning for link prediction

The link prediction task aims to predict missing entities or relations in the knowledge graph and is essential for the downstream application. Existing well-known models deal with this task by mainly focusing on representing knowledge graph triplets in the distance space or semantic space. However, they can not fully capture the information of head and tail entities, nor even make good use of hierarchical level information. Thus, in this paper, we propose a novel knowledge graph embedding model for the link prediction task, namely, HIE, which models each triplet (\textit{h}, \textit{r}, \textit{t}) into distance measurement space and semantic measurement space, simultaneously. Moreover, HIE is introduced into hierarchical-aware space to leverage rich hierarchical information of entities and relations for better representation learning. Specifically, we apply distance transformation operation on the head entity in distance space to obtain the tail entity instead of translation-based or rotation-based approaches. Experimental results of HIE on four real-world datasets show that HIE outperforms several existing state-of-the-art knowledge graph embedding methods on the link prediction task and deals with complex relations accurately.Comment: Submitted to Big Data research one year ag