Symbolic Knowledge Distillation: from General Language Models to Commonsense Models

The common practice for training commonsense models has gone from-human-to-corpus-to-machine: humans author commonsense knowledge graphs in order to train commonsense models. In this work, we investigate an alternative, from-machine-to-corpus-to-machine: general language models author these commonsense knowledge graphs to train commonsense models. Our study leads to a new framework, Symbolic Knowledge Distillation. As with prior art in Knowledge Distillation (Hinton et al., 2015), our approach uses larger models to teach smaller models. A key difference is that we distill knowledge symbolically-as text-in addition to the neural model. We also distill only one aspect-the commonsense of a general language model teacher, allowing the student to be a different type, a commonsense model. Altogether, we show that careful prompt engineering and a separately trained critic model allow us to selectively distill high-quality causal commonsense from GPT-3, a general language model. Empirical results demonstrate that, for the first time, a human-authored commonsense knowledge graph is surpassed by our automatically distilled variant in all three criteria: quantity, quality, and diversity. In addition, it results in a neural commonsense model that surpasses the teacher model's commonsense capabilities despite its 100x smaller size. We apply this to the ATOMIC resource, and share our new symbolic knowledge graph and commonsense models

More than a Metric:How Training Load is Used in Elite Sport for Athlete Management

Training load monitoring is a core aspect of modern-day sport science practice. Collecting, cleaning, analysing, interpreting, and disseminating load data is usually undertaken with a view to improve player performance and/or manage injury risk. To target these outcomes, practitioners attempt to optimise load at different stages throughout the training process, like adjusting individual sessions, planning day-to-day, periodising the season, and managing athletes with a long-term view. With greater investment in training load monitoring comes greater expectations, as stakeholders count on practitioners to transform data into informed, meaningful decisions. In this editorial we highlight how training load monitoring has many potential applications and cannot be simply reduced to one metric and/or calculation. With experience across a variety of sporting backgrounds, this editorial details the challenges and contextual factors that must be considered when interpreting such data. It further demonstrates the need for those working with athletes to develop strong communication channels with all stakeholders in the decision-making process. Importantly, this editorial highlights the complexity associated with using training load for managing injury risk and explores the potential for framing training load with a performance and training progression mindset.</p

More than a Metric:How Training Load is Used in Elite Sport for Athlete Management

Training load monitoring is a core aspect of modern-day sport science practice. Collecting, cleaning, analysing, interpreting, and disseminating load data is usually undertaken with a view to improve player performance and/or manage injury risk. To target these outcomes, practitioners attempt to optimise load at different stages throughout the training process, like adjusting individual sessions, planning day-to-day, periodising the season, and managing athletes with a long-term view. With greater investment in training load monitoring comes greater expectations, as stakeholders count on practitioners to transform data into informed, meaningful decisions. In this editorial we highlight how training load monitoring has many potential applications and cannot be simply reduced to one metric and/or calculation. With experience across a variety of sporting backgrounds, this editorial details the challenges and contextual factors that must be considered when interpreting such data. It further demonstrates the need for those working with athletes to develop strong communication channels with all stakeholders in the decision-making process. Importantly, this editorial highlights the complexity associated with using training load for managing injury risk and explores the potential for framing training load with a performance and training progression mindset.</p

A First Look at White Dwarf - M Dwarf Pairs in the Sloan Digital Sky Survey (SDSS)

We have identified 109 White Dwarf (WD) - M dwarf pairs in the Sloan Digital Sky Survey (SDSS) with g < 20th magnitude. For each system we determined the temperature of the WD primary and the spectral type of the M dwarf secondary. Using H-alpha emission as a proxy for the chromospheric activity level of the M dwarf, we investigated correlations between the activity level and properties of the system. Compared with field M dwarfs (Hawley et al. 1996), we see a slightly higher active fraction of early-type stars, with activity levels similar to the field. We have conducted followup observations at the ARC 3.5m telescope to obtain radial velocity information and to search for short period binaries, which may be on the verge of interacting. We report on one system with a 4.1 hour period, and several additional systems with significant velocity variations.Comment: 23 pages with 10 figures. Accepted by AJ, to appear in May 2003 issu

Long-term patterns of hillslope erosion by earthquake-induced landslides shape mountain landscapes

Widespread triggering of landslides by large storms or earthquakes is a dominant mechanism of erosion in mountain landscapes. If landslides occur repeatedly in particular locations within a mountain range, then they will dominate the landscape evolution of that section and could leave a fingerprint in the topography. Here, we track erosion provenance using a novel combination of the isotopic and molecular composition of organic matter deposited in Lake Paringa, New Zealand. We find that the erosion provenance has shifted markedly after four large earthquakes over 1000 years. Postseismic periods eroded organic matter from a median elevation of 722 +329/−293 m and supplied 43% of the sediment in the core, while interseismic periods sourced from lower elevations (459 +256/−226 m). These results are the first demonstration that repeated large earthquakes can consistently focus erosion at high elevations, while interseismic periods appear less effective at modifying the highest parts of the topography

A Strategy for Finding Near Earth Objects with the SDSS Telescope

We present a detailed observational strategy for finding Near Earth Objects (NEOs) with the Sloan Digital Sky Survey (SDSS) telescope. We investigate strategies in normal, unbinned mode as well as binning the CCDs 2x2 or 3x3, which affects the sky coverage rate and the limiting apparent magnitude. We present results from 1 month, 3 year and 10 year simulations of such surveys. For each cadence and binning mode, we evaluate the possibility of achieving the Spaceguard goal of detecting 90% of 1 km NEOs (absolute magnitude H <= 18 for an albedo of 0.1). We find that an unbinned survey is most effective at detecting H <= 20 NEOs in our sample. However, a 3x3 binned survey reaches the Spaceguard Goal after only seven years of operation. As the proposed large survey telescopes (PanStarss; LSST) are at least 5-10 years from operation, an SDSS NEO survey could make a significant contribution to the detection and photometric characterization of the NEO population.Comment: Accepted by AJ -- 12 pages, 11 figure

Faith and Fate: Limits of Transformers on Compositionality

Transformer large language models (LLMs) have sparked admiration for their exceptional performance on tasks that demand intricate multi-step reasoning. Yet, these models simultaneously show failures on surprisingly trivial problems. This begs the question: Are these errors incidental, or do they signal more substantial limitations? In an attempt to demystify Transformers, we investigate the limits of these models across three representative compositional tasks -- multi-digit multiplication, logic grid puzzles, and a classic dynamic programming problem. These tasks require breaking problems down into sub-steps and synthesizing these steps into a precise answer. We formulate compositional tasks as computation graphs to systematically quantify the level of complexity, and break down reasoning steps into intermediate sub-procedures. Our empirical findings suggest that Transformers solve compositional tasks by reducing multi-step compositional reasoning into linearized subgraph matching, without necessarily developing systematic problem-solving skills. To round off our empirical study, we provide theoretical arguments on abstract multi-step reasoning problems that highlight how Transformers' performance will rapidly decay with increased task complexity.Comment: 10 pages + appendix (21 pages

Sediment Cores from White Pond, South Carolina, contain a Platinum Anomaly, Pyrogenic Carbon Peak, and Coprophilous Spore Decline at 12.8 ka

A widespread platinum (Pt) anomaly was recently documented in Greenland ice and 11 North American sedimentary sequences at the onset of the Younger Dryas (YD) event (~12,800 cal yr BP), consistent with the YD Impact Hypothesis. We report high-resolution analyses of a 1-meter section of a lake core from White Pond, South Carolina, USA. After developing a Bayesian age-depth model that brackets the late Pleistocene through early Holocene, we analyzed and quantified the following: (1) Pt and palladium (Pd) abundance, (2) geochemistry of 58 elements, (3) coprophilous spores, (4) sedimentary organic matter (OC and sedaDNA), (5) stable isotopes of C (δ13C) and N (δ15N), (6) soot, (7) aciniform carbon, (8) cryptotephra, (9) mercury (Hg), and (10) magnetic susceptibility. We identified large Pt and Pt/Pd anomalies within a 2-cm section dated to the YD onset (12,785 ± 58 cal yr BP). These anomalies precede a decline in coprophilous spores and correlate with an abrupt peak in soot and C/OC ratios, indicative of large-scale regional biomass burning. We also observed a relatively large excursion in δ15N values, indicating rapid climatic and environmental/hydrological changes at the YD onset. Our results are consistent with the YD Impact Hypothesis and impact-related environmental and ecological changes

The Durability of Public Goods Changes the Dynamics and Nature of Social Dilemmas

An implicit assumption underpins basic models of the evolution of cooperation, mutualism and altruism: The benefits (or pay-offs) of cooperation and defection are defined by the current frequency or distribution of cooperators. In social dilemmas involving durable public goods (group resources that can persist in the environment–ubiquitous from microbes to humans) this assumption is violated. Here, we examine the consequences of relaxing this assumption, allowing pay-offs to depend on both current and past numbers of cooperators. We explicitly trace the dynamic of a public good created by cooperators, and define pay-offs in terms of the current public good. By raising the importance of cooperative history in determining the current fate of cooperators, durable public goods cause novel dynamics (e.g., transient increases in cooperation in Prisoner's Dilemmas, oscillations in Snowdrift Games, or shifts in invasion thresholds in Stag-hunt Games), while changes in durability can transform one game into another, by moving invasion thresholds for cooperation or conditions for coexistence with defectors. This enlarged view challenges our understanding of social cheats. For instance, groups of cooperators can do worse than groups of defectors, if they inherit fewer public goods, while a rise in defectors no longer entails a loss of social benefits, at least not in the present moment (as highlighted by concerns over environmental lags). Wherever durable public goods have yet to reach a steady state (for instance due to external perturbations), the history of cooperation will define the ongoing dynamics of cooperators