Pretrain on just structure: Understanding linguistic inductive biases using transfer learning

Both humans and transformer language models are able to learn language without explicit structural supervision. What inductive learning biases make this learning possible? In this study, we examine the effect of different inductive learning biases by predisposing language models with structural biases through pretraining on artificial structured data, and then evaluating by fine-tuning on English. Our experimental setup gives us the ability to actively control the inductive bias of language models. With our experiments, we investigate the comparative success of three types of inductive bias: 1) an inductive bias for recursive, hierarchical processing 2) an inductive bias for unrestricted token-token dependencies that can't be modeled by context-free grammars, and 3) an inductive bias for a Zipfian power-law vocabulary distribution. We show that complex token-token interactions form the best inductive biases, and that this is strongest in the non-context-free case. We also show that a Zipfian vocabulary distribution forms a good inductive bias independently from grammatical structure. Our study leverages the capabilities of transformer models to run controlled language learning experiments that are not possible to run in humans, and surfaces hypotheses about the structures that facilitate language learning in both humans and machines

The Effects of Mental Health Literacy and Perceived Social Support on Mental Health Stigma Across Racial and Ethnic Groups in the United States: A Survey-based Analysis

The goals of this study were to determine a) whether minoritized groups in the United States had lower levels of mental health literacy (MHL) and higher levels of stigma, b) whether people with higher rates of MHL and social support had lower rates of stigma and c) whether MHL and perceived social support were stronger predictors of stigma levels depending on racial group membership. I hypothesized that people with both high levels of MHL and high levels of social support would have lower levels of mental health stigma, minoritized groups would have lower levels of MHL and social support and therefore higher levels of stigma in comparison to Non-Hispanic White Americans, and finally, race would be a significant moderator in the relationship between MHL and perceived social support on stigma levels. This study contained a total of 510 participants: 93 Asian Americans, 107 Black or African Americans, 95 Hispanic or Latinx Americans, and 215 White Americans. Participants were recruited through Amazon’s MTurk service and were given a self-reported survey to measure MHL, perceived social support, and stigma levels. The results indicated MHL and social support significantly affected stigma levels in participants, with MHL having a significant negative relationship with stigma and social support having a significant positive relationship with stigma. However further analysis showed no significant difference in MHL or stigma rates across the different races, refuting the second hypothesis. Finally, race was not found to be a significant moderator in the relationship between MHL, social support, and stigma

TGFβ signaling is associated with changes in inflammatory gene expression and perineuronal net degradation around inhibitory neurons following various neurological insults.

Brain damage due to stroke or traumatic brain injury (TBI), both leading causes of serious long-term disability, often leads to the development of epilepsy. Patients who develop post-injury epilepsy tend to have poor functional outcomes. Emerging evidence highlights a potential role for blood-brain barrier (BBB) dysfunction in the development of post-injury epilepsy. However, common mechanisms underlying the pathological hyperexcitability are largely unknown. Here, we show that comparative transcriptome analyses predict remodeling of extracellular matrix (ECM) as a common response to different types of injuries. ECM-related transcriptional changes were induced by the serum protein albumin via TGFβ signaling in primary astrocytes. In accordance with transcriptional responses, we found persistent degradation of protective ECM structures called perineuronal nets (PNNs) around fast-spiking inhibitory interneurons, in a rat model of TBI as well as in brains of human epileptic patients. Exposure of a naïve brain to albumin was sufficient to induce the transcriptional and translational upregulation of molecules related to ECM remodeling and the persistent breakdown of PNNs around fast-spiking inhibitory interneurons, which was contingent on TGFβ signaling activation. Our findings provide insights on how albumin extravasation that occurs upon BBB dysfunction in various brain injuries can predispose neural circuitry to the development of chronic inhibition deficits

Plant Community Composition and Structure Monitoring for Agate Fossil Beds National Monument

Agate Fossil Beds National Monument (AGFO) plays a vital role in protecting and managing some of the last remnants of native mixed-grass prairie in the region. The Northern Great Plains Inventory & Monitoring Network (NGPN) and Fire Ecology Program (FireEP) surveyed 12 long-term monitoring plots in Agate Fossil Beds National Monument in 2012 as part of an effort to better understand the condition of plant communities in the park. We measured plant diversity and cover, looked for the presence of exotic species that may be newly invading the park, and evaluated the amount of human and natural disturbance at all plots. This effort was the second year in a multiple-year venture to document the current status and long-term trends in plant communities in AGFO. At the end of five years, there will be an in-depth report describing the status of the plant community. In addition to upland plant monitoring, we also sampled vegetation at 12 sites along the riparian corridor at AGFO as part of a pilot study to develop a long-term monitoring approach for this area. The riparian corridor is narrow and not adequately represented in our standard sampling, but is of great ecological and management importance to the park. In 2013, we will also revisit legacy plots that were established as part of the Prairie Cluster prototype monitoring. In this report, we provide a simple summary of our results from sampling in 2012

Individual and group preferences of water taxi owners for electric outboard engines

This paper investigates individual and collective preferences of water taxi owners regarding electric outboard engines and batteries in the Galápagos Islands. The study addresses a gap in the literature on clean maritime transportation and deploys a novel mixed-method approach, using qualitative methods to complement discrete choice experiments. Results from mixed logit models and focus groups reveal a preference for smaller, lighter electric systems that can both provide similar speed and power to internal combustion counterparts while avoiding economic loss or excessive costs to service providers. Water taxi owners on different islands arrived at different collective preferences for charging infrastructure, thus illustrating the relevance of local context, and the importance of the participation of local actors in the successful design of systems and strategies facilitating maritime transport decarbonization. Additionally, identified instruments to reduce uncertainties and proposed subsidies offer practical considerations for accelerating the transition from internal combustion engines to electric propulsion.</p

Binding of per- and polyfluoroalkyl substances (PFASs) by organic soil materials with different structural composition - Charge- and concentration-dependent sorption behavior

The charge-and concentration-dependent sorption behavior of a range of per-and polyfluoroalkyl substances (PFASs) was studied for three organic soil samples with different organic matter quality, one Spodosol Oe horizon (Mor Oe) and two Sphagnum peats with different degrees of decomposition (Peat Oi and Peat Oe). Sorption to the two peat materials was, on average, four times stronger compared to that onto the Mor Oe material. In particular, longer-chained PFASs were more strongly bound by the two peats as compared to the Mor Oe sample. The combined results of batch sorption experiments and 13C NMR spectroscopy suggested sorption to be positively related to the content of carbohydrates (i.e., O-alkyl carbon). Sorption of all PFAS subclasses was inversely related to the pH value in all soils, with the largest pH effects being observed for perfluoroalkyl carboxylates (PFCAs) with C-10 and C-11 perfluorocarbon chain lengths. Experimentally determined sorption isotherms onto the poorly humified Peat Oi did not deviate significantly from linearity for most substances, while for the Mor Oe horizon, sorption nonlinearity was generally more pronounced. This work should prove useful in assessing PFAS sorption and leaching in organic soil horizons within environmental risk assessment

Report on second selection of resources, revising selection in D2.1

The central objective of the Metanet4u project is to contribute to the establishment of a pan-European digital platform that makes available language resources and services, encompassing both datasets and software tools, for speech and language processing, and supports a new generation of exchange facilities for them.Peer ReviewedPreprin

Report on first selection of resources

The central objective of the Metanet4u project is to contribute to the establishment of a pan-European digital platform that makes available language resources and services, encompassing both datasets and software tools, for speech and language processing, and supports a new generation of exchange facilities for them.Peer ReviewedPreprin

Consistency of the Disposition Index in the Face of Diet Induced Insulin Resistance: Potential Role of FFA

Objective Insulin resistance induces hyperinsulinemic compensation, which in turn maintains almost a constant disposition index. However, the signal that gives rise to the hyperinsulinemic compensation for insulin resistance remains unknown. Methods In a dog model of obesity we examined the possibility that potential early-week changes in plasma FFA, glucose, or both could be part of a cascade of signals that lead to compensatory hyperinsulinemia induced by insulin resistance. Results Hypercaloric high fat feeding in dogs resulted in modest weight gain, and an increase in adipose tissue with no change in the non-adipose tissue size. To compensate for the drop in insulin sensitivity, there was a significant rise in plasma insulin, which can be attributed in part to a decrease in the metabolic clearance rate of insulin and increased insulin secretion. In this study we observed complete compensation for high fat diet induced insulin resistance as measured by the disposition index. The compensatory hyperinsulinemia was coupled with significant changes in plasma FFAs and no change in plasma glucose. Conclusions We postulate that early in the development of diet induced insulin resistance, a change in plasma FFAs may directly, through signaling at the level of β-cell, or indirectly, by decreasing hepatic insulin clearance, result in the observed hyperinsulinemic compensation