Probabilistic inference for determining options in reinforcement learning

Tasks that require many sequential decisions or complex solutions are hard to solve using conventional reinforcement learning algorithms. Based on the semi Markov decision process setting (SMDP) and the option framework, we propose a model which aims to alleviate these concerns. Instead of learning a single monolithic policy, the agent learns a set of simpler sub-policies as well as the initiation and termination probabilities for each of those sub-policies. While existing option learning algorithms frequently require manual specification of components such as the sub-policies, we present an algorithm which infers all relevant components of the option framework from data. Furthermore, the proposed approach is based on parametric option representations and works well in combination with current policy search methods, which are particularly well suited for continuous real-world tasks. We present results on SMDPs with discrete as well as continuous state-action spaces. The results show that the presented algorithm can combine simple sub-policies to solve complex tasks and can improve learning performance on simpler tasks

Are autistic traits in the general population stable across development?

There is accumulating evidence that autistic traits (AT) are on a continuum in the general population, with clinical autism representing the extreme end of a quantitative distribution. While the nature and severity of symptoms in clinical autism are known to persist over time, no study has examined the long-term stability of AT among typically developing toddlers. The current investigation measured AT in 360 males and 400 males from the general population close to two decades apart, using the Pervasive Developmental Disorder subscale of the Child Behavior Checklist in early childhood (M = 2.14 years; SD = 0.15), and the Autism-Spectrum Quotient in early adulthood (M = 19.50 years; SD = 0.70). Items from each scale were further divided into social (difficulties with social interaction and communication) and non-social (restricted and repetitive behaviours and interests) AT. The association between child and adult measurements of AT as well the influence of potentially confounding sociodemographic, antenatal and obstetric variables were assessed using Pearson's correlations and linear regression. For males, Total AT in early childhood were positively correlated with total AT (r = .16, p = .002) and social AT (r = .16, p = .002) in adulthood. There was also a positive correlation for males between social AT measured in early childhood and Total (r = .17, p = .001) and social AT (r = .16, p = .002) measured in adulthood. Correlations for non-social AT did not achieve significance in males. Furthermore, there was no significant longitudinal association in AT observed for males or females. Despite the constraints of using different measures and different raters at the two ages, this study found modest developmental stability of social AT from early childhood to adulthood in boys

Characteristic Evolution and Matching

I review the development of numerical evolution codes for general relativity based upon the characteristic initial value problem. Progress in characteristic evolution is traced from the early stage of 1D feasibility studies to 2D axisymmetric codes that accurately simulate the oscillations and gravitational collapse of relativistic stars and to current 3D codes that provide pieces of a binary black hole spacetime. Cauchy codes have now been successful at simulating all aspects of the binary black hole problem inside an artificially constructed outer boundary. A prime application of characteristic evolution is to extend such simulations to null infinity where the waveform from the binary inspiral and merger can be unambiguously computed. This has now been accomplished by Cauchy-characteristic extraction, where data for the characteristic evolution is supplied by Cauchy data on an extraction worldtube inside the artificial outer boundary. The ultimate application of characteristic evolution is to eliminate the role of this outer boundary by constructing a global solution via Cauchy-characteristic matching. Progress in this direction is discussed.Comment: New version to appear in Living Reviews 2012. arXiv admin note: updated version of arXiv:gr-qc/050809

Defining language impairments in a subgroup of children with autism spectrum disorder

Autism spectrum disorder (ASD) is diagnosed on the basis of core impairments in pragmatic language skills, which are found across all ages and subtypes. In contrast, there is significant heterogeneity in language phenotypes, ranging from nonverbal to superior linguistic abilities, as defined on standardized tests of vocabulary and grammatical knowledge. The majority of children are verbal but impaired in language, relative to age-matched peers. One hypothesis is that this subgroup has ASD and co-morbid specific language impairment (SLI). An experiment was conducted comparing children with ASD to children with SLI and typically developing controls on aspects of language processing that have been shown to be impaired in children with SLI: repetition of nonsense words. Patterns of performance among the children with ASD and language impairment were similar to those with SLI, and contrasted with the children with ASD and no language impairment and typical controls, providing further evidence for the hypothesis that a subgroup of children with ASD has co-morbid SLI. The findings are discussed in the context of brain imaging studies that have explored the neural bases of language impairment in ASD and SLI, and overlap in the genes associated with elevated risk for these disorders.M01 RR00533 - NCRR NIH HHS; R01 DC10290 - NIDCD NIH HHS; U19 DC03610 - NIDCD NIH HH

Combinatorial Roles of Heparan Sulfate Proteoglycans and Heparan Sulfates in Caenorhabditis elegans Neural Development

Heparan sulfate proteoglycans (HSPGs) play critical roles in the development and adult physiology of all metazoan organisms. Most of the known molecular interactions of HSPGs are attributed to the structurally highly complex heparan sulfate (HS) glycans. However, whether a specific HSPG (such as syndecan) contains HS modifications that differ from another HSPG (such as glypican) has remained largely unresolved. Here, a neural model in C. elegans is used to demonstrate for the first time the relationship between specific HSPGs and HS modifications in a defined biological process in vivo. HSPGs are critical for the migration of hermaphrodite specific neurons (HSNs) as genetic elimination of multiple HSPGs leads to 80% defect of HSN migration. The effects of genetic elimination of HSPGs are additive, suggesting that multiple HSPGs, present in the migrating neuron and in the matrix, act in parallel to support neuron migration. Genetic analyses suggest that syndecan/sdn-1 and HS 6-O-sulfotransferase, hst-6, function in a linear signaling pathway and glypican/lon-2 and HS 2-O-sulfotransferase, hst-2, function together in a pathway that is parallel to sdn-1 and hst-6. These results suggest core protein specific HS modifications that are critical for HSN migration. In C. elegans, the core protein specificity of distinct HS modifications may be in part regulated at the level of tissue specific expression of genes encoding for HSPGs and HS modifying enzymes. Genetic analysis reveals that there is a delicate balance of HS modifications and eliminating one HS modifying enzyme in a compromised genetic background leads to significant changes in the overall phenotype. These findings are of importance with the view of HS as a critical regulator of cell signaling in normal development and disease

Convergent genetic linkage and associations to language, speech and reading measures in families of probands with Specific Language Impairment

We analyzed genetic linkage and association of measures of language, speech and reading phenotypes to candidate regions in a single set of families ascertained for SLI. Sib-pair and family-based analyses were carried out for candidate gene loci for Reading Disability (RD) on chromosomes 1p36, 3p12-q13, 6p22, and 15q21, and the speech-language candidate region on 7q31 in a sample of 322 participants ascertained for Specific Language Impairment (SLI). Replication or suggestive replication of linkage was obtained in all of these regions, but the evidence suggests that the genetic influences may not be identical for the three domains. In particular, linkage analysis replicated the influence of genes on chromosome 6p for all three domains, but association analysis indicated that only one of the candidate genes for reading disability, KIAA0319, had a strong effect on language phenotypes. The findings are consistent with a multiple gene model of the comorbidity between language impairments and reading disability and have implications for neurocognitive developmental models and maturational processes

Movement of the human foot in 100 pain free individuals aged 18–45 : implications for understanding normal foot function

Background: Understanding motion in the normal healthy foot is a prerequisite for understanding the effects of pathology and thereafter setting targets for interventions. Quality foot kinematic data from healthy feet will also assist the development of high quality and research based clinical models of foot biomechanics. To address gaps in the current literature we aimed to describe 3D foot kinematics using a 5 segment foot model in a population of 100 pain free individuals. Methods: Kinematics of the leg, calcaneus, midfoot, medial and lateral forefoot and hallux were measured in 100 self reported healthy and pain free individuals during walking. Descriptive statistics were used to characterise foot movements. Contributions from different foot segments to the total motion in each plane were also derived to explore functional roles of different parts of the foot. Results: Foot segments demonstrated greatest motion in the sagittal plane, but large ranges of movement in all planes. All foot segments demonstrated movement throughout gait, though least motion was observed between the midfoot and calcaneus. There was inconsistent evidence of movement coupling between joints. There were clear differences in motion data compared to foot segment models reported in the literature. Conclusions: The data reveal the foot is a multiarticular structure, movements are complex, show incomplete evidence of coupling, and vary person to person. The data provide a useful reference data set against which future experimental data can be compared and may provide the basis for conceptual models of foot function based on data rather than anecdotal observations

Parental phonological memory contributes to prediction of outcome of late talkers from 20 months to 4 years: a longitudinal study of precursors of specific language impairment

Background Many children who are late talkers go on to develop normal language, but others go on to have longer-term language difficulties. In this study, we considered which factors were predictive of persistent problems in late talkers. Methods Parental report of expressive vocabulary at 18 months of age was used to select 26 late talkers and 70 average talkers, who were assessed for language and cognitive ability at 20 months of age. Follow-up at 4 years of age was carried out for 24 late and 58 average talkers. A psychometric test battery was used to categorize children in terms of language status (unimpaired or impaired) and nonverbal ability (normal range or more than 1 SD below average). The vocabulary and non-word repetition skills of the accompanying parent were also assessed. Results Among the late talkers, seven (29%) met our criteria for specific language impairment (SLI) at 4 years of age, and a further two (8%) had low nonverbal ability. In the group of average talkers, eight (14%) met the criteria for SLI at 4 years, and five other children (8%) had low nonverbal ability. Family history of language problems was slightly better than late-talker status as a predictor of SLI.. The best predictors of SLI at 20 months of age were score on the receptive language scale of the Mullen Scales of Early Learning and the parent's performance on a non-word repetition task. Maternal education was not a significant predictor of outcome. Conclusions In this study, around three-quarters of late talkers did not have any language difficulties at 4 years of age, provided there was no family history of language impairment. A family history of language-literacy problems was found to be a significant predictor for persisting problems. Nevertheless, there are children with SLI for whom prediction is difficult because they did not have early language delay

The Effect of Epstein-Barr Virus Latent Membrane Protein 2 Expression on the Kinetics of Early B Cell Infection

Infection of human B cells with wild-type Epstein-Barr virus (EBV) in vitro leads to activation and proliferation that result in efficient production of lymphoblastoid cell lines (LCLs). Latent Membrane Protein 2 (LMP2) is expressed early after infection and previous research has suggested a possible role in this process. Therefore, we generated recombinant EBV with knockouts of either or both protein isoforms, LMP2A and LMP2B (Δ2A, Δ2B, Δ2A/Δ2B) to study the effect of LMP2 in early B cell infection. Infection of B cells with Δ2A and Δ2A/Δ2B viruses led to a marked decrease in activation and proliferation relative to wild-type (wt) viruses, and resulted in higher percentages of apoptotic B cells. Δ2B virus infection showed activation levels comparable to wt, but fewer numbers of proliferating B cells. Early B cell infection with wt, Δ2A and Δ2B viruses did not result in changes in latent gene expression, with the exception of elevated LMP2B transcript in Δ2A virus infection. Infection with Δ2A and Δ2B viruses did not affect viral latency, determined by changes in LMP1/Zebra expression following BCR stimulation. However, BCR stimulation of Δ2A/Δ2B cells resulted in decreased LMP1 expression, which suggests loss of stability in viral latency. Long-term outgrowth assays revealed that LMP2A, but not LMP2B, is critical for efficient long-term growth of B cells in vitro. The lowest levels of activation, proliferation, and LCL formation were observed when both isoforms were deleted. These results suggest that LMP2A appears to be critical for efficient activation, proliferation and survival of EBV-infected B cells at early times after infection, which impacts the efficient long-term growth of B cells in culture. In contrast, LMP2B did not appear to play a significant role in these processes, and long-term growth of infected B cells was not affected by the absence of this protein. © 2013 Wasil et al