Simultaneous Perturbation Algorithms for Batch Off-Policy Search

We propose novel policy search algorithms in the context of off-policy, batch mode reinforcement learning (RL) with continuous state and action spaces. Given a batch collection of trajectories, we perform off-line policy evaluation using an algorithm similar to that by [Fonteneau et al., 2010]. Using this Monte-Carlo like policy evaluator, we perform policy search in a class of parameterized policies. We propose both first order policy gradient and second order policy Newton algorithms. All our algorithms incorporate simultaneous perturbation estimates for the gradient as well as the Hessian of the cost-to-go vector, since the latter is unknown and only biased estimates are available. We demonstrate their practicality on a simple 1-dimensional continuous state space problem

Adhesion and non-linear rheology of adhesives with supramolecular crosslinking points

Soft supramolecular materials are promising for the design of innovative and highly tunable adhesives. These materials are composed of polymer chains functionalized by strongly interacting moieties, sometimes called "stickers". In order to systematically investigate the effect of the presence of associative groups on the debonding properties of a supramolecular adhesive, a series of supramolecular model systems has been characterized by probe-tack tests. These model materials, composed of linear and low dispersity poly(butylacrylate) chains functionalized in the middle by a single tri-urea sticker, are able to self-associate by six hydrogen bonds and range in molecular weight (M n) between 5 and 85 kg/mol. The linear rheology and the nanostructure of the same materials (called "PnBA3U") was the object of a previous study 1,2. At room temperature, the association of polymers via hydrogen bonds induces the formation of rod-like aggregates structured into bundles for M n \textless{} 40kg/mol and the behavior of a soft elastic material was observed (G'\textgreater{}\textgreater{}G "and G'~$\omega$ 0). For higher M n , the filaments were randomly oriented and polymers displayed a crossover towards viscous behavior although terminal relaxation was not reached in the experimental frequency window. All these materials show however similar adhesive properties characterized by a cohesive mode of failure and low debonding energies (W adh \textless{}40J/m 2 for a debonding speed of 100$\mu$m/s). The debonding mechanisms observed during the adhesion tests have been investigated in detail with an Image tools analysis developed by our group 3. The measure of the projected area covered by cavities growing in the adhesive layer during debonding can be used to estimate the true stress in the walls of the cavities and thus, to characterize the in-situ large strain deformation of the thin layer during the adhesion test itself. This analysis revealed in particular that the PnBA3U materials with M n \textless{} 40 kg/mol soften very markedly at large deformation like yield stress fluids, explaining the low adhesion energies measured for these viscoelastic gels.

Determining Crack Aperture Distribution in Rocks Using the C-14-PMMA Autoradiographic Method : Experiments and Simulations

Because cracks control the global mechanical and transport properties of crystalline rocks, it is of a crucial importance to suitably determine their aperture distribution, which evolves through alteration processes and rock weathering. Due to the high variability of crack networks in rocks, a multiscale approach is needed. The C-14-PMMA (polymethylmethacrylate) method was developed to determine crack apertures using a set of artificial crack samples with different controlled apertures and tilt angles and also using Monte Carlo simulations. The experiments and simulations show the same result: the estimation of apparent aperture w(A) was successful regardless of tilt angle, even if the estimates are less accurate for low tilt angles (Peer reviewe

Relating Dynamic Brain States to Dynamic Machine States: Human and Machine Solutions to the Speech Recognition Problem

There is widespread interest in the relationship between the neurobiological systems supporting human cognition and emerging computational systems capable of emulating these capacities. Human speech comprehension, poorly understood as a neurobiological process, is an important case in point. Automatic Speech Recognition (ASR) systems with near-human levels of performance are now available, which provide a computationally explicit solution for the recognition of words in continuous speech. This research aims to bridge the gap between speech recognition processes in humans and machines, using novel multivariate techniques to compare incremental ‘machine states’, generated as the ASR analysis progresses over time, to the incremen- tal ‘brain states’, measured using combined electro- and magneto-encephalography (EMEG), generated as the same inputs are heard by human listeners. This direct comparison of dynamic human and machine internal states, as they respond to the same incrementally delivered sensory input, revealed a significant correspondence between neural response patterns in human superior temporal cortex and the structural properties of ASR-derived phonetic models. Spatially coherent patches in human temporal cortex responded selectively to individual phonetic features defined on the basis of machine-extracted regularities in the speech to lexicon mapping process. These results demonstrate the feasibility of relating human and ASR solutions to the problem of speech recognition, and suggest the potential for further studies relating complex neural computations in human speech comprehension to the rapidly evolving ASR systems that address the same problem domain.This research was supported financially by an Advanced Investigator grant to WMW from the European Research Council (AdG 230570 NEUROLEX), by MRC Cognition and Brain Sciences Unit (CBSU) funding to WMW (U.1055.04.002.00001.01), and by a European Research Council Advanced Investigator grant under the European Community’s Horizon 2020 Research and Innovation Programme (2014-2020 ERC Grant agreement no 669820) to Lorraine K. Tyler. LS was partly supported by the NIHR Biomedical Research Centre and Biomedical Unit in Dementia based at Cambridge University Hospital NHS Foundation Trust

Relative Impacts of Adult Movement, Larval Dispersal and Harvester Movement on the Effectiveness of Reserve Networks

Movement of individuals is a critical factor determining the effectiveness of reserve networks. Marine reserves have historically been used for the management of species that are sedentary as adults, and, therefore, larval dispersal has been a major focus of marine-reserve research. The push to use marine reserves for managing pelagic and demersal species poses significant questions regarding their utility for highly-mobile species. Here, a simple conceptual metapopulation model is developed to provide a rigorous comparison of the functioning of reserve networks for populations with different admixtures of larval dispersal and adult movement in a home range. We find that adult movement produces significantly lower persistence than larval dispersal, all other factors being equal. Furthermore, redistribution of harvest effort previously in reserves to remaining fished areas (‘fishery squeeze’) and fishing along reserve borders (‘fishing-the-line’) considerably reduce persistence and harvests for populations mobile as adults, while they only marginally changes results for populations with dispersing larvae. Our results also indicate that adult home-range movement and larval dispersal are not simply additive processes, but rather that populations possessing both modes of movement have lower persistence than equivalent populations having the same amount of ‘total movement’ (sum of larval and adult movement spatial scales) in either larval dispersal or adult movement alone

Electrical Brain Responses in Language-Impaired Children Reveal Grammar-Specific Deficits

Background: Scientific and public fascination with human language have included intensive scrutiny of language disorders as a new window onto the biological foundations of language and its evolutionary origins. Specific language impairment (SLI), which affects over 7% of children, is one such disorder. SLI has received robust scientific attention, in part because of its recent linkage to a specific gene and loci on chromosomes and in part because of the prevailing question regarding the scope of its language impairment: Does the disorder impact the general ability to segment and process language or a specific ability to compute grammar? Here we provide novel electrophysiological data showing a domain-specific deficit within the grammar of language that has been hitherto undetectable through behavioural data alone. Methods and Findings: We presented participants with Grammatical(G)-SLI, age-matched controls, and younger child and adult controls, with questions containing syntactic violations and sentences containing semantic violations. Electrophysiological brain responses revealed a selective impairment to only neural circuitry that is specific to grammatical processing in G-SLI. Furthermore, the participants with G-SLI appeared to be partially compensating for their syntactic deficit by using neural circuitry associated with semantic processing and all non-grammar-specific and low-level auditory neural responses were normal. Conclusions: The findings indicate that grammatical neural circuitry underlying language is a developmentally unique system in the functional architecture of the brain, and this complex higher cognitive system can be selectively impaired. The findings advance fundamental understanding about how cognitive systems develop and all human language is represented and processed in the brain

Connected by sea, disconnected by tuna? Challenges to regionalism in the Southwest Indian Ocean

Madagascar, Mauritius and Seychelles are at the center of industrial tuna extraction in the Southwest Indian Ocean (SWIO). In this paper, we show that, while a discourse of regionalism between the three islands is prominent, the possibilities of regionalism face deep challenges in relation to the tuna industry. This is due to three factors. First, local perceptions, especially amongst those working in and on the tuna industry, are in disconnection with an ‘Indianoceania’ vision. Second, the geopolitics between coastal states and distant water fishing nations create various entanglements including through fishing access revenue and foreign aid. Finally, the materiality of tuna can at times create competition as countries seek to individually maximize benefits from the industry. We argue that the active reinforcement of regional identity and collaboration around this resource is necessary to sustain local benefits into the future

Combined antiviral activity of interferon-α and RNA interference directed against hepatitis C without affecting vector delivery and gene silencing

The current standard interferon-alpha (IFN-α)-based therapy for chronic hepatitis C virus (HCV) infection is only effective in approximately half of the patients, prompting the need for alternative treatments. RNA interference (RNAi) represents novel approach to combat HCV by sequence-specific targeting of viral or host factors involved in infection. Monotherapy of RNAi, however, may lead to therapeutic resistance by mutational escape of the virus. Here, we proposed that combining lentiviral vector-mediated RNAi and IFN-α could be more effective and avoid therapeutic resistance. In this study, we found that IFN-α treatment did not interfere with RNAi-mediated gene silencing. RNAi and IFN-α act independently on HCV replication showing combined antiviral activity when used simultaneously or sequentially. Transduction of mouse hepatocytes in vivo and in vitro was not effected by IFN-α treatment. In conclusion, RNAi and IFN-α can be effectively combined without cross-interference and may represent a promising combinational strategy for the treatment of hepatitis C

An Investigation to Validate the Grammar and Phonology Screening (GAPS) Test to Identify Children with Specific Language Impairment

The extraordinarily high incidence of grammatical language impairments in developmental disorders suggests that this uniquely human cognitive function is "fragile". Yet our understanding of the neurobiology of grammatical impairments is limited. Furthermore, there is no "gold-standard" to identify grammatical impairments and routine screening is not undertaken. An accurate screening test to identify grammatical abilities would serve the research, health and education communities, further our understanding of developmental disorders, and identify children who need remediation, many of whom are currently un-diagnosed. A potential realistic screening tool that could be widely administered is the Grammar and Phonology Screening (GAPS) test--a 10 minute test that can be administered by professionals and non-professionals alike. Here we provide a further step in evaluating the validity and accuracy (sensitivity and specificity) of the GAPS test in identifying children who have Specific Language Impairment (SLI)

Plasmacytoid Dendritic Cells Capture and Cross-Present Viral Antigens from Influenza-Virus Exposed Cells

Among the different subsets of dendritic cells (DC), plasmacytoid dendritic cells (PDC) play a unique role in secreting large amounts of type I interferons upon viral stimulation, but their efficiency as antigen-presenting cells has not been completely characterized. We show here, by flow cytometry, with human primary blood PDC and with a PDC cell line, that PDC display poor endocytic capacity for soluble or cellular antigens when compared to monocyte-derived myeloid DC. However, immature PDC efficiently take up cellular material from live influenza-exposed cells, subsequently mature and cross-present viral antigens very efficiently to specific CD8+ T cells. Therefore, during viral infection PDC not only secrete immunomodulatory cytokines, but also recognize infected cells and function as antigen cross-presenting cells to trigger the anti-viral immune response