Temporal-Difference Learning to Assist Human Decision Making during the Control of an Artificial Limb

In this work we explore the use of reinforcement learning (RL) to help with human decision making, combining state-of-the-art RL algorithms with an application to prosthetics. Managing human-machine interaction is a problem of considerable scope, and the simplification of human-robot interfaces is especially important in the domains of biomedical technology and rehabilitation medicine. For example, amputees who control artificial limbs are often required to quickly switch between a number of control actions or modes of operation in order to operate their devices. We suggest that by learning to anticipate (predict) a user's behaviour, artificial limbs could take on an active role in a human's control decisions so as to reduce the burden on their users. Recently, we showed that RL in the form of general value functions (GVFs) could be used to accurately detect a user's control intent prior to their explicit control choices. In the present work, we explore the use of temporal-difference learning and GVFs to predict when users will switch their control influence between the different motor functions of a robot arm. Experiments were performed using a multi-function robot arm that was controlled by muscle signals from a user's body (similar to conventional artificial limb control). Our approach was able to acquire and maintain forecasts about a user's switching decisions in real time. It also provides an intuitive and reward-free way for users to correct or reinforce the decisions made by the machine learning system. We expect that when a system is certain enough about its predictions, it can begin to take over switching decisions from the user to streamline control and potentially decrease the time and effort needed to complete tasks. This preliminary study therefore suggests a way to naturally integrate human- and machine-based decision making systems.Comment: 5 pages, 4 figures, This version to appear at The 1st Multidisciplinary Conference on Reinforcement Learning and Decision Making, Princeton, NJ, USA, Oct. 25-27, 201

Sexual recombination is a signature of a persisting malaria epidemic in Peru

<p>Abstract</p> <p>Background</p> <p>The aim of this study was to consider the impact that multi-clone, complex infections have on a parasite population structure in a low transmission setting. In general, complexity of infection (minimum number of clones within an infection) and the overall population level diversity is expected to be minimal in low transmission settings. Additionally, the parasite population structure is predicted to be clonal, rather than sexual due to infrequent parasite inoculation and lack of recombination between genetically distinct clones. However, in this low transmission of the Peruvian Amazon, complex infections are becoming more frequent, in spite of decreasing infection prevalence. In this study, it was hypothesized that sexual recombination between distinct clonal lineages of <it>Plasmodium falciparum </it>parasites were altering the subpopulation structure and effectively maintaining the population-level diversity.</p> <p>Methods</p> <p>Fourteen microsatellite markers were chosen to describe the genetic diversity in 313 naturally occurring <it>P. falciparum </it>infections from Peruvian Amazon. The population and subpopulation structure was characterized by measuring: clusteredness, expected heterozygosity (H_e), allelic richness, private allelic richness, and linkage disequilibrium. Next, microsatellite haplotypes and alleles were correlated with <it>P. falciparum </it>merozoite surface protein 1 Block 2 (<it>Pf</it>msp1-B2) to examine the presence of recombinant microsatellite haplotypes.</p> <p>Results</p> <p>The parasite population structure consists of six genetically diverse subpopulations of clones, called "clusters". Clusters 1, 3, 4, and 6 have unique haplotypes that exceed 70% of the total number of clones within each cluster, while Clusters 2 and 5 have a lower proportion of unique haplotypes, but still exceed 46%. By measuring the H_e, allelic richness, and private allelic richness within each of the six subpopulations, relatively low levels of genetic diversity within each subpopulation (except Cluster 4) are observed. This indicated that the number of alleles, and not the combination of alleles, are limited. Next, the standard index of association (I_A^S) was measured, which revealed a significant decay in linkage disequilibrium (LD) associated with Cluster 6, which is indicative of independent assortment of alleles. This decay in LD is a signature of this subpopulation approaching linkage equilibrium by undergoing sexual recombination. To trace possible recombination events, the two most frequent microsatellite haplotypes observed over time (defined by either a K1 or Mad20) were selected as the progenitors and then potential recombinants were identified in within the natural population.</p> <p>Conclusions</p> <p>Contrary to conventional low transmission models, this study provides evidence of a parasite population structure that is superficially defined by a clonal backbone. Sexual recombination does occur and even arguably is responsible for maintaining the substructure of this population.</p

The Plasmodium falciparum merozoite surface protein-1 19 KD antibody response in the Peruvian Amazon predominantly targets the non-allele specific, shared sites of this antigen

<p>Abstract</p> <p>Background</p> <p><it>Plasmodium falciparum </it>re-emerged in Iquitos, Peru in 1994 and is now hypoendemic (< 0.5 infections/person/year). Purportedly non-immune individuals with discrete (non-overlapping) <it>P. falciparum </it>infections can be followed using this population dynamic. Previous work demonstrated a strong association between this population's antibody response to <it>Pf</it>MSP1-19KD and protection against febrile illness and parasitaemia. Therefore, some selection for <it>Pf</it>MSP1-19KD allelic diversity would be expected if the protection is to allele-specific sites of <it>Pf</it>MSP1-19KD. Here, the potential for allele-specific polymorphisms in this population is investigated, and the allele-specificity of antibody responses to <it>Pf</it>MSP1-19KD are determined.</p> <p>Methods</p> <p>The 42KD region in <it>Pf</it>MSP1 was genotyped from 160 individual infections collected between 2003 and 2007. Additionally, the polymorphic block 2 region of <it>Pfmsp1 </it>(<it>Pfmsp1</it>-B2) was genotyped in 781 infection-months to provide a baseline for population-level diversity. To test whether <it>Pf</it>MSP1-19KD genetic diversity had any impact on antibody responses, ELISAs testing IgG antibody response were performed on individuals using all four allele-types of <it>Pf</it>MSP1-19KD. An antibody depletion ELISA was used to test the ability of antibodies to cross-react between allele-types.</p> <p>Results</p> <p>Despite increased diversity in <it>Pfmsp1</it>-B2, limited diversity within <it>Pfmsp1</it>-42KD was observed. All 160 infections genotyped were Mad20-like at the <it>Pfmsp1</it>-33KD locus. In the <it>Pfmsp1</it>-19KD locus, 159 (99.4%) were the Q-KSNG-F haplotype and 1 (0.6%) was the E-KSNG-L haplotype. Antibody responses in 105 individuals showed that Q-KNG and Q-TSR alleles generated the strongest immune responses, while Q-KNG and E-KNG responses were more concordant with each other than with those from Q-TSR and E-TSR, and vice versa. The immuno-depletion ELISAs showed all samples responded to the antigenic sites shared amongst all allelic forms of <it>Pf</it>MSP1-19KD.</p> <p>Conclusions</p> <p>A non-allele specific antibody response in <it>Pf</it>MSP1-19KD may explain why other allelic forms have not been maintained or evolved in this population. This has important implications for the use of <it>Pf</it>MSP1-19KD as a vaccine candidate. It is possible that Peruvians have increased antibody responses to the shared sites of <it>Pf</it>MSP1-19KD, either due to exposure/parasite characteristics or due to a human-genetic predisposition. Alternatively, these allelic polymorphisms are not immune-specific even in other geographic regions, implying these polymorphisms may be less important in immune evasion that previous studies suggest.</p

Bounding the graviton mass with binary pulsar observations

By comparing the observed orbital decay of the binary pulsars PSRB1913+16 and PSRB1534+12 to that predicted by general relativity due to gravitational-wave emission, we are able to bound the mass of the graviton to be less than $7.6\times10^{-20} \text{eV}/c^2$ at 90% confidence. This is the first such bound to be derived from dynamic gravitational fields. It is approximately two orders of magnitude weaker than the static-field bound from solar system observations, and will improve with further observations.Comment: 9 pages, 1 figure. Presented at Fourth Edoardo Amaldi Conference on Gravitational Waves, Perth, 200

Bayesian detection of unmodeled bursts of gravitational waves

The data analysis problem of coherently searching for unmodeled gravitational-wave bursts in the data generated by a global network of gravitational-wave observatories has been at the center of research for almost two decades. As data from these detectors is starting to be analyzed, a renewed interest in this problem has been sparked. A Bayesian approach to the problem of coherently searching for gravitational wave bursts with a network of ground-based interferometers is here presented. We demonstrate how to systematically incorporate prior information on the burst signal and its source into the analysis. This information may range from the very minimal, such as best-guess durations, bandwidths, or polarization content, to complete prior knowledge of the signal waveforms and the distribution of sources through spacetime. We show that this comprehensive Bayesian formulation contains several previously proposed detection statistics as special limiting cases, and demonstrate that it outperforms them.Comment: 18 pages, 3 figures, revisions based on referee comment

Plans for the LIGO-TAMA Joint Search for Gravitational Wave Bursts

We describe the plans for a joint search for unmodelled gravitational wave bursts being carried out by the LIGO and TAMA collaborations using data collected during February-April 2003. We take a conservative approach to detection, requiring candidate gravitational wave bursts to be seen in coincidence by all four interferometers. We focus on some of the complications of performing this coincidence analysis, in particular the effects of the different alignments and noise spectra of the interferometers.Comment: Proceedings of the 8th Gravitational Wave Data Analysis Workshop, Milwaukee, WI, USA. 10 pages, 3 figures, documentclass ``iopart'

Task-Specific Phosphonium Iongels by Fast UV-Photopolymerization for Solid-State Sodium Metal Batteries

Sodium metal batteries are an emerging technology that shows promise in terms of materials availability with respect to lithium batteries. Solid electrolytes are needed to tackle the safety issues related to sodium metal. In this work, a simple method to prepare a mechanically robust and efficient soft solid electrolyte for sodium batteries is demonstrated. A task-specific iongel electrolyte was prepared by combining in a simple process the excellent performance of sodium metal electrodes of an ionic liquid electrolyte and the mechanical properties of polymers. The iongel was synthesized by fast (<1 min) UV photopolymerization of poly(ethylene glycol) diacrylate (PEGDA) in the presence of a saturated 42%mol solution of sodium bis(fluorosulfonyl)imide (NaFSI) in trimethyl iso-butyl phosphonium bis(fluorosulfonyl)imide (P111i4FSI). The resulting soft solid electrolytes showed high ionic conductivity at room temperature (≥10−3 S cm−1) and tunable storage modulus (104–107 Pa). Iongel with the best ionic conductivity and good mechanical properties (Iongel10) showed excellent battery performance: Na/iongel/NaFePO4 full cells delivered a high specific capacity of 140 mAh g−1 at 0.1 C and 120 mAh g−1 at 1 C with good capacity retention after 30 cycles.L.P. has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska–Curie grant agreement No. 797295. P.S. has been funded by the SNSF (Swiss National Science Foundation) under project number P2FRP2_191846. D.M., M.G., and A.F.A. acknowledge the funding by the Basque Government through Elkartek KK-2020/00078 and Agencia Estatal de Investigación (PLEC2021-007929). D.M. and M.G. also acknowledge the funding by Agencia Estatal de Investigación (PID2020-119026GB-100 and PID2019-107468RB-C22), respectively. A.P.S. acknowledges financial support for dielectric measurements and data discussions by NSF (award CHE-2102425)

Hawaii coastal seawater CO2 network: A statistical evaluation of a decade of observations on tropical coral reefs.

© The Author(s), 2019. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Terlouw, G. J., Knor, L. A. C. M., De Carlo, E. H., Drupp, P. S., Mackenzie, F. T., Li, Y. H., Sutton, A. J., Plueddemann, A. J., & Sabine, C. L. Hawaii coastal seawater CO2 network: A statistical evaluation of a decade of observations on tropical coral reefs. Frontiers in Marine Science, 6, (2019):226, doi:10.3389/fmars.2019.00226.A statistical evaluation of nearly 10 years of high-resolution surface seawater carbon dioxide partial pressure (pCO2) time-series data collected from coastal moorings around O’ahu, Hawai’i suggest that these coral reef ecosystems were largely a net source of CO2 to the atmosphere between 2008 and 2016. The largest air-sea flux (1.24 ± 0.33 mol m−2 yr−1) and the largest variability in seawater pCO2 (950 μatm overall range or 8x the open ocean range) were observed at the CRIMP-2 site, near a shallow barrier coral reef system in Kaneohe Bay O’ahu. Two south shore sites, Kilo Nalu and Ala Wai, also exhibited about twice the surface water pCO2 variability of the open ocean, but had net fluxes that were much closer to the open ocean than the strongly calcifying system at CRIMP-2. All mooring sites showed the opposite seasonal cycle from the atmosphere, with the highest values in the summer and lower values in the winter. Average coastal diurnal variabilities ranged from a high of 192 μatm/day to a low of 32 μatm/day at the CRIMP-2 and Kilo Nalu sites, respectively, which is one to two orders of magnitude greater than observed at the open ocean site. Here we examine the modes and drivers of variability at the different coastal sites. Although daily to seasonal variations in pCO2 and air-sea CO2 fluxes are strongly affected by localized processes, basin-scale climate oscillations also affect the variability on interannual time scales.We acknowledge with gratitude the financial support of our research provided in part by a grant/cooperative agreement from the National Oceanic and Atmospheric Administration, Project R/IR-27, which is sponsored by the University of Hawaii Sea Grant College Program, SOEST, under Institutional Grant No. NA14OAR4170071 from NOAA Office of Sea Grant, Department of Commerce. Additional support was granted by the NOAA/Ocean Acidification Program (to EDC and AS) and the NOAA/Climate Program Office (AP), and the NOAA Ocean Observing and Monitoring Division, Climate Program Office (FundRef number 100007298) through agreement NA14OAR4320158 of the NOAA Cooperative Institute for the North Atlantic Region (AP). The views expressed herein are those of the author(s) and do not necessarily reflect the views of NOAA or any of its subagencies. This is SOEST contribution number 10684, PMEL contribution number 4845, and Hawai’i Sea Grant contribution UNIHI-SEAGRANT-JC-15-30