Warm-Start AlphaZero Self-Play Search Enhancements

Recently, AlphaZero has achieved landmark results in deep reinforcement learning, by providing a single self-play architecture that learned three different games at super human level. AlphaZero is a large and complicated system with many parameters, and success requires much compute power and fine-tuning. Reproducing results in other games is a challenge, and many researchers are looking for ways to improve results while reducing computational demands. AlphaZero's design is purely based on self-play and makes no use of labeled expert data ordomain specific enhancements; it is designed to learn from scratch. We propose a novel approach to deal with this cold-start problem by employing simple search enhancements at the beginning phase of self-play training, namely Rollout, Rapid Action Value Estimate (RAVE) and dynamically weighted combinations of these with the neural network, and Rolling Horizon Evolutionary Algorithms (RHEA). Our experiments indicate that most of these enhancements improve the performance of their baseline player in three different (small) board games, with especially RAVE based variants playing strongly

Circulating microRNAs Reveal Time Course of Organ Injury in a Porcine Model of Acetaminophen-Induced Acute Liver Failure

Acute liver failure is a rare but catastrophic condition which can progress rapidly to multi-organ failure. Studies investigating the onset of individual organ injury such as the liver, kidneys and brain during the evolution of acute liver failure, are lacking. MicroRNAs are short, non-coding strands of RNA that are released into the circulation following tissue injury. In this study, we have characterised the release of both global microRNA and specific microRNA species into the plasma using a porcine model of acetaminophen-induced acute liver failure. Pigs were induced to acute liver failure with oral acetaminophen over 19h±2h and death occurred 13h±3h thereafter. Global microRNA concentrations increased 4h prior to acute liver failure in plasma (P<0.0001) but not in isolated exosomes, and were associated with increasing plasma levels of the damage-associated molecular pattern molecule, genomic DNA (P<0.0001). MiR122 increased around the time of onset of acute liver failure (P<0.0001) and was associated with increasing international normalised ratio (P<0.0001). MiR192 increased 8h after acute liver failure (P<0.0001) and was associated with increasing creatinine (P<0.0001). The increase in miR124-1 occurred concurrent with the pre-terminal increase in intracranial pressure (P<0.0001) and was associated with decreasing cerebral perfusion pressure (P<0.002)

Correlations among Brain Gray Matter Volumes, Age, Gender, and Hemisphere in Healthy Individuals

To determine the relationship between age and gray matter structure and how interactions between gender and hemisphere impact this relationship, we examined correlations between global or regional gray matter volume and age, including interactions of gender and hemisphere, using a general linear model with voxel-based and region-of-interest analyses. Brain magnetic resonance images were collected from 1460 healthy individuals aged 20–69 years; the images were linearly normalized and segmented and restored to native space for analysis of global gray matter volume. Linearly normalized images were then non-linearly normalized and smoothed for analysis of regional gray matter volume. Analysis of global gray matter volume revealed a significant negative correlation between gray matter ratio (gray matter volume divided by intracranial volume) and age in both genders, and a significant interaction effect of age × gender on the gray matter ratio. In analyzing regional gray matter volume, the gray matter volume of all regions showed significant main effects of age, and most regions, with the exception of several including the inferior parietal lobule, showed a significant age × gender interaction. Additionally, the inferior temporal gyrus showed a significant age × gender × hemisphere interaction. No regional volumes showed significant age × hemisphere interactions. Our study may contribute to clarifying the mechanism(s) of normal brain aging in each brain region

Jet energy measurement with the ATLAS detector in proton-proton collisions at root s=7 TeV

The jet energy scale and its systematic uncertainty are determined for jets measured with the ATLAS detector at the LHC in proton-proton collision data at a centre-of-mass energy of √s = 7TeV corresponding to an integrated luminosity of 38 pb-1. Jets are reconstructed with the anti-kt algorithm with distance parameters R=0. 4 or R=0. 6. Jet energy and angle corrections are determined from Monte Carlo simulations to calibrate jets with transverse momenta pT≥20 GeV and pseudorapidities {pipe}η{pipe}<4. 5. The jet energy systematic uncertainty is estimated using the single isolated hadron response measured in situ and in test-beams, exploiting the transverse momentum balance between central and forward jets in events with dijet topologies and studying systematic variations in Monte Carlo simulations. The jet energy uncertainty is less than 2. 5 % in the central calorimeter region ({pipe}η{pipe}<0. 8) for jets with 60≤pT<800 GeV, and is maximally 14 % for pT<30 GeV in the most forward region 3. 2≤{pipe}η{pipe}<4. 5. The jet energy is validated for jet transverse momenta up to 1 TeV to the level of a few percent using several in situ techniques by comparing a well-known reference such as the recoiling photon pT, the sum of the transverse momenta of tracks associated to the jet, or a system of low-pT jets recoiling against a high-pT jet. More sophisticated jet calibration schemes are presented based on calorimeter cell energy density weighting or hadronic properties of jets, aiming for an improved jet energy resolution and a reduced flavour dependence of the jet response. The systematic uncertainty of the jet energy determined from a combination of in situ techniques is consistent with the one derived from single hadron response measurements over a wide kinematic range. The nominal corrections and uncertainties are derived for isolated jets in an inclusive sample of high-pT jets. Special cases such as event topologies with close-by jets, or selections of samples with an enhanced content of jets originating from light quarks, heavy quarks or gluons are also discussed and the corresponding uncertainties are determined. © 2013 CERN for the benefit of the ATLAS collaboration

Measurement of the inclusive and dijet cross-sections of b-jets in pp collisions at sqrt(s) = 7 TeV with the ATLAS detector

The inclusive and dijet production cross-sections have been measured for jets containing b-hadrons (b-jets) in proton-proton collisions at a centre-of-mass energy of sqrt(s) = 7 TeV, using the ATLAS detector at the LHC. The measurements use data corresponding to an integrated luminosity of 34 pb^-1. The b-jets are identified using either a lifetime-based method, where secondary decay vertices of b-hadrons in jets are reconstructed using information from the tracking detectors, or a muon-based method where the presence of a muon is used to identify semileptonic decays of b-hadrons inside jets. The inclusive b-jet cross-section is measured as a function of transverse momentum in the range 20 < pT < 400 GeV and rapidity in the range |y| < 2.1. The bbbar-dijet cross-section is measured as a function of the dijet invariant mass in the range 110 < m_jj < 760 GeV, the azimuthal angle difference between the two jets and the angular variable chi in two dijet mass regions. The results are compared with next-to-leading-order QCD predictions. Good agreement is observed between the measured cross-sections and the predictions obtained using POWHEG + Pythia. MC@NLO + Herwig shows good agreement with the measured bbbar-dijet cross-section. However, it does not reproduce the measured inclusive cross-section well, particularly for central b-jets with large transverse momenta.Comment: 10 pages plus author list (21 pages total), 8 figures, 1 table, final version published in European Physical Journal

Cortical Plasticity Induced by Short-Term Multimodal Musical Rhythm Training

Performing music is a multimodal experience involving the visual, auditory, and somatosensory modalities as well as the motor system. Therefore, musical training is an excellent model to study multimodal brain plasticity. Indeed, we have previously shown that short-term piano practice increase the magnetoencephalographic (MEG) response to melodic material in novice players. Here we investigate the impact of piano training using a rhythmic-focused exercise on responses to rhythmic musical material. Musical training with non musicians was conducted over a period of two weeks. One group (sensorimotor-auditory, SA) learned to play a piano sequence with a distinct musical rhythm, another group (auditory, A) listened to, and evaluated the rhythmic accuracy of the performances of the SA-group. Training-induced cortical plasticity was evaluated using MEG, comparing the mismatch negativity (MMN) in response to occasional rhythmic deviants in a repeating rhythm pattern before and after training. The SA-group showed a significantly greater enlargement of MMN and P2 to deviants after training compared to the A- group. The training-induced increase of the rhythm MMN was bilaterally expressed in contrast to our previous finding where the MMN for deviants in the pitch domain showed a larger right than left increase. The results indicate that when auditory experience is strictly controlled during training, involvement of the sensorimotor system and perhaps increased attentional recources that are needed in producing rhythms lead to more robust plastic changes in the auditory cortex compared to when rhythms are simply attended to in the auditory domain in the absence of motor production

Comparative Genomic Analyses of Copper Transporters and Cuproproteomes Reveal Evolutionary Dynamics of Copper Utilization and Its Link to Oxygen

Copper is an essential trace element in many organisms and is utilized in all domains of life. It is often used as a cofactor of redox proteins, but is also a toxic metal ion. Intracellular copper must be carefully handled to prevent the formation of reactive oxygen species which pose a threat to DNA, lipids, and proteins. In this work, we examined patterns of copper utilization in prokaryotes by analyzing the occurrence of copper transporters and copper-containing proteins. Many organisms, including those that lack copper-dependent proteins, had copper exporters, likely to protect against copper ions that inadvertently enter the cell. We found that copper use is widespread among prokaryotes, but also identified several phyla that lack cuproproteins. This is in contrast to the use of other trace elements, such as selenium, which shows more scattered and reduced usage, yet larger selenoproteomes. Copper transporters had different patterns of occurrence than cuproproteins, suggesting that the pathways of copper utilization and copper detoxification are independent of each other. We present evidence that organisms living in oxygen-rich environments utilize copper, whereas the majority of anaerobic organisms do not. In addition, among copper users, cuproproteomes of aerobic organisms were larger than those of anaerobic organisms. Prokaryotic cuproproteomes were small and dominated by a single protein, cytochrome c oxidase. The data are consistent with the idea that proteins evolved to utilize copper following the oxygenation of the Earth