Mega-Reward: Achieving Human-Level Play without Extrinsic Rewards

Intrinsic rewards were introduced to simulate how human intelligence works; they are usually evaluated by intrinsically-motivated play, i.e., playing games without extrinsic rewards but evaluated with extrinsic rewards. However, none of the existing intrinsic reward approaches can achieve human-level performance under this very challenging setting of intrinsically-motivated play. In this work, we propose a novel megalomania-driven intrinsic reward (called mega-reward), which, to our knowledge, is the first approach that achieves human-level performance in intrinsically-motivated play. Intuitively, mega-reward comes from the observation that infants' intelligence develops when they try to gain more control on entities in an environment; therefore, mega-reward aims to maximize the control capabilities of agents on given entities in a given environment. To formalize mega-reward, a relational transition model is proposed to bridge the gaps between direct and latent control. Experimental studies show that mega-reward (i) can greatly outperform all state-of-the-art intrinsic reward approaches, (ii) generally achieves the same level of performance as Ex-PPO and professional human-level scores, and (iii) has also a superior performance when it is incorporated with extrinsic rewards

Wave-packet treatment of neutrino oscillations and its implications on determining the neutrino mass hierarchy

We derive the neutrino flavor transition probabilities with the neutrino treated as a wave packet. The decoherence and dispersion effects from the wave-packet treatment show up as damping and phase-shifting of the plane-wave neutrino oscillation patterns. If the energy uncertainty in the initial neutrino wave packet is larger than around 0.01 of the neutrino energy, the decoherence and dispersion effects would degrade the sensitivity of reactor neutrino experiments to mass hierarchy measurement to lower than 3 $\sigma$ confidence level

Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence

Learning agents that are not only capable of taking tests, but also innovating is becoming a hot topic in AI. One of the most promising paths towards this vision is multi-agent learning, where agents act as the environment for each other, and improving each agent means proposing new problems for others. However, existing evaluation platforms are either not compatible with multi-agent settings, or limited to a specific game. That is, there is not yet a general evaluation platform for research on multi-agent intelligence. To this end, we introduce Arena, a general evaluation platform for multi-agent intelligence with 35 games of diverse logics and representations. Furthermore, multi-agent intelligence is still at the stage where many problems remain unexplored. Therefore, we provide a building toolkit for researchers to easily invent and build novel multi-agent problems from the provided game set based on a GUI-configurable social tree and five basic multi-agent reward schemes. Finally, we provide Python implementations of five state-of-the-art deep multi-agent reinforcement learning baselines. Along with the baseline implementations, we release a set of 100 best agents/teams that we can train with different training schemes for each game, as the base for evaluating agents with population performance. As such, the research community can perform comparisons under a stable and uniform standard. All the implementations and accompanied tutorials have been open-sourced for the community at https://sites.google.com/view/arena-unity/

Systematic Analysis of Security and Vulnerabilities in Miniapps

The past few years have witnessed a boom of miniapps, as lightweight applications, miniapps are of great importance in the mobile internet sector. Consequently, the security of miniapps can directly impact compromising the integrity of sensitive data, posing a potential threat to user privacy. However, after a thorough review of the various research efforts in miniapp security, we found that their actions in researching the safety of miniapp web interfaces are limited. This paper proposes a triad threat model focusing on users, servers and attackers to mitigate the security risk of miniapps. By following the principle of least privilege and the direction of permission consistency, we design a novel analysis framework for the security risk assessment of miniapps by this model. Then, we analyzed the correlation between the security risk assessment and the threat model associated with the miniapp. This analysis led to identifying potential scopes and categorisations with security risks. In the case study, we identify nine major categories of vulnerability issues, such as SQL injection, logical vulnerabilities and cross-site scripting. We also assessed a total of 50,628 security risk hazards and provided specific examples.Comment: 9 page

Variations in morphology and PSII photosynthetic capabilities during the early development of tetraspores of Gracilaria vermiculophylla (Ohmi) Papenfuss (Gracilariales, Rhodophyta)

<p>Abstract</p> <p>Background</p> <p>Red algae are primitive photosynthetic eukaryotes, whose spores are ideal subjects for studies of photosynthesis and development. Although the development of red alga spores has received considerable research attention, few studies have focused on the detailed morphological and photosynthetic changes that occur during the early development of tetraspores of <it>Gracilaria vermiculophylla </it>(Ohmi) Papenfuss (Gracilariales, Rhodophyta). Herein, we documented these changes in this species of red algae.</p> <p>Results</p> <p>In the tetraspores, we observed two types of division, cruciate and zonate, and both could develop into multicellular bodies (disks). During the first 84 hours, tetraspores divided several times, but the diameter of the disks changed very little; thereafter, the diameter increased significantly. Scanning electron microscopy observations and analysis of histological sections revealed that the natural shape of the disk remains tapered over time, and the erect frond grows from the central protrusion of the disk. Cultivation of tissue from excised disks demonstrated that the central protrusion of the disk is essential for initiation of the erect frond. Photosynthetic (i.e., PSII) activities were measured using chlorophyll fluorescence analysis. The results indicated that freshly released tetraspores retained limited PSII photosynthetic capabilities; when the tetraspores attached to a substrate, those capabilities increased significantly. In the disk, the PSII activity of both marginal and central cells was similar, although some degree of morphological polarity was present; the PSII photosynthetic capabilities in young germling exhibited an apico-basal gradient.</p> <p>Conclusions</p> <p>Attachment of tetraspores to a substrate significantly enhanced their PSII photosynthetic capabilities, and triggered further development. The central protrusion of the disk is the growth point, may have transfer of nutritive material with the marginal cells. Within the young germling, the hetero-distribution of PSII photosynthetic capabilities might be due to the differences in cell functions.</p

Quantum mechanical rippling of a MoS2 monolayer controlled by interlayer bilayer coupling

Physical Review Letters114606550

Non-Hermitian topological whispering gallery

In 1878, Lord Rayleigh observed the highly celebrated phenomenon of sound waves that creep around the curved gallery of St Paul's Cathedral in London1,2. These whispering-gallery waves scatter efficiently with little diffraction around an enclosure and have since found applications in ultrasonic fatigue and crack testing, and in the optical sensing of nanoparticles or molecules using silica microscale toroids. Recently, intense research efforts have focused on exploring non-Hermitian systems with cleverly matched gain and loss, facilitating unidirectional invisibility and exotic characteristics of exceptional points3,4. Likewise, the surge in physics using topological insulators comprising non-trivial symmetry-protected phases has laid the groundwork in reshaping highly unconventional avenues for robust and reflection-free guiding and steering of both sound and light5,6. Here we construct a topological gallery insulator using sonic crystals made of thermoplastic rods that are decorated with carbon nanotube films, which act as a sonic gain medium by virtue of electro-thermoacoustic coupling. By engineering specific non-Hermiticity textures to the activated rods, we are able to break the chiral symmetry of the whispering-gallery modes, which enables the out-coupling of topological "audio lasing" modes with the desired handedness. We foresee that these findings will stimulate progress in non-destructive testing and acoustic sensing.This work was supported by the National Basic Research Program of China (2017YFA0303702), NSFC (12074183, 11922407, 11904035, 11834008, 11874215 and 12104226) and the Fundamental Research Funds for the Central Universities (020414380181). Z.Z. acknowledges the support from the China National Postdoctoral Program for Innovative Talents (BX20200165) and the China Postdoctoral Science Foundation (2020M681541). L.Z. acknowledges support from the CONEX-Plus programme funded by Universidad Carlos III de Madrid and the European Union's Horizon 2020 research and innovation programme under Marie Sklodowska-Curie grant agreement 801538. J.C. acknowledges support from the European Research Council (ERC) through the Starting Grant 714577 PHONOMETA and from the MINECO through a Ramón y Cajal grant (grant number RYC-2015-17156)