1,964 research outputs found

    Bootstrapping Monte Carlo Tree Search with an Imperfect Heuristic

    Full text link
    We consider the problem of using a heuristic policy to improve the value approximation by the Upper Confidence Bound applied in Trees (UCT) algorithm in non-adversarial settings such as planning with large-state space Markov Decision Processes. Current improvements to UCT focus on either changing the action selection formula at the internal nodes or the rollout policy at the leaf nodes of the search tree. In this work, we propose to add an auxiliary arm to each of the internal nodes, and always use the heuristic policy to roll out simulations at the auxiliary arms. The method aims to get fast convergence to optimal values at states where the heuristic policy is optimal, while retaining similar approximation as the original UCT in other states. We show that bootstrapping with the proposed method in the new algorithm, UCT-Aux, performs better compared to the original UCT algorithm and its variants in two benchmark experiment settings. We also examine conditions under which UCT-Aux works well.Comment: 16 pages, accepted for presentation at ECML'1

    DIE MESSUNG DER UNGLEICHMÄSSIGKEIT VON FASERBÄNDERN BEIM SPINNPROZESS

    Get PDF

    Gravitational Waves and Intermediate-mass Black Hole Retention in Globular Clusters

    Get PDF
    The recent discovery of gravitational waves (GWs) has opened new horizons for physics. Current and upcoming missions, such as LIGO, VIRGO, KAGRA, and LISA, promise to shed light on black holes of every size from stellar mass (SBH) sizes up to supermassive black holes. The intermediate-mass black hole (IMBH) family has not been detected beyond any reasonable doubt. Recent analyses suggest observational evidence for the presence of IMBHs in the centers of two Galactic globular clusters (GCs). In this paper, we investigate the possibility that GCs were born with a central IMBH, which undergoes repeated merger events with SBHs in the cluster core. By means of a semi-analytical method, we follow the evolution of the primordial cluster population in the galactic potential and the mergers of the binary IMBH-SBH systems. Our models predict approximate to 1000 IMBHs within 1 kpc from the galactic center and show that the IMBH-SBH merger rate density changes from R approximate to 1000 Gpc(-3) yr(-1) beyond z approximate to 2 to R approximate to 1-10 Gpc(-3) yr(-1) at z approximate to 0. The rates at low redshifts may be significantly higher if young massive star clusters host IMBHs. The merger rates are dominated by IMBHs with masses between 10(3) and 10(4) M-circle dot. Currently, there are no LIGO/VIRGO upper limits for GW sources in this mass range, but our results show that at design sensitivity, these instruments will detect IMBH-SBH mergers in the coming years. LISA and the Einstein Telescope will be best suited to detect these events. The inspirals of IMBH-SBH systems may also generate an unresolved GW background

    Hi-Val: Iterative Learning of Hierarchical Value Functions for Policy Generation

    Get PDF
    Task decomposition is effective in manifold applications where the global complexity of a problem makes planning and decision-making too demanding. This is true, for example, in high-dimensional robotics domains, where (1) unpredictabilities and modeling limitations typically prevent the manual specification of robust behaviors, and (2) learning an action policy is challenging due to the curse of dimensionality. In this work, we borrow the concept of Hierarchical Task Networks (HTNs) to decompose the learning procedure, and we exploit Upper Confidence Tree (UCT) search to introduce HOP, a novel iterative algorithm for hierarchical optimistic planning with learned value functions. To obtain better generalization and generate policies, HOP simultaneously learns and uses action values. These are used to formalize constraints within the search space and to reduce the dimensionality of the problem. We evaluate our algorithm both on a fetching task using a simulated 7-DOF KUKA light weight arm and, on a pick and delivery task with a Pioneer robot

    Feature-Guided Black-Box Safety Testing of Deep Neural Networks

    Full text link
    Despite the improved accuracy of deep neural networks, the discovery of adversarial examples has raised serious safety concerns. Most existing approaches for crafting adversarial examples necessitate some knowledge (architecture, parameters, etc.) of the network at hand. In this paper, we focus on image classifiers and propose a feature-guided black-box approach to test the safety of deep neural networks that requires no such knowledge. Our algorithm employs object detection techniques such as SIFT (Scale Invariant Feature Transform) to extract features from an image. These features are converted into a mutable saliency distribution, where high probability is assigned to pixels that affect the composition of the image with respect to the human visual system. We formulate the crafting of adversarial examples as a two-player turn-based stochastic game, where the first player's objective is to minimise the distance to an adversarial example by manipulating the features, and the second player can be cooperative, adversarial, or random. We show that, theoretically, the two-player game can con- verge to the optimal strategy, and that the optimal strategy represents a globally minimal adversarial image. For Lipschitz networks, we also identify conditions that provide safety guarantees that no adversarial examples exist. Using Monte Carlo tree search we gradually explore the game state space to search for adversarial examples. Our experiments show that, despite the black-box setting, manipulations guided by a perception-based saliency distribution are competitive with state-of-the-art methods that rely on white-box saliency matrices or sophisticated optimization procedures. Finally, we show how our method can be used to evaluate robustness of neural networks in safety-critical applications such as traffic sign recognition in self-driving cars.Comment: 35 pages, 5 tables, 23 figure

    Role of shielding in modelling cryogenic deuterium pellet ablation

    Get PDF
    corecore