Hi-Val: Iterative Learning of Hierarchical Value Functions for Policy Generation

Task decomposition is effective in manifold applications where the global complexity of a problem makes planning and decision-making too demanding. This is true, for example, in high-dimensional robotics domains, where (1) unpredictabilities and modeling limitations typically prevent the manual specification of robust behaviors, and (2) learning an action policy is challenging due to the curse of dimensionality. In this work, we borrow the concept of Hierarchical Task Networks (HTNs) to decompose the learning procedure, and we exploit Upper Confidence Tree (UCT) search to introduce HOP, a novel iterative algorithm for hierarchical optimistic planning with learned value functions. To obtain better generalization and generate policies, HOP simultaneously learns and uses action values. These are used to formalize constraints within the search space and to reduce the dimensionality of the problem. We evaluate our algorithm both on a fetching task using a simulated 7-DOF KUKA light weight arm and, on a pick and delivery task with a Pioneer robot

Validity and practical utility of accelerometry for the measurement of in-hand physical activity in horses

Background: Accelerometers are valid, practical and reliable tools for the measurement of habitual physical activity (PA). Quantification of PA in horses is desirable for use in research and clinical settings. The objective of this study was to evaluate a triaxial accelerometer for objective measurement of PA in the horse by assessment of their practical utility and validity. Horses were recruited to establish both the optimal site of accelerometer attachment and questionnaire designed to explore owner acceptance. Validity and cut-off values were obtained by assessing PA at various gaits. Validation study- 20 horses wore the accelerometer while being filmed for 10 min each of rest, walking and trotting and 5 mins of canter work. Practical utility study- five horses wore accelerometers on polls and withers for 18 h; compliance and relative data losses were quantified. Results: Accelerometry output differed significantly between the four PA levels (P <0•001) for both wither and poll placement. For withers placement, ROC analyses found optimal sensitivity and specificity at a cut-off of <47 counts per minute (cpm) for rest (sensitivity 99.5 %, specificity 100 %), 967–2424 cpm for trotting (sensitivity 96.7 %, specificity 100 %) and ≥2425 cpm for cantering (sensitivity 96.0 %, specificity 97.0 %). Attachment at the poll resulted in optimal sensitivity and specificity at a cut-off of <707 counts per minute (cpm) for rest (sensitivity 97.5 %, specificity 99.6 %), 1546–2609 cpm for trotting (sensitivity 90.33 %, specificity 79.25 %) and ≥2610 cpm for cantering (sensitivity 100 %, specificity 100 %) In terms of practical utility, accelerometry was well tolerated and owner acceptance high. Conclusion: Accelerometry data correlated well with varying levels of in-hand equine activity. The use of accelerometers is a valid method for objective measurement of controlled PA in the horse

A new constant-pressure molecular dynamics method for finite system

In this letter, by writing the volume as a function of coordinates of atoms, we present a new constant-pressure molecular dynamics method with parameters free. This method is specially appropriate for the finite system in which the periodic boundary condition does not exist. Simulations on the carbon nanotube and the Ni nanoparticle clearly demonstrate the validity of the method. By using this method, one can easily obtain the equation of states for the finite system under the external pressure.Comment: RevTex, 5 pages, 3 figures, submitted to Phys. Rev. Let

Variable cavity volume tooling for high-performance resin infusion moulding

This article describes the research carried out by Warwick under the BAE Systems/EPSRC programme ‘Flapless Aerial Vehicles Integrated Interdisciplinary Research – FLAVIIR’. Warwick's aim in FLAVIIR was to develop low-cost innovative tooling technologies to enable the affordable manufacture of complex composite aerospace structures and to help realize the aim of the Grand Challenge of maintenance-free, low-cost unmanned aerial vehicle manufacture. This article focuses on the evaluation of a novel tooling process (variable cavity tooling) to enable the complete infusion of resin throughout non-crimp fabric within a mould cavity under low (0.1 MPa) injection pressure. The contribution of the primary processing parameters to the mechanical properties of a carbon composite component (bulk-head lug section), and the interactions between parameters, was determined. The initial mould gap (di) was identified as having the most significant effect on all measured mechanical properties, but complex interactions between di, n (number of fabric layers), and vc (mould closure rate) were observed. The process capability was low due to the manual processing, but was improved through process optimization, and delivered properties comparable to high-pressure resin transfer moulding

A simple environment-dependent overlap potential and Cauchy violation in solid argon

We develop an analytic and environment-dependent interatomic potential for the overlap repulsion in solid argon, based on an approximate treatment of the non-orthogonal Tight-Binding theory for the closed-shell systems. The present model can well reproduce the observed elastic properties of solid argon including Cauchy violation at high pressures, yet very simple. A useful and novel analysis is given to show how the elastic properties are related to the environment-dependence incorporated into a generic pairwise potential. The present study has a close link to the broad field of computational materials science, in which the inclusion of environment dependence in short-ranged repulsive part of a potential model is sometimes crucial in predicting the elastic properties correctly.Comment: 10 pages, 3 figure

A tight binding model for water

We demonstrate for the first time a tight binding model for water incorporating polarizable anions. A novel aspect is that we adopt a "ground up" approach in that properties of the monomer and dimer only are fitted. Subsequently we make predictions of the structure and properties of hexamer clusters, ice-XI and liquid water. A particular feature, missing in current tight binding and semiempirical hamiltonians, is that we reproduce the almost two-fold increase in molecular dipole moment as clusters are built up towards the limit of bulk liquid. We concentrate on properties of liquid water which are very well rendered in comparison with experiment and published density functional calculations. Finally we comment on the question of the contrasting densities of water and ice which is central to an understanding of the subtleties of the hydrogen bond

Temperature dependence of surface reconstructions of Au on Pd(110)

Surface reconstructions of Au film on Pd(110) substrate are studied using a local Einstein approximation to quasiharmonic theory with the Sutton-Chen interatomic potential. Temperature dependent surface free energies for different coverages and surface structures are calculated. Experimentally observed transformations from $(1\times1)$ to $(1 \times 2)$ and $(1 \times 3)$ structures can be explained in the framework of this model. Also conditions for Stranski-Krastanov growth mode are found to comply with experiments. The domain of validity of the model neglecting mixing entropy is analyzed.Comment: 7 pages, REVTeX two-column format, 3 postscript figures available on request from [email protected] To appear in Phys. Rev. Letter

Inelastic quantum transport: the self-consistent Born approximation and correlated electron-ion dynamics

A dynamical method for inelastic transport simulations in nanostructures is compared with a steady-state method based on non-equilibrium Green's functions. A simplified form of the dynamical method produces, in the steady state in the weak-coupling limit, effective self-energies analogous to those in the Born Approximation due to electron-phonon coupling. The two methods are then compared numerically on a resonant system consisting of a linear trimer weakly embedded between metal electrodes. This system exhibits enhanced heating at high biases and long phonon equilibration times. Despite the differences in their formulation, the static and dynamical methods capture local current-induced heating and inelastic corrections to the current with good agreement over a wide range of conditions, except in the limit of very high vibrational excitations, where differences begin to emerge.Comment: 12 pages, 7 figure

Learning from Monte Carlo Rollouts with Opponent Models for Playing Tron

This paper describes a novel reinforcement learning system for learning to play the game of Tron. The system combines Q-learning, multi-layer perceptrons, vision grids, opponent modelling, and Monte Carlo rollouts in a novel way. By learning an opponent model, Monte Carlo rollouts can be effectively applied to generate state trajectories for all possible actions from which improved action estimates can be computed. This allows to extend experience replay by making it possible to update the state-action values of all actions in a given game state simultaneously. The results show that the use of experience replay that updates the Q-values of all actions simultaneously strongly outperforms the conventional experience replay that only updates the Q-value of the performed action. The results also show that using short or long rollout horizons during training lead to similar good performances against two fixed opponents