Thinking Fast and Slow with Deep Learning and Tree Search

Sequential decision making problems, such as structured prediction, robotic control, and game playing, require a combination of planning policies and generalisation of those plans. In this paper, we present Expert Iteration (ExIt), a novel reinforcement learning algorithm which decomposes the problem into separate planning and generalisation tasks. Planning new policies is performed by tree search, while a deep neural network generalises those plans. Subsequently, tree search is improved by using the neural network policy to guide search, increasing the strength of new plans. In contrast, standard deep Reinforcement Learning algorithms rely on a neural network not only to generalise plans, but to discover them too. We show that ExIt outperforms REINFORCE for training a neural network to play the board game Hex, and our final tree search agent, trained tabula rasa, defeats MoHex 1.0, the most recent Olympiad Champion player to be publicly released.Comment: v1 to v2: - Add a value function in MCTS - Some MCTS hyper-parameters changed - Repetition of experiments: improved accuracy and errors shown. (note the reduction in effect size for the tpt/cat experiment) - Results from a longer training run, including changes in expert strength in training - Comparison to MoHex. v3: clarify independence of ExIt and AG0. v4: see appendix

Tritrophic interactions in forests: Direct and indirect interactions between birds, insect herbivores, and oaks

This study examines direct and indirect relationships between three trophic levels to determine effects on plant damage, herbivore abundance and community structure, and bird distribution in forest ecosystems. Exclusion experiments on white oak (Quercus alba) revealed that bird predation effects to not vary spatially despite variation in abundance of both birds and insects. Using a leaf quality manipulation, I demonstrated that bird impacts do not differ with host plant quality. Rather, birds and plant traits had additive effects on herbivore damage. Bottom-up effects of leaf quality were also more important than top-down effects of birds in structuring the insect herbivore community on white oak. Leaf quality influenced the total abundance and richness of herbivores as well as the abundance of different feeding guilds. These effects of leaf quality were strongest at the end of the growing season, when leaf quality is presumably lowest overall. Bottom-up effects may also be modified by the physical environment in which a plant grows. I studied abundance and distribution of a specialist oak herbivore and showed that individuals choosing a host plant may face a trade off between the optimal physical environment and suitable plant traits. Finally, I demonstrated a bottom-up effect of invasive prey on insectivorous birds: outbreaking gypsy moths alter the annual distribution of native cuckoos at a regional scale. This study indicates that complex interactions exist beyond a simple, unidirectional consumption model of plants, herbivores, and avian predators. The indirect positive effect of birds on plants appears robust to variation in the abundance and traits of the three trophic levels, but the mechanism for this effect may vary through time and space. The impact of birds, however, did not vary with plant characteristics. These characteristics, which can depend on environmental context, likely play a larger role in determining the abundance, structure, and impacts of herbivores than do insectivorous bird predators

Fiscal Impact of EU Migrants in Austria, Germany, the Netherlands and the UK

This study was undertaken to estimate some aspects of the net fiscal impact of EU migrants in four EU countries Austria, Germany, the Netherlands and the United Kingdom. The report outlines the role of Fiscal Impact of EU Migrants in Selected Countries migrants from EU countries as participants in the labour market, as taxpayers and as benefit recipients also. The fiscal contribution of EU foreigners has increased substantially in the past several years. Compared to 2009, inn 2013 EU migrants paid 31% more in direct taxes as their wages increased and more EU workers found employment opportunities in Austria, Germany, the Netherlands, and the UK. As migration accelerated, EU foreigners also paid 44% more on indirect taxes, as they spent more onconsumer purchases. EU foreigners in Austria, Germany, the Netherlands and the UK received 35% more benefits than they did in 2009, due to the overall expansion of the welfare state in addition to the inflow of EU migrants

A Case Study Of Determinants Of An Effective Cloud Computing Strategy

The cloud continues to be an area of information systems that is being adopted cautiously by business firms. The authors of this study analyze factors that can determine the effectiveness of a cloud strategy as firms invest in this computing method. The authors examine cloud computing strategy from a detailed case study and statistical interpretation of a sample of projects of firms and organizations. The findings impute that technical factors are driving cloud computing projects more than procedural factors and that projects in the study exhibit less discipline in methodology than might otherwise be helpful in enabling an initial cloud computing strategy. This study contributes a framework for a prudent cloud computing strategy that can help firms as they further invest in this method of technology

Meeting user needs in national healthcare systems: lessons from early adopter community pharmacists using the electronic prescriptions service

BACKGROUND: The Electronic Prescription Service release Two (EPS2) is a new national healthcare information and communication technology in England that aims to deliver effective prescription writing, dispensing and reimbursement service to benefit patients. The aim of the study was to explore initial user experiences of Community Pharmacists (CPs) using EPS2. METHODS: We conducted nonparticipant observations and interviews in eight EPS2 early adopter community pharmacies classified as ‘first-of-type’ in midlands and northern regions in England. We interviewed eight pharmacists and two dispensers in addition to 56 hours recorded nonparticipant observations as field notes. Line-by-line coding and thematic analysis was conducted on the interview transcripts and field notes. RESULTS: CPs faced two types of challenge. The first was to do with missing electronic prescriptions. This was sometimes very disrupting to work practice, but pharmacists considered it a temporary issue resolvable with minor modifications to the system and user familiarity. The second was to do with long term design-specific issues. Pharmacists could only overcome these by using the system in ways not intended by the developers. Some felt that these issues would not exist had ‘real’ users been involved in the initial development. The issues were: 1) printing out electronic prescriptions (tokens) to dispense from for safe dispensing practices and to free up monitors for other uses, 2) logging all dispensing activities with one user’s Smartcard for convenience and use all human resources in the pharmacy, and, 3) problematic interface causing issues with endorsing prescriptions and claiming reimbursements. CONCLUSIONS: We question if these unintended uses and barriers would have occurred had a more rigorous user-centric principles been applied at the earlier stages of design and implementation of EPS. We conclude that, since modification can occur at the evaluation stage, there is still scope for some of these barriers to be corrected to address the needs, and enhance the experiences, of CPs using the service, and make recommendations on how current challenges could be resolved

Rationalising sequence selection by ligand assemblies in the DNA minor groove : the case for thiazotropsin A

DNA-sequence and structure dependence on the formation of minor groove complexes at 5′-XCTAGY-3′ by the short lexitropsin thiazotropsin A are explored based on NMR spectroscopy, isothermal titration calorimetry (ITC), circular dichroism (CD) and qualitative molecular modeling. The structure and solution behaviour of the complexes are similar whether X = A, T, C or G and Z = T, A, I or C, CCTAGI being thermodynamically the most favoured (ΔG = -11.1 ± 0.1 kcal.mol-1). Binding site selectivity observed by NMR for ACTAGT in the presence of TCTAGA when both accessible sequences are concatenated in a 15-mer DNA duplex construct is consistent with thermodynamic parameters (ΙΔGΙACTAGT > ΙΔGΙTCTAGA) measured separately for the binding sites and with predictions from modeling studies. Steric bulk in the minor groove for Y = G causes unfavourable ligand-DNA interactions reflected in lower Gibbs free energy of binding (ΔG = -8.5 ± 0.01 kcal.mol-1). ITC and CD data establish that thiazotropsin A binds the ODNs with binding constants between 106 and 108 M-1 and reveal that binding is driven enthalpically through hydrogen bond formation and van der Waals interactions. The consequences of these findings are considered with respect to ligand self-association and the energetics responsible for driving DNA recognition by small molecule DNA minor groove binder

Approaching the mind of the builder: analysis of the physical, structural and social constraints on the construction of the broch towers of Iron Age Scotland

Following a review of the paradigmatic context of broch towers in 2012, a revised standard model (the RSM) was defined. The then prevailing paradigm supports a view of broch remains as single monuments of highly variable form that continued in use over perhaps a millennium or more, without significant modification of their original tectonics i.e. their people/constructed-space relationships. This thesis challenges the pre-2012 paradigm by testing the hypothesis that brochs were built to the standard canonical form of the RSM and that their apparent diversity results from anthropic and, or natural modification, not design variability. The fieldwork tests could but did not find refutation of these hypotheses in the observable evidence and offered more profound interpretations of several surviving feature-types. The loading on the stone lintels of the entrance passage through the massively built outer wall and the structurally overladen inner wall created a major structural challenge, evoking a complex engineering solution. Its elements were individually noted pre-2012 but the significance of the engineering response to compression management had not been identified. This structural response was necessary for a tall structure with massive loads, and meaningless without one and its elements are therefore, jointly and severally, clear diagnostics of a broch tower. The entrance engineering was probably the inspiration of one individual or of a small group of master mason-types, not vernacular responses, contra the 2012 paradigm. Isolated stacked voids high in the inner wall are relict features indicative of significant modification of the inner wall. Other anomalous features are shown to be relict stacked void fragments. The East/West differences in brochs across Scotland have long been identified and these are generally attributed to their lithologies. Accepting that, this thesis argues that the principal differences are attributable to the social processes that gave rise to centralisation of settlement around, in and over brochs in the east and north, possibly during the first century BC, and the absence of centralisation in the west; perhaps also explaining the differences in the scale and composition of the artefact assemblages between the two zones. The canonical form facilitates calculation of the relative social costs of broch building for hard-rock and sedimentary stone types. This indicates that the costs of building, increase between 16-, and 32- fold over the buildable range of brochs. Constraints of design down-scalability, design weakness in ground loading, and design cost were major constraints on the mind of the broch builders. Canonicity and the limitations of drystone building technologies predicated specific forms of decomposition on the canonical broch, further complicating their autobiographies and their conservation: the main challenge now being that of finding ways to conserve the evidence for a sequence of processes while conserving the products of those processes

Assessment of EGFR/HER2 dimerization by FRET-FLIM utilizing Alexa-conjugated secondary antibodies in relation to targeted therapies in cancers

The expression level of the HER family is unreliable as a predictive marker for targeted therapies in cancer. Thus, there is a need to develop other biomarkers, which can be used to accurately select responsive patients for targeted therapies. The HER dimerization status may be more important than HER receptor expression per se in determining sensitivity or resistance to a given therapeutic agent. The aim of the study is to develop a FRET assay using dye conjugated secondary antibodies to assess HER receptor dimerization. Using primary antibodies from different species in conjunction with Alexa488 and Alexa546 conjugated secondary antibodies, we validated our EGFR/HER2 dimerization assay in three cell lines, EGFR positive A431 cells as well as HER2 positive breast cell lines BT474 and SKBR3 cells. Finally, we applied our assay to assess EGFR/HER2 dimerization in paraffin embedded cell pellets. Our results show promise for the assay to be applied to tumor samples in order to assess the prognostic significance and predictive value of HER receptor dimerization in various cancers

Plasma Equilibrium inside Various Cross-Section Capillary Discharges

Plasma properties inside a hydrogen-filled capillary discharge waveguide were modeled with dissipative magnetohydrodynamic simulations to enable analysis of capillaries of circular and square cross-sections implying that square capillaries can be used to guide circularly-symmetric laser beams. When the quasistationary stage of the discharge is reached, the plasma and temperature in the vicinity of the capillary axis has almost the same profile for both the circular and square capillaries. The effect of cross-section on the electron beam focusing properties were studied using the simulation-derived magnetic field map. Particle tracking simulations showed only slight effects on the electron beam symmetry in the horizontal and diagonal directions for square capillary.Comment: 6 pages, 10 figure

LUVMI: an innovative payload for the sampling of volatiles at the Lunar poles

The ISECG identifies one of the first exploration steps as in situ investigations of the moon or asteroids. Europe is developing payload concepts for drilling and sample analysis, a contribution to a 250kg rover as well as for sample return. To achieve these missions, ESA depends on international partnerships. Such missions will be seldom, expensive and the drill/sample site selected will be based on observations from orbit not calibrated with ground truth data. Many of the international science community’s objectives can be met at lower cost, or the chances of mission success improved and the quality of the science increased by making use of an innovative, low mass, mobile robotic payload following the LEAG recommendations. LUVMI provides a smart, low mass, innovative, modular mobile payload comprising surface and subsurface sensing with an in-situ sampling technology capable of depth-resolved extraction of volatiles, combined with a volatile analyser (mass spectrometer) capable of identifying the chemical composition of the most important volatiles. This will allow LUVMI to: traverse the lunar surface prospecting for volatiles; sample subsurface up to a depth of 10 cm (with a goal of 20 cm); extract water and other loosely bound volatiles; identify the chemical species extracted; access and sample permanently shadowed regions (PSR). The main innovation of LUVMI is to develop an in situ sampling technology capable of depth-resolved extraction of volatiles, and then to package within this tool, the analyser itself, so as to maximise transfer efficiency and minimise sample handling and its attendant mass requirements and risk of sample alteration. By building on national, EC and ESA funded research and developments, this project will develop to TRL6 instruments that together form a smart modular mobile payload that could be flight ready in 2020. The LUVMI sampling instrument will be tested in a highly representative environment including thermal, vacuum and regolith simulant and the integrated payload demonstrated in a representative environment