258,577 research outputs found

    Evaluation Evaluation: a Monte Carlo study

    Get PDF
    Over the last decade there has been increasing concern about the biases embodied in traditional evaluation methods for Natural Language Processing/Learning, particularly methods borrowed from Information Retrieval. Without knowledge of the Bias and Prevalence of the contingency being tested, or equivalently the expectation due to chance, the simple conditional probabilities Recall, Precision and Accuracy are not meaningful as evaluation measures, either individually or in combinations such as F-factor. The existence of bias in NLP measures leads to the ‘improvement’ of systems by increasing their bias, such as the practice of improving tagging and parsing scores by using most common value (e.g. water is always a Noun) rather than the attempting to discover the correct one. The measures Cohen Kappa and Powers Informedness are discussed as unbiased alternative to Recall and related to the psychologically significant measure DeltaP. In this paper we will analyze both biased and unbiased measures theoretically, characterizing the precise relationship between all these measures as well as evaluating the evaluation measures themselves empirically using a Monte Carlo simulation

    Charge and spin correlations of a one dimensional electron gas on the continuum

    Full text link
    We present a variational Monte Carlo study of a model one dimensional electron gas on the continuum, with long-range interaction (1/r decay). At low density the reduced dimensionality brings about pseudonodes of the many-body wavefunction, yielding non-ergodic behavior of naive Monte Carlo sampling, which affects the evaluation of pair correlations and the related structure factors. The problem is however easily solved and we are able to carefully analyze the structure factors obtained from an optimal trial function, finding good agreement with the exact predictions for a Luttinger-like hamiltonian with an interaction similar to the one used in the present study.Comment: 4 pages, 3 figure

    Variational Monte Carlo for spin-orbit interacting systems

    Full text link
    Recently, a diffusion Monte Carlo algorithm was applied to the study of spin dependent interactions in condensed matter. Following some of the ideas presented therein, and applied to a Hamiltonian containing a Rashba-like interaction, a general variational Monte Carlo approach is here introduced that treats in an efficient and very accurate way the spin degrees of freedom in atoms when spin orbit effects are included in the Hamiltonian describing the electronic structure. We illustrate the algorithm on the evaluation of the spin-orbit splittings of isolated carbon and lead atoms. In the case of the carbon atom, we investigate the differences between the inclusion of spin-orbit in its realistic and effective spherically symmetrized forms. The method exhibits a very good accuracy in describing the small energy splittings, opening the way for a systematic quantum Monte Carlo studies of spin-orbit effects in atomic systems.Comment: 7 pages, 0 figure

    Sequential Matching Estimation of Dynamic Causal Models

    Get PDF
    This paper proposes sequential matching and inverse selection probability weighting to estimate dynamic casual effects. The sequential matching estimators extend simple, matching estimators based on propensity scores for static causal analysis that have been frequently applied in the evaluation literature. A Monte Carlo study shows that the suggested estimators perform well in small and medium seize samples. Based on the application of the sequential matching estimators to an empirical problem - an evaluation study of the Swiss active labour market policies - some implementational issues are discussed and results are provided.Dynamic treatment effects, nonparametric identification, causal effects, sequential randomisation, programme evaluation, panel data

    Sequential Quasi-Monte Carlo

    Full text link
    We derive and study SQMC (Sequential Quasi-Monte Carlo), a class of algorithms obtained by introducing QMC point sets in particle filtering. SQMC is related to, and may be seen as an extension of, the array-RQMC algorithm of L'Ecuyer et al. (2006). The complexity of SQMC is O(Nlog⁡N)O(N \log N), where NN is the number of simulations at each iteration, and its error rate is smaller than the Monte Carlo rate OP(N−1/2)O_P(N^{-1/2}). The only requirement to implement SQMC is the ability to write the simulation of particle xtnx_t^n given xt−1nx_{t-1}^n as a deterministic function of xt−1nx_{t-1}^n and a fixed number of uniform variates. We show that SQMC is amenable to the same extensions as standard SMC, such as forward smoothing, backward smoothing, unbiased likelihood evaluation, and so on. In particular, SQMC may replace SMC within a PMCMC (particle Markov chain Monte Carlo) algorithm. We establish several convergence results. We provide numerical evidence that SQMC may significantly outperform SMC in practical scenarios.Comment: 55 pages, 10 figures (final version

    Correcting the Minimization Bias in Searches for Small Signals

    Full text link
    We discuss a method for correcting the bias in the limits for small signals if those limits were found based on cuts that were chosen by minimizing a criterion such as sensitivity. Such a bias is commonly present when a "minimization" and an "evaluation" are done at the same time. We propose to use a variant of the bootstrap to adjust the limits. A Monte Carlo study shows that these new limits have correct coverage.Comment: 14 pages, 5 figue

    THE PROPERTIES OF SOME GOODNESS-OF-FIT TESTS

    Get PDF
    The properties of Pearson’s goodness-of-fit test, as used in density forecast evaluation, income distribution analysis and elsewhere, are analysed. The components-of-chi-squared or “Pearson analog” tests of Anderson (1994) are shown to be less generally applicable than was originally claimed. For the case of equiprobable classes, where the general components tests remain valid, a Monte Carlo study shows that tests directed towards skewness and kurtosis may have low power, due to differences between the class boundaries and the intersection points of the distributions being compared. The power of individual component tests can be increased by the use of nonequiprobable classes.Pearson’s Goodness-of-fit test ; Component tests ; Distributional assumptions ; Monte Carlo ; Normality ; Nonequiprobable partitions

    Quantum Monte Carlo Study of an Interaction-Driven Band Insulator to Metal Transition

    Full text link
    We study the transitions from band insulator to metal to Mott insulator in the ionic Hubbard model on a two dimensional square lattice using determinant Quantum Monte Carlo. Evaluation of the temperature dependence of the conductivity demonstrates that the metallic region extends for a finite range of interaction values. The Mott phase at strong coupling is accompanied by antiferromagnetic (AF) order. Inclusion of these intersite correlations changes the phase diagram qualitatively compared to dynamical mean field theory.Comment: 4 pages, 6 figure
    • 

    corecore