258,577 research outputs found
Evaluation Evaluation: a Monte Carlo study
Over the last decade there has been increasing concern
about the biases embodied in traditional evaluation methods for
Natural Language Processing/Learning, particularly methods
borrowed from Information Retrieval. Without knowledge of the
Bias and Prevalence of the contingency being tested, or equivalently
the expectation due to chance, the simple conditional probabilities
Recall, Precision and Accuracy are not meaningful as evaluation
measures, either individually or in combinations such as F-factor.
The existence of bias in NLP measures leads to the âimprovementâ
of systems by increasing their bias, such as the practice of improving
tagging and parsing scores by using most common value (e.g. water
is always a Noun) rather than the attempting to discover the correct
one. The measures Cohen Kappa and Powers Informedness are
discussed as unbiased alternative to Recall and related to the
psychologically significant measure DeltaP.
In this paper we will analyze both biased and unbiased measures
theoretically, characterizing the precise relationship between all
these measures as well as evaluating the evaluation measures
themselves empirically using a Monte Carlo simulation
Charge and spin correlations of a one dimensional electron gas on the continuum
We present a variational Monte Carlo study of a model one dimensional
electron gas on the continuum, with long-range interaction (1/r decay). At low
density the reduced dimensionality brings about pseudonodes of the many-body
wavefunction, yielding non-ergodic behavior of naive Monte Carlo sampling,
which affects the evaluation of pair correlations and the related structure
factors. The problem is however easily solved and we are able to carefully
analyze the structure factors obtained from an optimal trial function, finding
good agreement with the exact predictions for a Luttinger-like hamiltonian with
an interaction similar to the one used in the present study.Comment: 4 pages, 3 figure
Variational Monte Carlo for spin-orbit interacting systems
Recently, a diffusion Monte Carlo algorithm was applied to the study of spin
dependent interactions in condensed matter. Following some of the ideas
presented therein, and applied to a Hamiltonian containing a Rashba-like
interaction, a general variational Monte Carlo approach is here introduced that
treats in an efficient and very accurate way the spin degrees of freedom in
atoms when spin orbit effects are included in the Hamiltonian describing the
electronic structure. We illustrate the algorithm on the evaluation of the
spin-orbit splittings of isolated carbon and lead atoms. In the case of the
carbon atom, we investigate the differences between the inclusion of spin-orbit
in its realistic and effective spherically symmetrized forms. The method
exhibits a very good accuracy in describing the small energy splittings,
opening the way for a systematic quantum Monte Carlo studies of spin-orbit
effects in atomic systems.Comment: 7 pages, 0 figure
Sequential Matching Estimation of Dynamic Causal Models
This paper proposes sequential matching and inverse selection probability weighting to estimate dynamic casual effects. The sequential matching estimators extend simple, matching estimators based on propensity scores for static causal analysis that have been frequently applied in the evaluation literature. A Monte Carlo study shows that the suggested estimators perform well in small and medium seize samples. Based on the application of the sequential matching estimators to an empirical problem - an evaluation study of the Swiss active labour market policies - some implementational issues are discussed and results are provided.Dynamic treatment effects, nonparametric identification, causal effects, sequential randomisation, programme evaluation, panel data
Sequential Quasi-Monte Carlo
We derive and study SQMC (Sequential Quasi-Monte Carlo), a class of
algorithms obtained by introducing QMC point sets in particle filtering. SQMC
is related to, and may be seen as an extension of, the array-RQMC algorithm of
L'Ecuyer et al. (2006). The complexity of SQMC is , where is
the number of simulations at each iteration, and its error rate is smaller than
the Monte Carlo rate . The only requirement to implement SQMC is
the ability to write the simulation of particle given as a
deterministic function of and a fixed number of uniform variates.
We show that SQMC is amenable to the same extensions as standard SMC, such as
forward smoothing, backward smoothing, unbiased likelihood evaluation, and so
on. In particular, SQMC may replace SMC within a PMCMC (particle Markov chain
Monte Carlo) algorithm. We establish several convergence results. We provide
numerical evidence that SQMC may significantly outperform SMC in practical
scenarios.Comment: 55 pages, 10 figures (final version
Correcting the Minimization Bias in Searches for Small Signals
We discuss a method for correcting the bias in the limits for small signals
if those limits were found based on cuts that were chosen by minimizing a
criterion such as sensitivity. Such a bias is commonly present when a
"minimization" and an "evaluation" are done at the same time. We propose to use
a variant of the bootstrap to adjust the limits. A Monte Carlo study shows that
these new limits have correct coverage.Comment: 14 pages, 5 figue
THE PROPERTIES OF SOME GOODNESS-OF-FIT TESTS
The properties of Pearsonâs goodness-of-fit test, as used in density forecast evaluation, income distribution analysis and elsewhere, are analysed. The components-of-chi-squared or âPearson analogâ tests of Anderson (1994) are shown to be less generally applicable than was originally claimed. For the case of equiprobable classes, where the general components tests remain valid, a Monte Carlo study shows that tests directed towards skewness and kurtosis may have low power, due to differences between the class boundaries and the intersection points of the distributions being compared. The power of individual component tests can be increased by the use of nonequiprobable classes.Pearsonâs Goodness-of-fit test ; Component tests ; Distributional assumptions ; Monte Carlo ; Normality ; Nonequiprobable partitions
Quantum Monte Carlo Study of an Interaction-Driven Band Insulator to Metal Transition
We study the transitions from band insulator to metal to Mott insulator in
the ionic Hubbard model on a two dimensional square lattice using determinant
Quantum Monte Carlo. Evaluation of the temperature dependence of the
conductivity demonstrates that the metallic region extends for a finite range
of interaction values. The Mott phase at strong coupling is accompanied by
antiferromagnetic (AF) order. Inclusion of these intersite correlations changes
the phase diagram qualitatively compared to dynamical mean field theory.Comment: 4 pages, 6 figure
- âŠ