Unbiased Comparative Evaluation of Ranking Functions

Eliciting relevance judgments for ranking evaluation is labor-intensive and costly, motivating careful selection of which documents to judge. Unlike traditional approaches that make this selection deterministically, probabilistic sampling has shown intriguing promise since it enables the design of estimators that are provably unbiased even when reusing data with missing judgments. In this paper, we first unify and extend these sampling approaches by viewing the evaluation problem as a Monte Carlo estimation task that applies to a large number of common IR metrics. Drawing on the theoretical clarity that this view offers, we tackle three practical evaluation scenarios: comparing two systems, comparing $k$ systems against a baseline, and ranking $k$ systems. For each scenario, we derive an estimator and a variance-optimizing sampling distribution while retaining the strengths of sampling-based evaluation, including unbiasedness, reusability despite missing data, and ease of use in practice. In addition to the theoretical contribution, we empirically evaluate our methods against previously used sampling heuristics and find that they generally cut the number of required relevance judgments at least in half.Comment: Under review; 10 page

Energy-Momentum of a regular MMaS-class black hole

We compute the energy and momentum of a regular black hole of type defined by Mars, Martin-Prats, and Senovilla using the Einstein and Papapetrou definitions for energy-momentum density. Some other definitions of energy-momentum density are shown to give mutually contradictory and less reasonable results. Results support the Cooperstock hypothesis.Comment: 16 pages, 3 figures, LaTex2e; made minor corrections (in content and in references) at the behest of two anonymous referees. Paper to appear in IJMP

Transonic Flow Past a Symmetrical Airfoil at High Angle of Attack

The results of an experimental investigation of shock-induced stall and leading-edge stall on a 64A010 airfoil section are presented. Advanced nonintrusive techniques - laser velocimetry and holographic interferometry - were used in characterizing the inviscid and viscous flow regions. The measurements include Mach contours of the inviscid now regions, and mean velocity, flow direction, and Reynolds shear stress profiles in the separated regions. The experimental observations of this study are relevant to efforts to improve surface-pressure prediction methods for airfoils at or near stall

Farmer Awareness of Herbicide-resistant Weeds

One of the key concerns about managing pests, and specifically weeds, is time requirements. This reflects not only agricultural demographics such as farm size, but also whether individuals have off-farm job responsibilities. The distribution of farm sizes represented in the survey responses approached a normal distribution (Table 1) and is representative of the farm size distribution in Iowa based on recent information from the National Agricultural Statistics Service

Modified TAP equations for the SK spin glass

The stability of the TAP mean field equations is reanalyzed with the conclusion that the exclusive reason for the breakdown at the spin glass instability is an inconsistency for the value of the local susceptibility. A new alternative approach leads to modified equations which are in complete agreement with the original ones above the instability. Essentially altered results below the instability are presented and the consequences for the dynamical mean field equations are discussed.Comment: 7 pages, 2 figures, final revised version to appear in Europhys. Let