The Potential of Learned Index Structures for Index Compression

Inverted indexes are vital in providing fast key-word-based search. For every term in the document collection, a list of identifiers of documents in which the term appears is stored, along with auxiliary information such as term frequency, and position offsets. While very effective, inverted indexes have large memory requirements for web-sized collections. Recently, the concept of learned index structures was introduced, where machine learned models replace common index structures such as B-tree-indexes, hash-indexes, and bloom-filters. These learned index structures require less memory, and can be computationally much faster than their traditional counterparts. In this paper, we consider whether such models may be applied to conjunctive Boolean querying. First, we investigate how a learned model can replace document postings of an inverted index, and then evaluate the compromises such an approach might have. Second, we evaluate the potential gains that can be achieved in terms of memory requirements. Our work shows that learned models have great potential in inverted indexing, and this direction seems to be a promising area for future research.Comment: Will appear in the proceedings of ADCS'1

Phytoseiid mites associated with spider mites on hops in the Willamette Valley, Oregon

Densities and damage by twospotted spider mites, Tetranychus urticae Koch and levels of phytoseiid mites on hops were assessed in 34 commercial fields and at 11-19 sites of escaped hops in the Willamette valley of western Oregon in 1991-1992. Amblyseius fallacis (Garman), Typhlodromus pyri Scheuten, Amblyseius andersoni Chant and Metaseiulus occidentalis (Nesbitt) were most common. On escaped hops, T. pyri was more common than other phytoseiids. It occurred widely on plants surrounding commercial hops including blackberry and other rosaceous plants and probably is a vagrant on escaped hops. A. fallacis was most common in commercial hops making up 88% of all specimens, followed by many fewer M. occidentalis and T. pyri. Early spring survival of A Jallacis in commercial hops was poor because of certain cultural practices used in the spring. Means to improve biological control of spider mites on hops are discussed including amended methods of hop culture, use of selective pesticides and inoculative releases of predaceous mites.Keywords: Amblyseiusfallacis; Metaseiulus occidentalis; Typhlodromus pyri; Amblyseius andersoni; Tetranychus urtica

Recovering the Primordial Density Fluctuations: A comparison of methods

We present a comparative study of six different methods for reversing the gravitational evolution of a cosmological density field to recover the primordial fluctuations: linear theory, the Gaussianization mapping scheme, two different quasi-linear dynamical schemes based on the Zel'dovich approximation, a Hybrid dynamical-Gaussianization method and the Path Interchange Zel'dovich Approximation (PIZA). The final evolved density field from an N-body simulation constitutes our test case. We use a variety of statistical measures to compare the initial density field recovered from it to the true initial density field, using each of the six different schemes. These include point-by-point comparisons of the density fields in real space, the individual modes in Fourier space, as well as global statistical properties such as the genus, the PDF of the density, and the distribution of peak heights and their shapes. We find linear theory to be the most inaccurate of all the schemes. The Gaussianization scheme is the least accurate after linear theory. The two quasi-linear dynamical schemes are more accurate than Gaussianization, although they break down quite drastically when used outside their range of validity - the quasi-linear regime. The complementary beneficial aspects of the dynamical and the Gaussianization schemes are combined in the Hybrid method. We find this Hybrid scheme to be more accurate and robust than either Gaussianization or the dynamical method alone. The PIZA scheme performs substantially better than the others in all point-by-point comparisons. However, it produces an oversmoothed initial density field, with a smaller number of peaks than expected, but recovers the PDF of the initial density with impressive accuracy on scales as small as 3Mpc/h.Comment: 39 pages, including 13 Figures, submitted to Ap

Generation of Vorticity and Velocity Dispersion by Orbit Crossing

We study the generation of vorticity and velocity dispersion by orbit crossing using cosmological numerical simulations, and calculate the backreaction of these effects on the evolution of large-scale density and velocity divergence power spectra. We use Delaunay tessellations to define the velocity field, showing that the power spectra of velocity divergence and vorticity measured in this way are unbiased and have better noise properties than for standard interpolation methods that deal with mass weighted velocities. We show that high resolution simulations are required to recover the correct large-scale vorticity power spectrum, while poor resolution can spuriously amplify its amplitude by more than one order of magnitude. We measure the scalar and vector modes of the stress tensor induced by orbit crossing using an adaptive technique, showing that its vector modes lead, when input into the vorticity evolution equation, to the same vorticity power spectrum obtained from the Delaunay method. We incorporate orbit crossing corrections to the evolution of large scale density and velocity fields in perturbation theory by using the measured stress tensor modes. We find that at large scales (k~0.1 h/Mpc) vector modes have very little effect in the density power spectrum, while scalar modes (velocity dispersion) can induce percent level corrections at z=0, particularly in the velocity divergence power spectrum. In addition, we show that the velocity power spectrum is smaller than predicted by linear theory until well into the nonlinear regime, with little contribution from virial velocities.Comment: 27 pages, 14 figures. v2: reorganization of the material, new appendix. Accepted by PR

Spatial Correlation Function of X-ray Selected AGN

We present a detailed description of the first direct measurement of the spatial correlation function of X-ray selected AGN. This result is based on an X-ray flux-limited sample of 219 AGN discovered in the contiguous 80.7 deg^2 region of the ROSAT North Ecliptic Pole (NEP) Survey. Clustering is detected at the 4 sigma level at comoving scales in the interval r = 5-60 h^-1 Mpc. Fitting the data with a power law of slope gamma=1.8, we find a correlation length of r_0 = 7.4 (+1.8, -1.9) h^-1 Mpc (Omega_M=0.3, Omega_Lambda=0.7). The median redshift of the AGN contributing to the signal is z_xi=0.22. This clustering amplitude implies that X-ray selected AGN are spatially distributed in a manner similar to that of optically selected AGN. Furthermore, the ROSAT NEP determination establishes the local behavior of AGN clustering, a regime which is poorly sampled in general. Combined with high-redshift measures from optical studies, the ROSAT NEP results argue that the AGN correlation strength essentially does not evolve with redshift, at least out to z~2.2. In the local Universe, X-ray selected AGN appear to be unbiased relative to galaxies and the inferred X-ray bias parameter is near unity, b_X~1. Hence X-ray selected AGN closely trace the underlying mass distribution. The ROSAT NEP AGN catalog, presented here, features complete optical identifications and spectroscopic redshifts. The median redshift, X-ray flux, and X-ray luminosity are z=0.41, f_X=1.1*10^-13 cgs, and L_X=9.2*10^43 h_70^-2 cgs (0.5-2.0 keV), respectively. Unobscured, type 1 AGN are the dominant constituents (90%) of this soft X-ray selected sample of AGN.Comment: 17 pages, 8 figures, accepted for publication in ApJ, a version with high-resolution figures is available at http://www.eso.org/~cmullis/papers/Mullis_et_al_2004b.ps.gz, a machine-readable version of the ROSAT NEP AGN catalog is available at http://www.eso.org/~cmullis/research/nep-catalog.htm