Think Outside the Color Box: Probabilistic Target Selection and the SDSS-XDQSO Quasar Targeting Catalog

We present the SDSS-XDQSO quasar targeting catalog for efficient flux-based quasar target selection down to the faint limit of the Sloan Digital Sky Survey (SDSS) catalog, even at medium redshifts (2.5 <~ z <~ 3) where the stellar contamination is significant. We build models of the distributions of stars and quasars in flux space down to the flux limit by applying the extreme-deconvolution method to estimate the underlying density. We convolve this density with the flux uncertainties when evaluating the probability that an object is a quasar. This approach results in a targeting algorithm that is more principled, more efficient, and faster than other similar methods. We apply the algorithm to derive low-redshift (z < 2.2), medium-redshift (2.2 <= z 3.5) quasar probabilities for all 160,904,060 point sources with dereddened i-band magnitude between 17.75 and 22.45 mag in the 14,555 deg^2 of imaging from SDSS Data Release 8. The catalog can be used to define a uniformly selected and efficient low- or medium-redshift quasar survey, such as that needed for the SDSS-III's Baryon Oscillation Spectroscopic Survey project. We show that the XDQSO technique performs as well as the current best photometric quasar-selection technique at low redshift, and outperforms all other flux-based methods for selecting the medium-redshift quasars of our primary interest. We make code to reproduce the XDQSO quasar target selection publicly available

Accounting for Calibration Uncertainties in X-ray Analysis: Effective Areas in Spectral Fitting

While considerable advance has been made to account for statistical uncertainties in astronomical analyses, systematic instrumental uncertainties have been generally ignored. This can be crucial to a proper interpretation of analysis results because instrumental calibration uncertainty is a form of systematic uncertainty. Ignoring it can underestimate error bars and introduce bias into the fitted values of model parameters. Accounting for such uncertainties currently requires extensive case-specific simulations if using existing analysis packages. Here we present general statistical methods that incorporate calibration uncertainties into spectral analysis of high-energy data. We first present a method based on multiple imputation that can be applied with any fitting method, but is necessarily approximate. We then describe a more exact Bayesian approach that works in conjunction with a Markov chain Monte Carlo based fitting. We explore methods for improving computational efficiency, and in particular detail a method of summarizing calibration uncertainties with a principal component analysis of samples of plausible calibration files. This method is implemented using recently codified Chandra effective area uncertainties for low-resolution spectral analysis and is verified using both simulated and actual Chandra data. Our procedure for incorporating effective area uncertainty is easily generalized to other types of calibration uncertainties.Comment: 61 pages double spaced, 8 figures, accepted for publication in Ap

Bayesian astrostatistics: a backward look to the future

This perspective chapter briefly surveys: (1) past growth in the use of Bayesian methods in astrophysics; (2) current misconceptions about both frequentist and Bayesian statistical inference that hinder wider adoption of Bayesian methods by astronomers; and (3) multilevel (hierarchical) Bayesian modeling as a major future direction for research in Bayesian astrostatistics, exemplified in part by presentations at the first ISI invited session on astrostatistics, commemorated in this volume. It closes with an intentionally provocative recommendation for astronomical survey data reporting, motivated by the multilevel Bayesian perspective on modeling cosmic populations: that astronomers cease producing catalogs of estimated fluxes and other source properties from surveys. Instead, summaries of likelihood functions (or marginal likelihood functions) for source properties should be reported (not posterior probability density functions), including nontrivial summaries (not simply upper limits) for candidate objects that do not pass traditional detection thresholds.Comment: 27 pp, 4 figures. A lightly revised version of a chapter in "Astrostatistical Challenges for the New Astronomy" (Joseph M. Hilbe, ed., Springer, New York, forthcoming in 2012), the inaugural volume for the Springer Series in Astrostatistics. Version 2 has minor clarifications and an additional referenc

Detailed Study of the KL -> 3pi0 Dalitz Plot

Using a sample of 68 million KL -> 3pi0 decays collected in 1996-1999 by the KTeV (E832) experiment at Fermilab, we present a detailed study of the KL -> 3pi0 Dalitz plot density. We report the first observation of interference from KL->pi+pi-pi0 decays in which pi+pi- rescatters to 2pi0 in a final-state interaction. This rescattering effect is described by the Cabibbo-Isidori model, and it depends on the difference in pion scattering lengths between the isospin I=0 and I=2 states, a0-a2. Using the Cabibbo-Isidori model, we present the first measurement of the KL-> 3pi0 quadratic slope parameter that accounts for the rescattering effect.Comment: accepted by Phys. Rev

An objective method to associate local weather extremes with characteristic circulation structures

In this paper we give methods to find characteristic circulation patterns which are connected to local extreme temperature anomalies. Two data reduction techniques are applied: Legendre polynomial fitting and watershedding. For polynomial fitting a clear trend is found with respect to local temperatures. However, the trend is not distinctive enough to give clear answers on the type of circulation patterns belonging to local extremes. The main advantage of watershedding is that the physical properties of the circulation patterns are retained while the dimension of the data is largely reduced. Expert knowledge, however, is needed to model these main features as predictors

Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels

One important assumption underlying common classification models is the stationarity of the data. However, in real-world streaming applications, the data concept indicated by the joint distribution of feature and label is not stationary but drifting over time. Concept drift detection aims to detect such drifts and adapt the model so as to mitigate any deterioration in the model's predictive performance. Unfortunately, most existing concept drift detection methods rely on a strong and over-optimistic condition that the true labels are available immediately for all already classified instances. In this paper, a novel Hierarchical Hypothesis Testing framework with Request-and-Reverify strategy is developed to detect concept drifts by requesting labels only when necessary. Two methods, namely Hierarchical Hypothesis Testing with Classification Uncertainty (HHT-CU) and Hierarchical Hypothesis Testing with Attribute-wise "Goodness-of-fit" (HHT-AG), are proposed respectively under the novel framework. In experiments with benchmark datasets, our methods demonstrate overwhelming advantages over state-of-the-art unsupervised drift detectors. More importantly, our methods even outperform DDM (the widely used supervised drift detector) when we use significantly fewer labels.Comment: Published as a conference paper at IJCAI 201

Hydrocarbon ratios during PEM-WEST A: A model perspective

A useful application of the hydrocarbon measurements collected during the Pacific Exploratory Mission (PEM-West A) is as markers or indices of atmospheric processing. Traditionally, ratios of particular hydrocarbons have been interpreted as photochemical indices, since much of the effect due to atmospheric transport is assumed to cancel by using ratios. However, an ever increasing body of observatonial and theoretical evidence suggests that turbulent mixing associated with atmospheric transport influences certain hydrocarbon ratios significantly. In this study a three-dimensional mesoscale photochemical model is used to study the interaction of photochemistry and atmospheric mixing on select hydrocarbons. In terms of correlations and functional relationships between various alkanes the model results and PEM-West A hydrocarbon observations share many similar characteristics as well as explainable differences. When the three-dimensional model is applied to inert tracers, hydrocarbon ratios and other relationships exactly follow those expected by simple dilution with model-imposed "background air," and the three-dimensional results for reactive hydrocarbons are quite consistent with a combined influence of photochemistry and simple dilution. Analogous to these model results, relationships between various hydrocarbons collected during the PEM-West A experiment appear to be consistent with this simplified picture of photochemistry and dilution affecting individual air masses. When hydrocarbons are chosen that have negligeble contributions to clean background air, unambiguous determinations of the relative contributions to photochemistry and dilution can be estimated from the hydrocarbon samples. Both the three-dimensional model results and the observations imply an average characteristic lifetime for dilution with background air roughly equivalent to the photochemical lifetime of butane for the western Pacific lower troposphere. Moreover, the dominance of OH as the primary photochemical oxidant downwind of anthropogenic source regions can be inferred from correlations between the highly reactive alkane ratios. By incorporating back-trajectory information within the three-dimensional model analysis, a correspondence between time and a particular hydrocarbon or hydrocarbon ratio can be determined, and the influence of atmospheric mixing or photochemistry can be quantified. Results of the three-dimensional model study are compared and applied to the PEM-West A hydrocarbon dataset, yielding a practical methodology for determining average OH concentrations and atmospheric mixing rates from the hydrocarbon measurements. Aircraft data taken below 2 km during wall flights east of Japan imply a diurnal average OH concentration of ∼3 × 106 cm-3. The characteristic time for dilution with background air is estimated to be ∼2.5 days for the two study areas examined in this work. Copyright 1996 by the American Geophysical Union