lpEdit: an editor to facilitate reproducible analysis via literate programming

ArticleCopyright 2013 Adam J Richards et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited .There is evidence to suggest that a surprising proportion of published experiments in science are difficult if not impossible to reproduce. The concepts of data sharing, leaving an audit trail and extensive documentation are fundamental to reproducible research, whether it is in the laboratory or as part of an analysis. In this work, we introduce a tool for documentation that aims to make analyses more reproducible in the general scientific community. The application, lpEdit, is a cross-platform editor, written with PyQt4, that enables a broad range of scientists to carry out the analytic component of their work in a reproducible manner—through the use of literate programming. Literate programming mixes code and prose to produce a final report that reads like an article or book. lpEdit targets researchers getting started with statistics or programming, so the hurdles associated with setting up a proper pipeline are kept to a minimum and the learning burden is reduced through the use of templates and documentation. The documentation for lpEdit is centered around learning by example, and accordingly we use several increasingly involved examples to demonstrate the software’s capabilities. We first consider applications of lpEdit to process analyses mixing R and Python code with the LATEX documentation system. Finally, we illustrate the use of lpEdit to conduct a reproducible functional analysis of high-throughput sequencing data, using the transcriptome of the butterfly species Pieris brassica

Development of a multimodal foveated endomicroscope for the detection of oral cancer

A multimodal endomicroscope was developed for cancer detection that combines hyperspectral and confocal imaging through a single foveated objective and a vibrating optical fiber bundle. Standard clinical examination has a limited ability to identify early stage oral cancer. Optical detection methods are typically restricted by either achievable resolution or a small field-of-view. By combining high resolution and widefield spectral imaging into a single probe, a device was developed that provides spectral and spatial information over a 5 mm field to locate suspicious lesions that can then be inspected in high resolution mode. The device was evaluated on ex vivo biopsies of human oral tumors

A Simple Likelihood Method for Quasar Target Selection

We present a new method for quasar target selection using photometric fluxes and a Bayesian probabilistic approach. For our purposes we target quasars using Sloan Digital Sky Survey (SDSS) photometry to a magnitude limit of g=22. The efficiency and completeness of this technique is measured using the Baryon Oscillation Spectroscopic Survey (BOSS) data, taken in 2010. This technique was used for the uniformly selected (CORE) sample of targets in BOSS year one spectroscopy to be realized in the 9th SDSS data release. When targeting at a density of 40 objects per sq-deg (the BOSS quasar targeting density) the efficiency of this technique in recovering z>2.2 quasars is 40%. The completeness compared to all quasars identified in BOSS data is 65%. This paper also describes possible extensions and improvements for this techniqueComment: Updated to accepted version for publication in the Astrophysical Journal. 10 pages, 10 figures, 3 table

Photometric redshifts and quasar probabilities from a single, data-driven generative model

We describe a technique for simultaneously classifying and estimating the redshift of quasars. It can separate quasars from stars in arbitrary redshift ranges, estimate full posterior distribution functions for the redshift, and naturally incorporate flux uncertainties, missing data, and multi-wavelength photometry. We build models of quasars in flux-redshift space by applying the extreme deconvolution technique to estimate the underlying density. By integrating this density over redshift one can obtain quasar flux-densities in different redshift ranges. This approach allows for efficient, consistent, and fast classification and photometric redshift estimation. This is achieved by combining the speed obtained by choosing simple analytical forms as the basis of our density model with the flexibility of non-parametric models through the use of many simple components with many parameters. We show that this technique is competitive with the best photometric quasar classification techniques---which are limited to fixed, broad redshift ranges and high signal-to-noise ratio data---and with the best photometric redshift techniques when applied to broadband optical data. We demonstrate that the inclusion of UV and NIR data significantly improves photometric quasar--star separation and essentially resolves all of the redshift degeneracies for quasars inherent to the ugriz filter system, even when included data have a low signal-to-noise ratio. For quasars spectroscopically confirmed by the SDSS 84 and 97 percent of the objects with GALEX UV and UKIDSS NIR data have photometric redshifts within 0.1 and 0.3, respectively, of the spectroscopic redshift; this amounts to about a factor of three improvement over ugriz-only photometric redshifts. Our code to calculate quasar probabilities and redshift probability distributions is publicly available

Trauma Exposures, Resilience Factors, and Mental Health Outcomes in Persons Granted Asylum in the U.S. for Claims Related to Domestic Violence and Persecution by Organized Gangs.

Survivors of domestic violence (DV) and of violence perpetrated by organized gangs (GV) face barriers to legal protection under U.S. asylum law. We abstracted data from 132 affidavits based on forensic medical evaluations of asylum seekers granted legal protection in the U.S. on the basis of DV and/or GV. We described claimants’ trauma exposures and resilience factors and used multiple logistic regression to quantify associations with Diagnostic and Statistical Manual-5 (DSM-5) diagnoses and improvement in mental health. People seeking asylum based on DV and/or GV have endured multiple types of trauma with significant impacts on their mental health. New experiences of trauma following migration to the U.S. were common and associated with DSM-5 diagnoses. Conversely, resilience factors were associated with improved mental health. Policies that aim to reduce ongoing trauma in the U.S. and to bolster resilience factors may promote asylee mental health and well-being

Think Outside the Color Box: Probabilistic Target Selection and the SDSS-XDQSO Quasar Targeting Catalog

We present the SDSS-XDQSO quasar targeting catalog for efficient flux-based quasar target selection down to the faint limit of the Sloan Digital Sky Survey (SDSS) catalog, even at medium redshifts (2.5 <~ z <~ 3) where the stellar contamination is significant. We build models of the distributions of stars and quasars in flux space down to the flux limit by applying the extreme-deconvolution method to estimate the underlying density. We convolve this density with the flux uncertainties when evaluating the probability that an object is a quasar. This approach results in a targeting algorithm that is more principled, more efficient, and faster than other similar methods. We apply the algorithm to derive low-redshift (z < 2.2), medium-redshift (2.2 <= z 3.5) quasar probabilities for all 160,904,060 point sources with dereddened i-band magnitude between 17.75 and 22.45 mag in the 14,555 deg^2 of imaging from SDSS Data Release 8. The catalog can be used to define a uniformly selected and efficient low- or medium-redshift quasar survey, such as that needed for the SDSS-III's Baryon Oscillation Spectroscopic Survey project. We show that the XDQSO technique performs as well as the current best photometric quasar-selection technique at low redshift, and outperforms all other flux-based methods for selecting the medium-redshift quasars of our primary interest. We make code to reproduce the XDQSO quasar target selection publicly available

Eight-Dimensional Mid-Infrared/Optical Bayesian Quasar Selection

We explore the multidimensional, multiwavelength selection of quasars from mid-IR (MIR) plus optical data, specifically from Spitzer-IRAC and the Sloan Digital Sky Survey (SDSS). We apply modern statistical techniques to combined Spitzer MIR and SDSS optical data, allowing up to 8-D color selection of quasars. Using a Bayesian selection method, we catalog 5546 quasar candidates to an 8.0 um depth of 56 uJy over an area of ~24 sq. deg; ~70% of these candidates are not identified by applying the same Bayesian algorithm to 4-color SDSS optical data alone. Our selection recovers 97.7% of known type 1 quasars in this area and greatly improves the effectiveness of identifying 3.5<z<5 quasars. Even using only the two shortest wavelength IRAC bandpasses, it is possible to use our Bayesian techniques to select quasars with 97% completeness and as little as 10% contamination. This sample has a photometric redshift accuracy of 93.6% (Delta Z +/-0.3), remaining roughly constant when the two reddest MIR bands are excluded. While our methods are designed to find type 1 (unobscured) quasars, as many as 1200 of the objects are type 2 (obscured) quasar candidates. Coupling deep optical imaging data with deep mid-IR data could enable selection of quasars in significant numbers past the peak of the quasar luminosity function (QLF) to at least z~4. Such a sample would constrain the shape of the QLF and enable quasar clustering studies over the largest range of redshift and luminosity to date, yielding significant gains in our understanding of quasars and the evolution of galaxies.Comment: 49 pages, 14 figures, 7 tables. AJ, accepte

The SAMI Galaxy Survey: Revising the Fraction of Slow Rotators in IFS Galaxy Surveys

The fraction of galaxies supported by internal rotation compared to galaxies stabilized by internal pressure provides a strong constraint on galaxy formation models. In integral field spectroscopy surveys, this fraction is biased because survey instruments typically only trace the inner parts of the most massive galaxies. We present aperture corrections for the two most widely used stellar kinematic quantities $V/\sigma$ and $\lambda_{R}$. Our demonstration involves integral field data from the SAMI Galaxy Survey and the ATLAS$^{\rm{3D}}$ Survey. We find a tight relation for both $V/\sigma$ and $\lambda_{R}$ when measured in different apertures that can be used as a linear transformation as a function of radius, i.e., a first-order aperture correction. We find that $V/\sigma$ and $\lambda_{R}$ radial growth curves are well approximated by second order polynomials. By only fitting the inner profile (0.5$R_{\rm{e}}$), we successfully recover the profile out to one $R_{\rm{e}}$ if a constraint between the linear and quadratic parameter in the fit is applied. However, the aperture corrections for $V/\sigma$ and $\lambda_{R}$ derived by extrapolating the profiles perform as well as applying a first-order correction. With our aperture-corrected $\lambda_{R}$ measurements, we find that the fraction of slow rotating galaxies increases with stellar mass. For galaxies with $\log M_{*}/M_{\odot}>$ 11, the fraction of slow rotators is $35.9\pm4.3$ percent, but is underestimated if galaxies without coverage beyond one $R_{\rm{e}}$ are not included in the sample ($24.2\pm5.3$ percent). With measurements out to the largest aperture radius the slow rotator fraction is similar as compared to using aperture corrected values ($38.3\pm4.4$ percent). Thus, aperture effects can significantly bias stellar kinematic IFS studies, but this bias can now be removed with the method outlined here.Comment: Accepted for Publication in the Monthly Notices of the Royal Astronomical Society. 16 pages and 11 figures. The key figures of the paper are: 1, 4, 9, and 1

Binary Quasars at High Redshift I: 24 New Quasar Pairs at z ~ 3-4

The clustering of quasars on small scales yields fundamental constraints on models of quasar evolution and the buildup of supermassive black holes. This paper describes the first systematic survey to discover high redshift binary quasars. Using color-selection and photometric redshift techniques, we searched 8142 deg^2 of SDSS imaging data for binary quasar candidates, and confirmed them with follow-up spectroscopy. Our sample of 27 high redshift binaries (24 of them new discoveries) at redshifts 2.9 < z < 4.3 with proper transverse separations 10 kpc < R_{\perp} < 650 kpc increases the number of such objects known by an order of magnitude. Eight members of this sample are very close pairs with R_{\perp} 3.5. The completeness and efficiency of our well-defined selection algorithm are quantified using simulated photometry and we find that our sample is ~ 50% complete. Our companion paper uses this knowledge to make the first measurement of the small scale clustering (R < 1 Mpc/h comoving) of high-redshift quasars. High redshift binaries constitute exponentially rare coincidences of two extreme (M >~ 10^9 Msun) supermassive black holes. At z ~ 4 there is about one close binary per 10 Gpc^3, thus these could be the highest sigma peaks, the analogs of superclusters, in the early Universe.Comment: Submitted to Ap

Clustering Analyses of 300,000 Photometrically Classified Quasars--I. Luminosity and Redshift Evolution in Quasar Bias

Using ~300,000 photometrically classified quasars, by far the largest quasar sample ever used for such analyses, we study the redshift and luminosity evolution of quasar clustering on scales of ~50 kpc/h to ~20 Mpc/h from redshifts of z~0.75 to z~2.28. We parameterize our clustering amplitudes using realistic dark matter models, and find that a LCDM power spectrum provides a superb fit to our data with a redshift-averaged quasar bias of b_Q = 2.41+/-0.08 ($P_{<\chi^2}=0.847$) for $\sigma_8=0.9$. This represents a better fit than the best-fit power-law model ($\omega = 0.0493\pm0.0064\theta^ {-0.928\pm0.055}$; $P_{<\chi^2}=0.482$). We find b_Q increases with redshift. This evolution is significant at >99.6% using our data set alone, increasing to >99.9999% if stellar contamination is not explicitly parameterized. We measure the quasar classification efficiency across our full sample as a = 95.6 +/- ^{4.4}_{1.9}%, a star-quasar separation comparable with the star-galaxy separation in many photometric studies of galaxy clustering. We derive the mean mass of the dark matter halos hosting quasars as MDMH=(5.2+/-0.6)x10^{12} M_solar/h. At z~1.9 we find a $1.5\sigma$ deviation from luminosity-independent quasar clustering; this suggests that increasing our sample size by a factor of 1.8 could begin to constrain any luminosity dependence in quasar bias at z~2. Our results agree with recent studies of quasar environments at z < 0.4, which detected little luminosity dependence to quasar clustering on proper scales >50 kpc/h. At z < 1.6, our analysis suggests that b_Q is constant with luminosity to within ~0.6, and that, for g < 21, angular quasar autocorrelation measurements are unlikely to have sufficient statistical power at z < 1.6 to detect any luminosity dependence in quasars' clustering.Comment: 13 pages, 9 figures, 2 tables; uses amulateapj; accepted to Ap