Some Aspects of Measurement Error in Linear Regression of Astronomical Data

I describe a Bayesian method to account for measurement errors in linear regression of astronomical data. The method allows for heteroscedastic and possibly correlated measurement errors, and intrinsic scatter in the regression relationship. The method is based on deriving a likelihood function for the measured data, and I focus on the case when the intrinsic distribution of the independent variables can be approximated using a mixture of Gaussians. I generalize the method to incorporate multiple independent variables, non-detections, and selection effects (e.g., Malmquist bias). A Gibbs sampler is described for simulating random draws from the probability distribution of the parameters, given the observed data. I use simulation to compare the method with other common estimators. The simulations illustrate that the Gaussian mixture model outperforms other common estimators and can effectively give constraints on the regression parameters, even when the measurement errors dominate the observed scatter, source detection fraction is low, or the intrinsic distribution of the independent variables is not a mixture of Gaussians. I conclude by using this method to fit the X-ray spectral slope as a function of Eddington ratio using a sample of 39 z < 0.8 radio-quiet quasars. I confirm the correlation seen by other authors between the radio-quiet quasar X-ray spectral slope and the Eddington ratio, where the X-ray spectral slope softens as the Eddington ratio increases.Comment: 39 pages, 11 figures, 1 table, accepted by ApJ. IDL routines (linmix_err.pro) for performing the Markov Chain Monte Carlo are available at the IDL astronomy user's library, http://idlastro.gsfc.nasa.gov/homepage.htm

A Comparative Study of Two Real Root Isolation Methods

Recent progress in polynomial elimination has rendered the computation of the real roots of ill-conditioned polynomials of high degree (over 1000) with huge coefficients (several thousand digits) a critical operation in computer algebra. To rise to the occasion, the only method-candidate that has been considered by various authors for modification and improvement has been the Collins-Akritas bisection method [1], which is a based on a variation of Vincent’s theorem [2]. The most recent example is the paper by Rouillier and Zimmermann [3], where the authors present&nbsp;“... a new algorithm, which is optimal in terms of memory usage and as fast as both Collins and Akritas’ algorithm and Krandick variant ...” [3] In this paper we compare our own continued fractions method CF [4] (which is directly based on Vincent’s theorem) with the best bisection method REL described in [3]. Experimentation with the data presented in [3] showed that, with respect to time, our continued fractions method CF is by far superior to REL, whereas the two are about equal with respect to space

Linear Regression for Astronomical Data with Measurement Errors and Intrinsic Scatter

Two new methods are proposed for linear regression analysis for data with measurement errors. Both methods are designed to accommodate intrinsic scatter in addition to measurement errors. The first (BCES) is a direct extension of the ordinary least squares (OLS) estimator to allow for measurement errors. It is quite general, allowing a) for measurement errors on both variables, b) the measurement errors for the two variables to be dependent, c) the magnitudes of the measurement errors to depend on the measurements, and d) other `symmetric' lines such as the bisector and the orthogonal regression can be constructed. The second method is a weighted least squares (WLS) estimator, which applies only in the case where the `independent' variable is measured without error and the magnitudes of the measurement errors on the 'dependent' variable are independent from the measurements. Several applications are made to extragalactic astronomy: The BCES method, when applied to data describing the color-luminosity relations for field galaxies, yields significantly different slopes than OLS and other estimators used in the literature. Simulations with artificial data sets are used to evaluate the small sample performance of the estimators. Unsurprisingly, the least-biased results are obtained when color is treated as the dependent variable. The Tully-Fisher relation is another example where the BCES method should be used because errors in luminosity and velocity are correlated due to inclination corrections. We also find, via simulations, that the WLS method is by far the best method for the Tolman surface-brightness test, producing the smallest variance in slope by an order of magnitude. Moreover, with WLS it is not necessary to ``reduce'' galaxies to a fiducial surface-brightness, since this model incorporates intrinsic scatter.Comment: 23 pages LaTeX, 1 postscript table, and style files (uuencoded, gzipp'ed tar file). To appear in the Astrophysical Journal, October 10, 1996 issue, Vol. 47

The SVD-Fundamental Theorem of Linear Algebra

Given an m × n matrix A, with m ≥ n, the four subspaces associated&nbsp;with it are shown in Fig. 1 (see [1]). Fig. 1. The row spaces and the nullspaces of A and AT; a1 through an and&nbsp;h1 through hm are abbreviations of the alignerframe and hangerframe vectors&nbsp;respectively (see [2]). The Fundamental Theorem of Linear Algebra tells us that N(A) is the orthogonal complement of R(AT). These four subspaces tell the whole story of the&nbsp;Linear System Ax = y.&nbsp; So, for example, the absence of N(AT) indicates that a&nbsp;solution always exists, whereas the absence of N(A) indicates that this solution&nbsp;is unique. Given the importance of these subspaces, computing bases for them&nbsp;is the gist of Linear Algebra. In “Classical” Linear Algebra, bases for these&nbsp;subspaces are computed using Gaussian Elimination; they are orthonormalized&nbsp;with the help of the Gram-Schmidt method. Continuing our previous work [3]&nbsp;and following Uhl’s excellent approach [2] we use SVD analysis to compute&nbsp;orthonormal bases for the four subspaces associated with A, and give a 3D&nbsp;explanation. We then state and prove what we call the “SVD-Fundamental&nbsp;Theorem” of Linear Algebra, and apply it in solving systems of linear equations

The Ks-band Tully-Fisher Relation - A Determination of the Hubble Parameter from 218 ScI Galaxies and 16 Galaxy Clusters

The value of the Hubble Parameter (H0) is determined using the morphologically type dependent Ks-band Tully-Fisher Relation (K-TFR). The slope and zero point are determined using 36 calibrator galaxies with ScI morphology. Calibration distances are adopted from direct Cepheid distances, and group or companion distances derived with the Surface Brightness Fluctuation Method or Type Ia Supernova. Distances are determined to 16 galaxy clusters and 218 ScI galaxies with minimum distances of 40.0 Mpc. From the 16 galaxy clusters a weighted mean Hubble Parameter of H0=84.2 +/-6 km s-1 Mpc-1 is found. From the 218 ScI galaxies a Hubble Parameter of H0=83.4 +/-8 km s-1 Mpc-1 is found. When the zero point of the K-TFR is corrected to account for recent results that find a Large Magellanic Cloud distance modulus of 18.39 +/-0.05 a Hubble Parameter of 88.0 +/-6 km s-1 Mpc-1 is found. A comparison with the results of the Hubble Key Project (Freedman et al 2001) is made and discrepancies between the K-TFR distances and the HKP I-TFR distances are discussed. Implications for Lamda-CDM cosmology are considered with H0=84 km s-1 Mpc-1. (Abridged)Comment: 37 pages including 12 tables and 7 figures. Final version accepted for publication in the Journal of Astrophysics & Astronom

A Project Based Approach to Statistics and Data Science

In an increasingly data-driven world, facility with statistics is more important than ever for our students. At institutions without a statistician, it often falls to the mathematics faculty to teach statistics courses. This paper presents a model that a mathematician asked to teach statistics can follow. This model entails connecting with faculty from numerous departments on campus to develop a list of topics, building a repository of real-world datasets from these faculty, and creating projects where students interface with these datasets to write lab reports aimed at consumers of statistics in other disciplines. The end result is students who are well prepared for interdisciplinary research, who are accustomed to coping with the idiosyncrasies of real data, and who have sharpened their technical writing and speaking skills

A Complete Catalog of Swift GRB Spectra and Durations: Demise of a Physical Origin for Pre-Swift High-Energy Correlations

We calculate durations and spectral paramaters for 218 Swift bursts detected by the BAT instrument between and including GRBs 041220 and 070509, including 77 events with measured redshifts. Incorporating prior knowledge into the spectral fits, we are able to measure the characteristic $\nu F_{\nu}$ spectral peak energy $E_{\rm pk,obs}$ and the isotropic equivalent energy $E_{\rm iso}$ (1--$10^4$ keV) for all events. This complete and rather extensive catalog, analyzed with a unified methodology, allows us to address the persistence and origin of high-energy correlations suggested in pre-Swift observations. We find that the $E_{\rm pk,obs}$-$E_{\rm iso}$ correlation is present in the Swift sample; however, the best-fit powerlaw relation is inconsistent with the best-fit pre-Swift relation at >5 sigma significance. Moreover, it has a factor >~ 2 larger intrinsic scatter, after accounting for large errors on $E_{\rm pk,obs}$. A large fraction of the Swift events are hard and subluminous relative to (and inconsistent with) the pre-Swift relation, in agreement with indications from BATSE GRBs without redshift. Moreover, we determine an experimental threshold for the BAT detector and show how the $E_{\rm pk,obs}$--$E_{\rm iso}$ correlation arises artificially due to partial correlation with the threshold. We show that pre-Swift correlations found by Amati et al.(2002), Yonetoku et al. (2004), Firmani et al.(2006) (and independently by others) are likely unrelated to the physical properties of GRBs and are likely useless for tests of cosmology. Also, an explanation of these correlations in terms of a detector threshold provides a natural and quantitative explanation for why short-duration GRBs and events at low redshift tend to be outliers to the correlations.Comment: 25 pages, 9 figures, 2 tables, Accepted to Ap

Evolution of the X-ray Emission of Radio-Quiet Quasars

We report new Chandra observations of seven optically faint, z \sim 4 radio-quiet quasars. We have combined these new observations with previous Chandra observations of radio-quiet quasars to create a sample of 174 sources. These sources have 0.1 < z < 4.7, and 10^{44} ergs s^{-1} < nu L_{nu} (2500 \AA) < 10^{48} ergs s^{-1}. The X-ray detection fraction is 90%. We find that the X-ray loudness of radio-quiet quasars decreases with UV luminosity and increases with redshift. The model that is best supported by the data has a linear dependence of optical-to-X-ray ratio, alpha_{ox}, on cosmic time, and a quadratic dependence of alpha_{ox} on log L_{UV}, where alpha_{ox} becomes X-ray quiet more rapidly at higher log L_{UV}. We find no significant evidence for a relationship between the X-ray photon index, Gamma_X, and the UV luminosity, and we find marginally significant evidence that the X-ray continuum flattens with increasing z (2 sigma). The Gamma_X-z anti-correlation may be the result of X-ray spectral curvature, redshifting of a Compton reflection component into the observed Chandra band, and/or redshifting of a soft excess out of the observed Chandra band. Using the results for Gamma_X, we show that the alpha_{ox}-z relationship is unlikely to be a spurious result caused by redshifting of the observable X-ray spectral region. A correlation between alpha_{ox} and z implies evolution of the accretion process. We present a qualitative comparison of these new results with models for accretion disk emission.Comment: Accepted by ApJ, 48 pages, 10 figures, 5 table

The origins of X-ray emission from the hotspots of FRII radio sources

We use new and archival Chandra data to investigate the X-ray emission from a large sample of compact hotspots of FRII radio galaxies and quasars from the 3C catalogue. We find that only the most luminous hotspots tend to be in good agreement with the predictions of a synchrotron self-Compton model with equipartition magnetic fields. At low hotspot luminosities inverse-Compton predictions are routinely exceeded by several orders of magnitude, but this is never seen in more luminous hotspots. We argue that an additional synchrotron component of the X-ray emission is present in low-luminosity hotspots, and that the hotspot luminosity controls the ability of a given hotspot to produce synchrotron X-rays, probably by determining the high-energy cutoff of the electron energy spectrum. It remains plausible that all hotspots are close to the equipartition condition.Comment: 49 pages, 16 figures. ApJ accepted. Revised version fixes a typo in one of the Tables and corrects a statement about 3C27

What Powers the Compact Radio Emission in Nearby Elliptical and S0 Galaxies?

Many nearby early-type (elliptical and S0) galaxies contain weak (milli-Jansky level) nuclear radio sources on scales a few hundred parsecs or less. The origin of the radio emission, however, has remained unclear, especially in volume-limited samples that select intrinsically less luminous galaxies. Both active galactic nuclei and nuclear star formation have been suggested as possible mechanisms for producing the radio emission. This paper utilizes optical spectroscopic information to address this issue. A substantial fraction of the early-type galaxies surveyed with the Very Large Array by Wrobel & Heeschen (1991) exhibits detectable optical emission lines in their nuclei down to very sensitive limits. Comparison of the observed radio continuum power with that expected from the thermal gas traced by the optical emission lines implies that the bulk of the radio emission is nonthermal. Both the incidence and the strength of optical line emission correlate with the radio power. At a fixed line luminosity, ellipticals have stronger radio cores than S0s. The relation between radio power and line emission observed in this sample is consistent with the low-luminosity extension of similar relations seen in classical radio galaxies and luminous Seyfert nuclei. A plausible interpretation of this result is that the weak nuclear sources in nearby early-type galaxies are the low-luminosity counterparts of more powerful AGNs. The spectroscopic evidence supports this picture. Most of the emission-line objects are optically classified as Seyfert nuclei or low-ionization nuclear emission-line regions (LINERs), the majority of which are likely to be accretion-powered sources.Comment: LaTex, 16 pages including embedded figures. Accepted for publication in the Astrophysical Journa