Nonparametric Methods in Astronomy: Think, Regress, Observe -- Pick Any Three

Telescopes are much more expensive than astronomers, so it is essential to minimize required sample sizes by using the most data-efficient statistical methods possible. However, the most commonly used model-independent techniques for finding the relationship between two variables in astronomy are flawed. In the worst case they can lead without warning to subtly yet catastrophically wrong results, and even in the best case they require more data than necessary. Unfortunately, there is no single best technique for nonparametric regression. Instead, we provide a guide for how astronomers can choose the best method for their specific problem and provide a python library with both wrappers for the most useful existing algorithms and implementations of two new algorithms developed here.Comment: 19 pages, PAS

A Bayesian approach to the analysis of time symmetry in light curves: Reconsidering Scorpius X-1 occultations

We present a new approach to the analysis of time symmetry in light curves, such as those in the x-ray at the center of the Scorpius X-1 occultation debate. Our method uses a new parameterization for such events (the bilogistic event profile) and provides a clear, physically relevant characterization of each event's key features. We also demonstrate a Markov Chain Monte Carlo algorithm to carry out this analysis, including a novel independence chain configuration for the estimation of each event's location in the light curve. These tools are applied to the Scorpius X-1 light curves presented in Chang et al. (2007), providing additional evidence based on the time series that the events detected thus far are most likely not occultations by TNOs.Comment: 24 pages, 18 figures. Preprint typeset using LaTeX style emulateapj v. 04/20/0

Monitoring Networked Applications With Incremental Quantile Estimation

Networked applications have software components that reside on different computers. Email, for example, has database, processing, and user interface components that can be distributed across a network and shared by users in different locations or work groups. End-to-end performance and reliability metrics describe the software quality experienced by these groups of users, taking into account all the software components in the pipeline. Each user produces only some of the data needed to understand the quality of the application for the group, so group performance metrics are obtained by combining summary statistics that each end computer periodically (and automatically) sends to a central server. The group quality metrics usually focus on medians and tail quantiles rather than on averages. Distributed quantile estimation is challenging, though, especially when passing large amounts of data around the network solely to compute quality metrics is undesirable. This paper describes an Incremental Quantile (IQ) estimation method that is designed for performance monitoring at arbitrary levels of network aggregation and time resolution when only a limited amount of data can be transferred. Applications to both real and simulated data are provided.Comment: This paper commented in: [arXiv:0708.0317], [arXiv:0708.0336], [arXiv:0708.0338]. Rejoinder in [arXiv:0708.0339]. Published at http://dx.doi.org/10.1214/088342306000000583 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Оптимальний вибір площин, на яких розміщені томограми, в комп’ютерній томографії

The solution of the problem of reconstructing the internal structure of a three-dimensional body by the known tomograms produced by a computer to-mograph using interflatation of functions and blending approximation is proposed. The known methods ofapproximating functions of one and two variables by interpolation type piecewise constant splines using means and medians are also considered. The paper presents an algorithm for optimizing the choice of the planes in which the tomogramsproduced by a computer tomograph are placed. The case is considered when all the tomograms are parallel to each other. The algorithm developeduses approximations of objects by classical piecewise constant splines. The internal structure of a three-dimensional body (density or absorption coefficient) is assumed to be given by a function of three variables of the form h(x, y, z ) = f (x)g( y, z), where g is an arbitrary function, provided that f is a monotone function on a closed segment. The method of optimal choice of the planes for placing the tomograms is implemented using MathCad computer software.Представлено розв’язок задачі відновлення внутрішньої структури тривимірного тіла за відомими томограмами, що поступають з комп’ютерного томографу, за допомогою інтерфлетації функцій та мішаної апроксимації. Розглянуто також відомі методи наближення функцій однієї та двоx змінних кусково-сталими сплайнами інтерполяційного типу, з використанням середніх та медіан. В статті пропонується алгоритм оптимізації вибору площин, на яких розміщені томограми, що поступають з комп’ютерного томографу. Розглядається випадок, коли всі томограми паралельні одна одній. Запропонований алгоритм використовує наближення об’єктів класичними кусково-сталими сплайнами. При побудові алгоритму істотно використовується припущення про те, що внутрішня структура тривимірного тіла (щільність або коефіцієнт поглинання) є функцією від трьох змінних вигляду h(x, y, z ) = f (x)g( y, z), де g – довільна функція, при умові, що f – монотонна функція на замкненому відрізку. Представлена чисельна реалізація методу оптимального вибору площин, на яких лежать томограми, в системі компʼютерної математики MathCad

Sequential Bayesian inference for static parameters in dynamic state space models

A method for sequential Bayesian inference of the static parameters of a dynamic state space model is proposed. The method is based on the observation that many dynamic state space models have a relatively small number of static parameters (or hyper-parameters), so that in principle the posterior can be computed and stored on a discrete grid of practical size which can be tracked dynamically. Further to this, this approach is able to use any existing methodology which computes the filtering and prediction distributions of the state process. Kalman filter and its extensions to non-linear/non-Gaussian situations have been used in this paper. This is illustrated using several applications: linear Gaussian model, Binomial model, stochastic volatility model and the extremely non-linear univariate non-stationary growth model. Performance has been compared to both existing on-line method and off-line methods

Fast Deterministic Selection

The Median of Medians (also known as BFPRT) algorithm, although a landmark theoretical achievement, is seldom used in practice because it and its variants are slower than simple approaches based on sampling. The main contribution of this paper is a fast linear-time deterministic selection algorithm QuickselectAdaptive based on a refined definition of MedianOfMedians. The algorithm's performance brings deterministic selection---along with its desirable properties of reproducible runs, predictable run times, and immunity to pathological inputs---in the range of practicality. We demonstrate results on independent and identically distributed random inputs and on normally-distributed inputs. Measurements show that QuickselectAdaptive is faster than state-of-the-art baselines.Comment: Pre-publication draf