Multiple testing via FDRLFDR_L for large-scale imaging data

The multiple testing procedure plays an important role in detecting the presence of spatial signals for large-scale imaging data. Typically, the spatial signals are sparse but clustered. This paper provides empirical evidence that for a range of commonly used control levels, the conventional $\operatorname {FDR}$ procedure can lack the ability to detect statistical significance, even if the $p$-values under the true null hypotheses are independent and uniformly distributed; more generally, ignoring the neighboring information of spatially structured data will tend to diminish the detection effectiveness of the $\operatorname {FDR}$ procedure. This paper first introduces a scalar quantity to characterize the extent to which the "lack of identification phenomenon" ($\operatorname {LIP}$) of the $\operatorname {FDR}$ procedure occurs. Second, we propose a new multiple comparison procedure, called $\operatorname {FDR}_L$, to accommodate the spatial information of neighboring $p$-values, via a local aggregation of $p$-values. Theoretical properties of the $\operatorname {FDR}_L$ procedure are investigated under weak dependence of $p$-values. It is shown that the $\operatorname {FDR}_L$ procedure alleviates the $\operatorname {LIP}$ of the $\operatorname {FDR}$ procedure, thus substantially facilitating the selection of more stringent control levels. Simulation evaluations indicate that the $\operatorname {FDR}_L$ procedure improves the detection sensitivity of the $\operatorname {FDR}$ procedure with little loss in detection specificity. The computational simplicity and detection effectiveness of the $\operatorname {FDR}_L$ procedure are illustrated through a real brain fMRI dataset.Comment: Published in at http://dx.doi.org/10.1214/10-AOS848 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

FDR and victims of family violence: Ensuring a safe process and outcomes

Family dispute resolution (FDR) is a positive first-stop process for family law matters, particularly those relating to disputes about children. FDR provides the parties with flexibility within a positive, structured and facilitated framework for what are often difficult and emotional negotiations. However, there are a range of issues that arise for victims of family violence in FDR that can make it a dangerous and unsafe process for them unless appropriate precautions are taken. This article discusses the nature of FDR and identifies the many positive aspects of it for women participants. The article then considers the nature and dynamic of family violence in order to contextualise the discussion that follows regarding concerns for the safety of participants in the FDR process. Finally, it offers some suggestions about how Australia could approach FDR differently to make it safer for victims of family violence

False discovery and false nondiscovery rates in single-step multiple testing procedures

Results on the false discovery rate (FDR) and the false nondiscovery rate (FNR) are developed for single-step multiple testing procedures. In addition to verifying desirable properties of FDR and FNR as measures of error rates, these results extend previously known results, providing further insights, particularly under dependence, into the notions of FDR and FNR and related measures. First, considering fixed configurations of true and false null hypotheses, inequalities are obtained to explain how an FDR- or FNR-controlling single-step procedure, such as a Bonferroni or \u{S}id\'{a}k procedure, can potentially be improved. Two families of procedures are then constructed, one that modifies the FDR-controlling and the other that modifies the FNR-controlling \u{S}id\'{a}k procedure. These are proved to control FDR or FNR under independence less conservatively than the corresponding families that modify the FDR- or FNR-controlling Bonferroni procedure. Results of numerical investigations of the performance of the modified \u{S}id\'{a}k FDR procedure over its competitors are presented. Second, considering a mixture model where different configurations of true and false null hypotheses are assumed to have certain probabilities, results are also derived that extend some of Storey's work to the dependence case.Comment: Published at http://dx.doi.org/10.1214/009053605000000778 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Controlling the False Discovery Rate in Astrophysical Data Analysis

The False Discovery Rate (FDR) is a new statistical procedure to control the number of mistakes made when performing multiple hypothesis tests, i.e. when comparing many data against a given model hypothesis. The key advantage of FDR is that it allows one to a priori control the average fraction of false rejections made (when comparing to the null hypothesis) over the total number of rejections performed. We compare FDR to the standard procedure of rejecting all tests that do not match the null hypothesis above some arbitrarily chosen confidence limit, e.g. 2 sigma, or at the 95% confidence level. When using FDR, we find a similar rate of correct detections, but with significantly fewer false detections. Moreover, the FDR procedure is quick and easy to compute and can be trivially adapted to work with correlated data. The purpose of this paper is to introduce the FDR procedure to the astrophysics community. We illustrate the power of FDR through several astronomical examples, including the detection of features against a smooth one-dimensional function, e.g. seeing the ``baryon wiggles'' in a power spectrum of matter fluctuations, and source pixel detection in imaging data. In this era of large datasets and high precision measurements, FDR provides the means to adaptively control a scientifically meaningful quantity -- the number of false discoveries made when conducting multiple hypothesis tests.Comment: 15 pages, 9 figures. Submitted to A

Nonlinear response and fluctuation dissipation relations

A unified derivation of the off equilibrium fluctuation dissipation relations (FDR) is given for Ising and continous spins to arbitrary order, within the framework of Markovian stochastic dynamics. Knowledge of the FDR allows to develop zero field algorithms for the efficient numerical computation of the response functions. Two applications are presented. In the first one, the problem of probing for the existence of a growing cooperative length scale is considered in those cases, like in glassy systems, where the linear FDR is of no use. The effectiveness of an appropriate second order FDR is illustrated in the test case of the Edwards-Anderson spin glass in one and two dimensions. In the second one, the important problem of the definition of an off equilibrium effective temperature through the nonlinear FDR is considered. It is shown that, in the case of coarsening systems, the effective temperature derived from the second order FDR is consistent with the one obtained from the linear FDR.Comment: 24 pages, 6 figure

Generalized fluctuation-dissipation relation and effective temperature in off-equilibrium colloids

The fluctuation-dissipation relation (FDR), a fundamental result of equilibrium statistical physics, ceases to be valid when a system is taken out of the equilibrium. A generalization of FDR has been theoretically proposed for out-of-equilibrium systems: the kinetic temperature entering FDR is substituted by a time-scale dependent effective temperature. We combine the measurements of the correlation function of the rotational dynamics of colloidal particles obtained via dynamic light scattering with those of the birefringence response to study the generalized FDR in an off-equilibrium clay suspension undergoing aging. We i) find that FDR is strongly violated in the early stage of the aging process and is gradually recovered as the aging time increases and, ii), we determine the aging time evolution of the effective temperature, giving support to the proposed generalization scheme.Comment: 4 pages, 3 figure