Inverse normal transformations applied to the partially overlapping samples t-tests by Derrick et.al. (2017) are considered for their Type I error robustness and power. The inverse normal transformation solutions proposed in this paper are shown to maintain Type I error robustness. For increasing degrees of skewness they also offer improved power relative to the parametric partially overlapping samples t-tests. The power when using inverse normal transformation solutions are comparable to rank based non-parametric solutions

Derrick, Ben

Toher, Deirdre

White, Paul

English

UWE Bristol Research Repository

Derrick, B., White, P., and Toher, D.  Page 1 of 14  An Inverse Normal Transformation Solution for the comparison of two samples that contain both paired observations and independent observations.  Abstract  Inverse normal transformations applied to the partially overlapping samples t-tests by Derrick et.al. (2017) are considered for their Type I error robustness and power. The inverse normal transformation solutions proposed in this paper are shown to maintain Type I error robustness. For increasing degrees of skewness they also offer improved power relative to the parametric partially overlapping samples t-tests. The power when using inverse normal transformation solutions are comparable to rank based non-parametric solutions.  Introduction  A frequently asked question in research is how to compare means between two samples that include both paired observations and unpaired observations (Derrick, Toher and White, 2017). This is referred to as ‘partially overlapping samples’ (Derrick et.al., 2015).  Parametric test statistics for the comparison of means of two samples that contain both paired observations and independent observations are given by Derrick et.al. (2017). These partially overlapping samples t-tests use all of the available data, and are Type I error robust and more powerful than conventional methods, when any missing data is missing completely at random.   The partially overlapping samples t-tests are based on the assumption of normality (Derrick, 2017). To remove the restriction of the normality assumption, the application Derrick, B., White, P., and Toher, D.  Page 2 of 14  of an Inverse Normal Transformation (INT) is proposed in this paper. This method is compared to the existing partially overlapping samples t-tests, and a simple ranking based non-parametric method.   The parametric partially overlapping samples test statistic, new1T , is an interpolation between the paired samples t-test and the independent samples t-test assuming equal variances, and is given by Derrick et.al. (2017) as:  −+−=212121new1211XXnnnrnnSTcp where =pS )1()1()1()1(21222211−+−−+−nnSnSn The test statistic new1T  is referenced against the t-distribution with degrees of freedom:  1v  = ( )bacbacbac nnnnnnnnn +++−+++−21)1( . where for j = {Group 1, Group 2},  =jX mean of Sample j, =an  number of unpaired observations exclusive to Sample 1, =bn  number of unpaired observations exclusive to Sample 2, =cn  number of pairs, =jn  total number of observations in Sample j, =2jS  variance of Sample j, and r  = Pearson’s correlation coefficient for the paired observations.   The independent samples t-test is not Type I error robust when variances are not equal and sample sizes are not equal. It follows that new1T  is also sensitive to deviations from the equal variances assumption. If equal variances cannot be assumed, then alternatively Welch’s t-test is Type I error robust under normality (Derrick, Toher and White, 2016). The partially overlapping samples t-test not constrained to equal variances, new2T , is an interpolation between the paired samples t-test and Welch’s t-test, is given by Derrick et.al. (2017) as: Derrick, B., White, P., and Toher, D.  Page 3 of 14  −+−=212122212121new22XXnnnSSrnSnSTc The test statistic new2T  is referenced against the t-distribution with degrees of freedom:  2v  = ( )bacbacc nnnnnnn ++++−+−21)1( γ  where ( ) ( )1/1/ 22222121212222121−+−+=nnSnnSnSnSγ   For simple ranking based non-parametric solutions, observations are pooled and assigned rank values in ascending order. The rank values are substituted into the elements of the calculation for new1T  and new2T  in place of the observed values. This gives the test statistics 1RNKT  and 2RNKT  respectively. The degrees of freedom 1υ  and 2υ  respectively, are calculated using the pooled rank values.   The transforming of data attempts to overcome violations of the normality assumption, so that the traditional parametric tests can be applied. For equal sample sizes, Cohen and Arthur (1991) found that the independent samples t-tests performed on log or squared data, exhibits satisfactory Type I error robustness. However, popular transformations such as the Box-Cox transformations do not necessarily lead to Type I error robust tests when sample sizes are unequal or variances are unequal (Zarembka, 1974). In addition the Box-Cox transformation rarely results in both normality and equal variances at the same time (Sakia, 1992). Even if researchers are comfortable with the hypothesis of comparing means of transformed data, a suitable transformation may not always be found.   A transformation that should always give the appearance of normally distributed data, is an Inverse Normal Transformation (INT), which derives properties from the Normal distribution. Methods based on Fisher and Yates Normal approximation are the most commonly used INT (Beasley and Erickson, 2009), and are the most Derrick, B., White, P., and Toher, D.  Page 4 of 14  powerful (Bradley, 1968). Example INT approaches include Blom’s method and Van der Waerden’s method, but it is of little consequence which of them is selected because they are linear transformations of one-another (Tukey, 1962). It should be noted that an INT does not make a population Normal, but it makes a sample appear Normal. This is not the same as directly ensuring the assumption of normally distributed residuals is true (Servin and Stephens, 2007). In addition, care needs to be taken with the interpretation of results. The null hypothesis of equal means for the transformed data, is assumed to be equivalent to a null hypothesis of equal means.  The Fisher and Yates INT procedure requires pooling all of the observations, sorting into ascending order and ranking each observation over the entire sample so that: +−−Φ= −121cNcyX ii  where iy  is the ordinary rank of observation i, N  is the total pooled sample size, and 1−Φ is the standard Normal quantile function. The most simple INT method is attributed to Van Der Waerden (1952) uses 0=c . Penfield (1994) found that Van Der Waerden transformations applied to the independent samples t-test, are Type I errors robust across a variety of distributions.  Calculating the Van Der Waerden scores, mX , and using these within the calculation of new1T  and new2T , gives distribution free test statistics 1INTT  and 2INTT  respectively. The degrees of freedom 1υ  and 2υ  respectively, are calculated using the pooled transformed values. The calculation of r is Pearson’s correlation coefficient between the transformed paired observations.    Methodology  For each of the six test statistics defined above ( new1T , new2T , 1RNKT , 2RNKT , 1INTT , 2INTT ), the robustness for validity (i.e. Type I errors) and efficacy (i.e. power) are explored Derrick, B., White, P., and Toher, D.  Page 5 of 14  using simulation. Values of an , bn , cn  {5, 10, 30, 50, 100, 500} are varied in a factorial design, as well as values of the population correlation coefficient ρ  {-.75 ,-.50, -.25, .00, .25, .50, .75}.   To generate the independent observations, firstly the Mersenne-Twister algorithm (Matsumoto and Nishimura, 1998) generates random U(0,1) deviates. These uniform deviates are transformed into N(0,1) deviates using the Paley and Wiener (1934) transformation.  To generate the paired observations, the approach used is equivalent to that used by Derrick and White (2017) and Derrick et.al. (2017). In this approach, additional Standard Normal deviates are transformed to correlated Standard Normal bivariates, jiz , as follows:  =iz1 ii zz 21 2121 ρρ −++  and =iz2 ii zz 21 2121 ρρ −−+  where i = (1, 2, …., cn )   Each of the test statistics are assessed firstly under the Standard Normal distribution. For the comparison of test statistics under non-normality, observations are generated by transformation of the Standard Normal deviates as given in Table 1.  Table 1. Transformations applied to Standard Normal deviates (N) to obtain non-normally distributed deviates (X), with the resulting skewness and kurtosis. Note that Uniform (U) deviates are calculated as the cumulative density function of N. Distribution Transformation Skewness Kurtosis Normal (N) X 0.000 3.000 Gumbel X= -log(-log U) 1.140 5.400 Exponential X= -log (U) -1 2.004 9.000 Lognormal X= exponential (N) 6.145 107.256  Derrick, B., White, P., and Toher, D.  Page 6 of 14  Following the transformations as per Table 1, the calculation of ranks / inverse normal transformations, are performed as appropriate for each statistical test. Each of the tests are performed at the 5% significance level, two sided.  For each of the parameter combinations within the factorial design, the null hypothesis rejection rate (NHRR) is recorded as the proportion of the 10,000 replicates where the null hypothesis is rejected. The methodology is depicted in Figure 1. Analyses under the alternative hypothesis proceeds similarly, but with the addition of 0.5 to the 2n  observations directly following the transformation in Table 1.  Figure 1. Overview of the simulation process.    Derrick, B., White, P., and Toher, D.  Page 7 of 14  Results  Under the null hypothesis, 10,000 iterations are obtained for each of the 1,512 parameter combinations. The Type I error rates for each of the test statistics across the simulation design are given through Figure 2 to Figure 5. Each parameter combination which has a Type I error rate between 2.5% and 7.5%, is considered to be maintaining reasonable Type I error robustness   Figure 2. Type I error rates for the Standard Normal distribution.  Figure 2 shows that when both samples are taken from the Standard Normal distribution, all of the test statistics are Type I error robust for all of the parameter combinations within the simulation design. Derrick, B., White, P., and Toher, D.  Page 8 of 14   Figure 3. Type I error rates for the Gumbel distribution.   Figure 3 shows that when both samples are taken from the Gumbel distribution, all of the test statistics are Type I error robust for all of the parameter combinations within the simulation design. This suggests that all of the test statistics are Type I error robust for distributions with a relatively small degree of skewness.  Derrick, B., White, P., and Toher, D.  Page 9 of 14   Figure 4. Type I error rates for the Exponential distribution.   Figure 4 shows that when both samples are taken from the Exponential distribution, all of the test statistics are Type I error robust when sample sizes are equal. However the test statistic new2T  is not Type I error robust when there is a large imbalance between the size of the two samples. This suggests that all of the test statistics except new2T  are Type I error robust for distributions with a moderate skew.   Derrick, B., White, P., and Toher, D.  Page 10 of 14   Figure 5. Type I error rates for the Lognormal distribution.  Figure 5 shows that when both samples are taken from the Lognormal distribution, all of the test statistics are Type I error robust with the exception of new2T  which can be occasionally conservative when sample sizes are equal, and can be liberal when sample sizes are not equal. This suggests that all of the test statistics except new2T  maintain Type I error robustness for heavily skewed distributions.  For the simulations under the alternative hypothesis Table 2 gives the average power across the simulation design of sample size and correlation coefficient for each of the distributions. Power is only recorded for scenarios where the test statistic maintains Type I error robustness.    Derrick, B., White, P., and Toher, D.  Page 11 of 14  Table 2. Power for =α .05, =− 12 µµ .5, two sided, over all values of cn .  Distribution Sample size ρ   new1T  new2T  RNK1T RNK2T   INT1T  INT2T Normal an = bn  >0 .865 .864 .856 .855 .855 .854 0 .819 .819 .811 .811 .811 .811 <0 .779 .779 .772 .771 .770 .769 an  ≠  bn  >0 .839 .832 .829 .824 .827 .824 0 .806 .798 .795 .790 .795 .790 <0 .774 .767 .763 .760 .761 .758 Gumbel an = bn  >0 .783 .782 .718 .678 .815 .814 .824 .823 0 .720 .761 .760 .774 .774 <0 .678 .719 .719 .727 .726 an  ≠  bn  >0 .740 .735 .689 .651 .779 .776 .789 .786 0 .693 .740 .736 .749 .747 <0 .655 .702 .699 .712 .710 Exponential an = bn  >0 .867 .824 .795 .864 .938 .937 .946 .944 0 .824 .915 .914 .926 .925 <0 .795 .894 .894 .906 .906 an  ≠  bn  >0 .841 .811 .786 - .933 .930 .943 .938 0 .919 .917 .930 .926 <0 .904 .903 .918 .915 Lognormal an = bn  >0 .596 .590 .893 .891 .905 .904 0 .535 .533 .857 .856 .911 .912 <0 .506 .506 .826 .826 .918 .925 an  ≠  bn  >0 .514 - .874 .873 .879 .876 0 .467 .851 .850 .850 .851 <0 .438 .825 .826 .848 .849  When population variances are equal, Table 2 shows that the test statistics not assuming equal variances, new2T , RNK2T  and INT2T , perform similarly to their counterparts where equal variances are assumed new1T , RNK1T  and INT1T  respectively.   From Table 2 it can be seen that for normally distributed data, the parametric statistics new1T  and new2T  are marginally more powerful than the other test statistics considered, but not to any meaningful extent.   Derrick, B., White, P., and Toher, D.  Page 12 of 14  The relative power advantage of INT1T  and INT2T  over new1T  and new2T  increases as the degree of skewness increases. However, the proposed statistics using inverse normal transformations, INT1T  and INT2T  , yield very similar results to RNK1T  and RNK2T .   Conclusion  Test statistics making use of all of the available data in a partially overlapping samples design are compared using simulation. Assuming normality the partially overlapping samples t-test proposed by Derrick et.al. (2017) for equal variances, new1T , is more powerful than non-parametric equivalents and Inverse Normal transformations.   The test statistics making use of Inverse Normal Transformations offer no substantial improvement over the non-parametric tests. Due to its Type I error robustness, power properties and relative simplicity, RNK1T  is recommended over INT1T  as the best solution for comparing partially overlapping samples from non-normal distributions.   References  Beasley, T. M., Erickson, S. & Allison, D. B. (2009). Rank-based inverse normal transformations are increasingly used, but are they merited?. Behavior genetics. 39(5), 580-595.  Bradley, J. V. (1968). Distribution-free statistical tests. Prentice-Hall; New York. Derrick, B., White, P., and Toher, D.  Page 13 of 14   Cohen, M. & Arthur, J. (1991). Randomization analysis of dental data characterized by skew and variance heterogeneity. Community dentistry and oral epidemiology. 19(4), 185-189.  Derrick, B. (2017) Statistics: New t-tests for the comparison of two partially overlapping samples. In: Faculty of Environment and Technology Degree Show, UWE, Frenchay Campus, UWE, 1 June 2017. Available from: http://eprints.uwe.ac.uk/31766.  Derrick, B., Dobson-McKittrick, A., Toher, D. & White P. (2015). Test statistics for comparing two proportions with partially overlapping samples. Journal of Applied Quantitative Methods. 10(3), 1-14.  Derrick, B., Russ, B., Toher, D. & White P. (2017). Test statistics for the comparison of means for two samples which include both paired observations and independent observations. Journal of Modern Applied Statistical Methods. 16(1), 137-157.  Derrick, B., Toher, D. & White, P. (2016). Why Welch’s test is Type I error robust. The Quantitative Methods for Psychology. 12(1), 30-38.  Derrick, B., Toher, D. & White, P. (2017). How to compare the means of two samples that include paired observations and independent observations: A companion to Derrick, Russ, Toher and White (2017). The Quantitative Methods for Psychology. 13(2), 120-126.  Derrick, B. and White, P. (2017) Comparing two samples from an individual Likert question. International Journal of Mathematics and Statistics. 18(3), 1-13. Derrick, B., White, P., and Toher, D.  Page 14 of 14  Paley, R. E. A. C. & Wiener, N. (1934). Fourier transforms in the complex domain. American Mathematical Society. 19  Sakia, R. M. (1992). The Box-Cox transformation technique: a review. The Statistician. 169-178.  Servin, B. & Stephens, M. (2007). Imputation-based analysis of association studies: candidate regions and quantitative traits. PLoS Genet, 3(7), e114.  Tukey, J. W. (1962). The future of data analysis. The Annals of Mathematical Statistics, 33(1), 1-67.  Van der Waerden, B. L. (1952). Order tests for the two-sample problem and their power. Indagationes Mathematicae, 14(253), 453-458.  Zarembka, P. (1972). Transformation of variables in econometrics. Transformation of variables in econometrics. In Econometrics, 261-264.    

An inverse normal transformation solution for the comparison of two samples that contain both paired observations and independent observations

https://uwe-repository.worktribe.com/883340/1/INT solution to the partially overlapping samples problem.pdf

An inverse normal transformation solution for the comparison of two samples that contain both paired observations and independent observations

Abstract

Similar works

Full text

Available Versions

UWE Bristol Research Repository