Comparison of and in the 1000 genomes data.

Being able to properly quantify genetic differentiation is key to understanding the evolutionary potential of a species. One central parameter in this context is FST, the mean coancestry within populations relative to the mean coancestry between populations. Researchers have been estimating FST globally or between pairs of populations for a long time. More recently, it has been proposed to estimate population-specific FST values, and population-pair mean relative coancestry. Here, we review the several definitions and estimation methods of FST, and stress that they provide values relative to a reference population. We show the good statistical properties of an allele-sharing, method of moments based estimator of FST (global, population-specific and population-pair) under a very general model of population structure. We point to the limitation of existing likelihood and Bayesian estimators when the populations are not independent. Last, we show that recent attempts to estimate absolute, rather than relative, mean coancestry fail to do so.</div

RMSEs of in the 1000 genomes.

Being able to properly quantify genetic differentiation is key to understanding the evolutionary potential of a species. One central parameter in this context is FST, the mean coancestry within populations relative to the mean coancestry between populations. Researchers have been estimating FST globally or between pairs of populations for a long time. More recently, it has been proposed to estimate population-specific FST values, and population-pair mean relative coancestry. Here, we review the several definitions and estimation methods of FST, and stress that they provide values relative to a reference population. We show the good statistical properties of an allele-sharing, method of moments based estimator of FST (global, population-specific and population-pair) under a very general model of population structure. We point to the limitation of existing likelihood and Bayesian estimators when the populations are not independent. Last, we show that recent attempts to estimate absolute, rather than relative, mean coancestry fail to do so.</div

Unequal sample sizes and subsampling populations.

Being able to properly quantify genetic differentiation is key to understanding the evolutionary potential of a species. One central parameter in this context is FST, the mean coancestry within populations relative to the mean coancestry between populations. Researchers have been estimating FST globally or between pairs of populations for a long time. More recently, it has been proposed to estimate population-specific FST values, and population-pair mean relative coancestry. Here, we review the several definitions and estimation methods of FST, and stress that they provide values relative to a reference population. We show the good statistical properties of an allele-sharing, method of moments based estimator of FST (global, population-specific and population-pair) under a very general model of population structure. We point to the limitation of existing likelihood and Bayesian estimators when the populations are not independent. Last, we show that recent attempts to estimate absolute, rather than relative, mean coancestry fail to do so.</div

Estimates of F_ST from the 1, 000 genomes.

The top row shows estimates based on the 2,504 individuals and 77, 818, 345 SNPs from phase 3 of the 1, 000 genomes project. Panel A shows ; the darker and the larger the circle, the larger the elements, either positive (blue) or negative (red); scale on the right. Panel B shows pairwise , all positives; The darker the colour and the larger the circle, the larger the element; scale at the bottom. Panel C shows the distribution of RMSEs for . ‘2.2m L, 220k L, 22k L, 2.2k L’ (in blue): subsampling of the corresponding number of SNPs from the total data set; ‘50 i, 20 i, 10 i, 5 i’ (in red); subsampling of the corresponding number of individuals from the total dataset. Panel D shows estimated from 10 individuals (100 replicates) against estimated from all individuals.</p

in a 1D stepping stone model.

10 populations with N = 1, 000; m = 0.005 between adjacent populations. The default sample size and number of loci for estimates are 50 individuals in each population and 104 SNPs. Panel A shows the expected FST after 4, 000 generations from the transition equations (Eq 4); The darker and the larger the circle, the larger the elements, either positive (blue) or negative (red); scale at the bottom. Panel B shows the distributions of Root Mean Square errors (RMSE) for all the elements of based on 20 or 100 replicates. ‘10k L, 1k L, 100 L’: 104, 103 and 100 SNPs respectively (red); ‘AFM’: 100 loci, using the Bayesian estimate from AFM (black); ‘20 i, 10 i, 5 i, 2 i’: Subsampling of 20, 10, 5 or 2 individuals, 104 loci (blue); the vertical bars separate the different sampling schemes / estimates. The four lower panels show the relation between expected (E[FST]) and estimated FST for 104 SNPs (panel C); 104 SNPs but only two individuals (panel D); 100 SNPs (panel E); and 100 SNPs, AFM method (panel F); red color shows the effect of subsampling of loci, blue subsampling of individuals and black the Bayesian estimates.</p

Chromosome specific against from the 1000 genomes data.

Chromosome specific against from the 1000 genomes data.</p

Allele-sharing, kinship and inbreeding for a <i>k</i>−ploid species.

Allele-sharing, kinship and inbreeding for a k−ploid species.</p

Migration, coancestries and <i>F</i>_ST for three migration models.

The mutation rate for all models is set to μ = 10−8. The top row shows three migration matrices for sets of six populations, a continent-islands (panel A), where the continent (leftmost column, N = 109) sends migrants to all other columns, and receives none, and six islands of sizes N = 10, 50, 100, 500, 1000, 5000 (orange, red, brown, purple, blue, green respectively) only receive immigrants from the continent at rate m = 0.001; a finite island (panel B) where each island (N = 10, 50, 100, 500, 1000, 5000; orange, red, brown, purple, blue, green respectively) sends and receives the same proportion m = 0.001/5 to/from all other islands; and a finite one dimensional (1D) stepping stone model(panel C) where populations of size N = 10, 50, 1000, 1000, 100, 100 (orange, red, brown, purple, blue, green respectively) receive a proportion m = 0.01/2 of immigrants from their left and right neighbours. White: cells with zero migration; light grey: cells with a positive migration term; dark grey: self. Middle row: Dynamics of within-population coancestries for The continent islands model (D), the finite islands model (E) and the stepping-stone model (F) through time, for the six different populations. Bottom row: Dynamics of population-specific FST for the continent islands model (G), the finite islands model (H) and the stepping-stone model (I) through time, for the six different populations.</p

in a finite island model.

10 populations with different effective sizes: N1 = N2 = 1, 000; N3 = N4 = 10; N5 = N6 = 100; N7 = N8 = 500; N9 = N10 = 2, 000, and a migration rate m = 0.001. The default sample size and number of loci for estimates are 50 individuals in each population and 104 SNPs. Panel A shows the expected (FST) after 2, 000 generations from the transition equations (Eq 4); The darker and the larger the circle, the larger the elements, either positive (blue) or negative (red); scale at the bottom. Panel B shows the distributions of Root Mean Square errors (RMSE) for all the elements of based on 20 or 100 replicates. ‘10k L, 1k L’: subsampling of 104, 103 SNPs respectively (red); ‘AFM’: Bayesian estimator from AFM, based on 1, 000 SNPs (black); ‘20 i, 10 i, 5 i, 2 i’: Subsampling of 20, 10, 5 or 2 individuals, 104 SNPs (blue); the vertical bars separate the different sampling schemes / estimates. The four lower panels show the relation between expected and estimated FST for 104 SNPs (panel C), 104 SNPs, 2 individuals (panel D), 103 SNPs (panel E), AFM method with 103 SNPs (panel F). Red color illustrates subsampling of loci, blue subsampling of individuals and black the Bayesian estimator.</p

Population-specific estimates of <i>F</i>_ST from the 1000 genomes.

Population-specific estimates of FST from the 1000 genomes.</p