Fixation index

The fixation index (F_ST) is a measure of population differentiation due to genetic structure. It is frequently estimated from genetic polymorphism data, such as single-nucleotide polymorphisms (SNP) or microsatellites. Developed as a special case of Wright's F-statistics, it is one of the most commonly used statistics in population genetics.

Definition

Two of the most commonly used definitions for F_ST at a given locus are based on the variance of allele frequencies between populations, and on the probability of Identity by descent.

If ${\displaystyle {\bar {p}}}$ is the average frequency of an allele in the total population, ${\displaystyle \sigma _{S}^{2}}$ is the variance in the frequency of the allele between different subpopulations, weighted by the sizes of the subpopulations, and ${\displaystyle \sigma _{T}^{2}}$ is the variance of the allelic state in the total population, F_ST is defined as [1]

${\displaystyle F_{ST}={\frac {\sigma _{S}^{2}}{\sigma _{T}^{2}}}={\frac {\sigma _{S}^{2}}{{\bar {p}}(1-{\bar {p}})}}}$

Wright's definition illustrates that F_ST measures the amount of genetic variance that can be explained by population structure. This can also be thought of as the fraction of total diversity that is not a consequence of the average diversity within subpopulations, where diversity is measured by the probability that two randomly selected alleles are different, namely ${\displaystyle 2p(1-p)}$. If the allele frequency in the ${\displaystyle i}$th population is ${\displaystyle p_{i}}$ and the relative size of the ${\displaystyle i}$th population is ${\displaystyle c_{i}}$, then

${\displaystyle F_{ST}={\frac {{\bar {p}}(1-{\bar {p}})-\sum c_{i}p_{i}(1-p_{i})}{{\bar {p}}(1-{\bar {p}})}}={\frac {{\bar {p}}(1-{\bar {p}})-{\overline {p(1-p)}}}{{\bar {p}}(1-{\bar {p}})}}}$

Alternatively,[2]

${\displaystyle F_{ST}={\frac {f_{0}-{\bar {f}}}{1-{\bar {f}}}}}$

where ${\displaystyle f_{0}}$ is the probability of identity by descent of two individuals given that the two individuals are in the same subpopulation, and ${\displaystyle {\bar {f}}}$ is the probability that two individuals from the total population are identical by descent. Using this definition, F_ST can be interpreted as measuring how much closer two individuals from the same subpopulation are, compared to the total population. If the mutation rate is small, this interpretation can be made more explicit by linking the probability of identity by descent to coalescent times: Let T₀ and T denote the average time to coalescence for individuals from the same subpopulation and the total population, respectively. Then,

${\displaystyle F_{ST}\approx 1-{\frac {T_{0}}{T}}}$

This formulation has the advantage that the expected time to coalescence can easily be estimated from genetic data, which led to the development of various estimators for F_ST.

Estimation

In practice, none of the quantities used for the definitions can be easily measured. As a consequence, various estimators have been proposed. A particularly simple estimator applicable to DNA sequence data is:[3]

${\displaystyle F_{ST}={\frac {\pi _{\text{Between}}-\pi _{\text{Within}}}{\pi _{\text{Between}}}}}$

where ${\displaystyle \pi _{\text{Between}}}$ and ${\displaystyle \pi _{\text{Within}}}$ represent the average number of pairwise differences between two individuals sampled from different sub-populations (${\displaystyle \pi _{\text{Between}}}$) or from the same sub-population (${\displaystyle \pi _{\text{Within}}}$). The average pairwise difference within a population can be calculated as the sum of the pairwise differences divided by the number of pairs. However, this estimator is biased when sample sizes are small or if they vary between populations. Therefore, more elaborate methods are used to compute F_ST in practice. Two of the most widely used procedures are the estimator by Weir & Cockerham (1984),[4] or performing an Analysis of molecular variance. A list of implementations is available at the end of this article.

Interpretation

This comparison of genetic variability within and between populations is frequently used in applied population genetics. The values range from 0 to 1. A zero value implies complete panmixis; that is, that the two populations are interbreeding freely. A value of one implies that all genetic variation is explained by the population structure, and that the two populations do not share any genetic diversity.

For idealized models such as Wright's finite island model, F_ST can be used to estimate migration rates. Under that model, the migration rate is

${\displaystyle M={\frac {m}{\mu }}\approx {\frac {1}{4}}\left({\frac {1}{F_{ST}}}-1\right)}$,

where m is the migration rate per generation, and ${\displaystyle \mu }$ is the mutation rate per generation.[5]

The interpretation of F_ST can be difficult when the data analyzed are highly polymorphic. In this case, the probability of identity by descent is very low and F_ST can have an arbitrarily low upper bound, which might lead to misinterpretation of the data. Also, strictly speaking F_ST is not a distance in the mathematical sense, as it does not satisfy the triangle inequality.

For populations of plants which clearly belong to the same species, values of F_ST greater than 15% are considered "great" or "significant" differentiation, while values below 5% are considered "small" or "insignificant" differentiation.[6] Values for mammal populations between subspecies, or closely related species, typical values are of the order of 5% to 20%. F_ST between the Eurasian and North American populations of the gray wolf were reported at 9.9%, those between the Red wolf and Gray wolf populations at between 17% and 18%. The Eastern wolf, a recently recognized highly admixed "wolf-like species" has values of F_ST below 10% in comparison with both Eurasian (7.6%) and North American gray wolves (5.7%), with the Red wolf (8.5%), and even an even lower value when paired with the Coyote (4.5%).[7]

F_ST in humans

F_ST values depend strongly on the choice of populations. Closely related ethnic groups, such as the Danes vs. the Dutch, or the French vs. the Spaniards show values significantly below 1%, indistinguishable from panmixia. Within Europe, the most divergent ethnic groups have been found to have values of the order of 7% (Lapps vs. Sardinians).

Larger values are found if highly divergent homogenous groups are compared: the highest such value found was at close to 46%, between Mbuti and Papuans.[8]

Autosomal genetic distances based on classical markers

In their study The History and Geography of Human Genes (1994), Cavalli-Sforza, Menozzi and Piazza provide some of the most detailed and comprehensive estimates of genetic distances between human populations, within and across continents. Their initial database contains 76,676 gene frequencies (using 120 blood polymorphisms), corresponding to 6,633 samples in different locations. By culling and pooling such samples, they restrict their analysis to 491 populations. They focus on aboriginal populations that were at their present location at the end of the 15th century when the great European migrations began.[9] When studying genetic difference at the world level, the number is reduced to 42 representative populations, aggregating subpopulations characterized by a high level of genetic similarity. For these 42 populations, Cavalli-Sforza and coauthors report bilateral distances computed from 120 alleles. Among this set of 42 world populations, the greatest genetic distance observed is between Mbuti Pygmies and Papua New Guineans, where the Fst distance is 0.4573, while the smallest genetic distance (0.0021) is between the Danish and the English. When considering more disaggregated data for 26 European populations, the smallest genetic distance (0.0009) is between the Dutch and the Danes, and the largest (0.0667) is between the Lapps and the Sardinians. The mean genetic distance among the 861 available pairings of the 42 selected populations was found to be 0.1338.. A genetic distance of 0.1338 implies that kinship between unrelated individuals of the same ancestry relative to the world population is equivalent to kinship between half siblings in a randomly mating population. This also implies that if a human from a given ancestral population has a mixed half-sibling, that human is closer genetically to an unrelated individual of their ancestral population than to their mixed half-sibling. [10]

Autosomal genetic distances based on SNPs

A 2012 study based on International HapMap Project data estimated F_ST between the three major "continental" populations of Europeans (combined from Utah residents of Northern and Western European ancestry from the CEPH collection and Italians from Tuscany), East Asians (combining Han Chinese from Beijing, Chinese from metropolitan Denver and Japanese from Tokyo, Japan) and Sub-Saharan Africans (combining Luhya of Webuye, Kenya, Maasai of Kinyawa, Kenya and Yoruba of Ibadan, Nigeria). It reported a value close to 12% between continental populations, and values close to panmixia (smaller than 1%) within continental populations.[11]

Intercontinental autosomal genetic distances based on SNPs[12]

	Europe (CEU)	Sub-Saharan Africa (Yoruba)	East-Asia (Japanese)
Sub-Saharan Africa (Yoruba)	0.153
East-Asia (Japanese)	0.111	0.190
East-Asia (Chinese)	0.110	0.192	0.007

Intra-European/mediterranean autosomal genetic distances based on SNPs[12][13]

	Italians	Palestinians	Swedish	Finns	Spanish	Germans	Russians	French	Greeks
Palestinians	0.0064
Swedish	0.0064-0.0090	0.0191
Finns	0.0130-0.0230		0.0050-0.0110
Spanish	0.0010-0.0050	0.0101	0.0040-0055	0.0110-0.0170
Germans	0.0029-0.0080	0.0136	0.0007-0.0010	0.0060-0.0130	0.0015-0.0030
Russians	0.0088-0.0120	0.0202	0.0030-0.0036	0.0060-0.0120	0.0070-0.0079	0.0030-0.0037
French	0.0030-0.0050		0.0020	0.0080-0.0150	0.0010	0.0010	0.0050
Greeks	0.0000	0.0057	0.0084		0.0035	0.0039	0.0108

Programs for calculating F_ST

• Arlequin
• Fstat
• SMOGD[14]
• diveRsity (R package)
• hierfstat (R package)
• FinePop[15] (R package)
• Microsatellite Analyzer (MSA)
• VCFtools
• DnaSP

Modules for calculating F_ST

• BioPerl
• BioPython

References

1. Holsinger, Kent E.; Bruce S. Weir (2009). "Genetics in geographically structured populations: defining, estimating and interpreting FST". Nat Rev Genet. 10 (9): 639–650. doi:10.1038/nrg2611. ISSN 1471-0056. PMC 4687486. PMID 19687804.
2. Richard Durrett (12 August 2008). Probability Models for DNA Sequence Evolution. Springer. ISBN 978-0-387-78168-6. Retrieved 25 October 2012.
3. Hudson, RR.; Slatkin, M.; Maddison, WP. (Oct 1992). "Estimation of Levels of Gene Flow from DNA Sequence Data". Genetics. 132 (2): 583–9. PMC 1205159. PMID 1427045.
4. Weir, B. S.; Cockerham, C. Clark (1984). "Estimating F-Statistics for the Analysis of Population Structure". Evolution. 38 (6): 1358–1370. doi:10.2307/2408641. ISSN 0014-3820. JSTOR 2408641. PMID 28563791.
5. Peter Beerli, Estimation of migration rates and population sizes in geographically structured populations (1998), Advances in molecular ecology (ed. G. Carvalho). NATO Science Series A: Life Sciences, IOS Press, Amsterdam, 39-53.
6. Frankham, R., Ballou, J.D., Briscoe, D.A., 2002. Introduction to Conservation Genetics. Cambridge University Press, Cambridge. Hartl DL, Clark AG (1997) Principles of Population Genetics, 3nd edn. Sinauer Associates, Inc, Sunderland, MA.
7. B. M. von Holdt et al., "Whole-genome sequence analysis shows that two endemic species of North American wolf are admixtures of the coyote and gray wolf", Science Advances 27 Jul 2016: Vol. 2, no. 7, e1501714, doi: 10.1126/sciadv.1501714.
8. Cavalli-Sforza et al. (1994), cited after V. Ginsburgh, S. Weber, The Palgrave Handbook of Economics and Language, Springer (2016), p. 182.
9. Cavalli-Sforza et al., 1994, p. 24
10. Harpending, Henry (2002). "Kinship and Population Subdivision". Population and Environment. 24 (2): 141–147. doi:10.1023/A:1020815420693. S2CID 15208802.
11. Elhaik, Eran (2012). "Empirical Distributions of FST from Large-Scale Human Polymorphism Data". PLOS ONE. 7 (11): e49837. Bibcode:2012PLoSO...749837E. doi:10.1371/journal.pone.0049837. PMC 3504095. PMID 23185452.
12. Nelis, Mari; et al. (2009-05-08). Fleischer, Robert C. (ed.). "Genetic Structure of Europeans: A View from the North–East". PLOS ONE. 4 (5): e5472. Bibcode:2009PLoSO...4.5472N. doi:10.1371/journal.pone.0005472. PMC 2675054. PMID 19424496., see table
13. Tian, Chao; et al. (November 2009). "European Population Genetic Substructure: Further Definition of Ancestry Informative Markers for Distinguishing among Diverse European Ethnic Groups". Molecular Medicine. 15 (11–12): 371–383. doi:10.2119/molmed.2009.00094. ISSN 1076-1551. PMC 2730349. PMID 19707526., see table
14. Crawford, Nicholas G. (2010). "smogd: software for the measurement of genetic diversity". Molecular Ecology Resources. 10 (3): 556–557. doi:10.1111/j.1755-0998.2009.02801.x. PMID 21565057.
15. Kitada S, Kitakado T, Kishino H (2007). "Empirical Bayes inference of pairwise F(ST) and its distribution in the genome". Genetics. 177 (2): 861–73. doi:10.1534/genetics.107.077263. PMC 2034649. PMID 17660541.

Further reading

• Evolution and the Genetics of Populations Volume 2: the Theory of Gene Frequencies, pg 294–295, S. Wright, Univ. of Chicago Press, Chicago, 1969
• A haplotype map of the human genome, The International HapMap Consortium, Nature 2005

External links

• BioPerl - Bio::PopGen::PopStats