The IBD of the two siblings in a Nuc4 pedigree along the chromosome 1.

<p>A) is the simulated true IBD of the siblings, B), C) and D) are the inferred IBD at 30X, 15X and 2X respectively.</p

The FNR (%) of the heterozygotes in 6 pedigrees from four callers (Polymutt2, Beagle4, GATK and Polymutt) for variants with alternative allele frequencies in 4 bins in the range of [0, 0.1] at sequencing coverage of 20X.

<p>Results of different pedigrees are shown in panel A) for Sib2, B) for Sib4, C) for Sib6, D) for Nuc4, E) for Nuc6 and F) for Ext10.</p

The average Mendelian inconsistency rates of Beagle4 and GATK calls per parents-offspring trio in the Nuc4 pedigrees at sequencing coverage of 10X, 15X, 20X when GQ = 5 (panel A) or GQ = 10 (panel B) was used to filter low quality genotypes.

<p>The average Mendelian inconsistency rates of Beagle4 and GATK calls per parents-offspring trio in the Nuc4 pedigrees at sequencing coverage of 10X, 15X, 20X when GQ = 5 (panel A) or GQ = 10 (panel B) was used to filter low quality genotypes.</p

The FNR (%) in Panel A) and FDR (%) in Panel B) of heterozygous genotypes at variant sites with MAF<0.02 for Polymutt2 and Beagle4 calls for different numbers of simulated Nuc6 pedigrees at 10X coverage.

<p>The FNR (%) in Panel A) and FDR (%) in Panel B) of heterozygous genotypes at variant sites with MAF<0.02 for Polymutt2 and Beagle4 calls for different numbers of simulated Nuc6 pedigrees at 10X coverage.</p

FNR (%) vs. FDR (%) curves of the overall genotypes of Polymutt2 and Beagle4 calls when 30% (~15X, panel A, B and C) or when 15% (~7.5X, panel D, E and F) of the original data were used for genotype calling.

<p>Panel A) and D) are for all variants, and panel B) and E) are for variants with MAF<0.01 and panel C) and F) are for variants with MAF<0.05. See <i>Application to real data</i> for the details on the pedigree and the calculation of error rates.</p

The FNR (%) (panel A, B and C) and FDR (%) (panel D, E and F) of the overall genotypes in 6 pedigrees from four callers (Polymutt2, Beagle4, GATK and Polymutt).

<p>Panels A, B and C show FNR (%) for sequencing coverage of 10X, 20X and 30X, and panels D, E and F show the FDR (%) for the same set of coverage.</p

Additional file 1: Table S1. of A computational method for genotype calling in family-based sequencing data

Genotype mismatch rate of heterozygous calls and SNPs with maf <5 % (Simulation I). Table S2. Genotype discordance rate of heterozygous calls (Simulation II). Table S3. Phasing error rate (Simulation I). Table S4. Phasing error rate (Simulation II). Table S5. Mendelian error rate (Simulation I). Table S6. Genotype discordance rate of heterozygous calls (Simulation III). Table S7. Phasing error rate (Simulation III). Table S8. Mendelian error rate (Simulation III). Figure S1. Pedigree of each family in second simulation scheme. Figure S2. Genotype mismatch rate of heterozygous calls (Simulation I). C1: Trios result summarized by ranodmly selected a child to form a trio and all the other children as independent individuals for all 80 families with 100 repeats; C2: nuclear families of two offspring; C3: nuclear families with three offspring and C4: nuclear families of four offspring. (DOCX 1452 kb

Number of false positive <i>de novo</i> mutations per billion bases detected by PolyMutt of jointly modeling for sequencing at coverage 5×–40× with Phred-scaled base quality Q20 (1% error rate) without mapping error in different pedigrees structures.

<p>Number of false positive <i>de novo</i> mutations per billion bases detected by PolyMutt of jointly modeling for sequencing at coverage 5×–40× with Phred-scaled base quality Q20 (1% error rate) without mapping error in different pedigrees structures.</p

Three-generation extended pedigrees.

<p>A) is a 3-generation extended pedigree with numbers labeling the individual heterozygous genotype mismatch rates (%) at coverage of 15× with base quality of Q20 without mapping error and panel B) labels the corresponding mismatch rates for the standard approach of ignoring relatedness. Panel C) and D) display the heterozygous mismatch rates (%) when a fixed sequencing effort of 150× is allocated differently to family members: Panel C) is for the situation where the founders are allocated 30× while non-founders have 5× and in Panel D) founders and non-founders have coverage of 6× and 21× respectively.</p

Genotype mismatch rates (%) for different family structures with sequencing coverage of 5×, 15×, and 30× and input bases with Phred-scaled quality Q20 (1% error rate) or Q30 (0.1% error rate) without mapping error.

<p>The mismatch rates are shown for 4 genotype categories: all genotypes (All), homozygous reference allele (HomRef), heterozygotes (Het), and homozygous alternative allele (HomAlt).</p