Gardner-Stephen, Paul Mark

Knowles, Gregory Percy

In this paper we present our genomic and proteomic se-quence alignment algorithm, DASH, which results in order of magnitude speed improvement when compared to NCBI-BLAST 2.2.6[1], with superior sensitivity. Dynamic programming (DP) is the predominant con-tributor to search time for algorithms such as BLAST and FastA/P[2]. Improving the efficiency of DP provides an opportunity to increase sensitivity, or significantly reduce search times and help offset the effects of the continuing ex-ponential growth in database sizes. Specifically, for nucleotide searching we have demon-strated an order of magnitude speed improvement with sig-nificantly improved sensitivity, or alternatively moderate speed up with further sensitivity gains, depending on the pa-rameters selected. Smith-Waterman[3] complete DP is used as the sensitivity benchmark. Similar speed and sensitivity results are presented for protein searching. Since our algorithm is highly parallel, we have devel-oped dedicated hardware which we will present in a com-panion paper[4], and a distributed version of our software (DDASH), which we expect to provide linear speedup on a cluster. 1

Paul Gardner-stephen

Greg Knowles

CiteSeerX

DASH: Localising dynamic programming for order of magnitude faster, accurate sequence alignment

Flinders Academic Commons

DASH: Localising Dynamic Programming for Order of Magnitude Faster,Accurate Sequence AlignmentPaul Gardner-Stephen, Greg Knowles,Embedded Systems Laboratory,School of Informatics & Engineering, Flinders University,GPO BOX 2100, Adelaide 5001, AUSTRALIAgardners@infoeng.ﬂinders.edu.au gknowles@infoeng.ﬂinders.edu.auAbstractIn this paper we present our genomic and proteomic se-quence alignment algorithm, DASH, which results in orderof magnitude speed improvement when compared to NCBI-BLAST 2.2.6[1], with superior sensitivity.Dynamic programming (DP) is the predominant con-tributor to search time for algorithms such as BLAST andFastA/P[2]. Improving the efﬁciency of DP provides anopportunity to increase sensitivity, or signiﬁcantly reducesearch times and help offset the effects of the continuing ex-ponential growth in database sizes.Speciﬁcally, for nucleotide searching we have demon-strated an order of magnitude speed improvement with sig-niﬁcantly improved sensitivity, or alternatively moderatespeed up with further sensitivity gains, depending on the pa-rameters selected. Smith-Waterman[3] complete DP is usedas the sensitivity benchmark. Similar speed and sensitivityresults are presented for protein searching.Since our algorithm is highly parallel, we have devel-oped dedicated hardware which we will present in a com-panion paper[4], and a distributed version of our software(DDASH), which we expect to provide linear speedup on acluster.1. IntroductionMuch has been published regarding genomic and pro-teomic searching. An over-simpliﬁed summary could bethat a number of algorithm developments, such as FastA/P,NCBI-BLAST (BLAST) versions 1[5] and 2, FLASH[6]and CAFE[7], have progressively resulted in speed im-provements of around three to four orders of magnitude overa complete search algorithm, such as Smith-Waterman. Ingeneral, each successive speed improvement has been at theexpense of sensitivity. This process has been largely drivenby database size (and hence search time) increasing fasterthan Moore’s Law (which relates to the rate of increase incomputational power over time). This successive replace-ment of search algorithms with faster and less sensitive oneshas serious consequences for biologists. It is recognised thatrelatively few biologists have a good understanding of whatis likely to be omitted by a particular search algorithm [8].1.1. The DASH AlgorithmWe have developed the Diagonal Aggregating SearchHeuristic (DASH) algorithm[9] with this dilemma in mind.This has culminated in a three stage algorithm. First, non-gapped alignments (diagonals) are identiﬁed with the aidof an indexed version of the database. Second, these diag-onals are aggregated where possible by performing globalDP alignment of the regions between them. Finally, DP isperformed at each end of the aggregated diagonals. Thisprocess, and the regions of DP are illustrated in ﬁgure 1,where the products of each stage are appropriately num-bered. The reduction in DP load can clearly be seen in thatthe bands around the non-gapped alignments are not evalu-ated.Query sequence ﬁltering in DASH is performed by ex-cluding from the ﬁrst stage those k-tuples which occur ex-cessively often. This frequency information is stored in,and pertains to each division of the database. This resultsin a localised ﬁltering algorithm, which takes into accountlocalised sequence frequency aberrations in the database,without penalising the sensitivity of the entire search. A sec-ond effect is to allow extension of alignments over excludedregions, since neither the query or subject sequences are al-tered. Hence DP is able to cover any sequence segmentswhich have been ﬁltered.To reduce the incidence of outrageously scored low com-plexity sequences, DASH scores nucleotide alignments in-volving approximate matches (e.g. N vs G) differentially toexact matches (e.g. G vs G).Proceedings of the 2004 IEEE Computational Systems Bioinformatics Conference (CSB 2004) 0-7695-2194-0/04 $20.00 © 2004 IEEE1112233Figure 1. Illustration of DASH algorithm and Dy-namic Programming ZonesProgram BProgram A Reference132574612345671234678Figure 2. Example of User Satisfaction SensitivityMeasure1.2. Measuring Sensitivity From a User’s Perspec-tiveWe have developed a sensitivity metric which is designedto mimic the satisfaction of biologists actually using anon-line search facility. We consider the optimal alignmentsreturned by Smith-Waterman as our benchmark. Only themost signiﬁcant alignment against any given sequence ofthe database is considered, since that determines the rela-tive rank of the alignment in the results. From the top, itis tested whether they occur perfectly in the results of theprogram being scrutinised. As soon as an omission, or par-tial omission is identiﬁed, testing stops. The rank reachedis recorded as the sensitivity score. Thus we measure therank above which we trust that the algorithm has not omit-ted an alignment. This is illustrated in ﬁgure 2. ProgramA omits part of the fourth alignment of the reference re-sults, and thus scores three. Program B omits the entirety ofthe ﬁfth alignment, and so scores four. The order of align-ments is not considered.We also use an alternative metric, to demonstrate that theperformance of DASH is not dependent on a single mea-sure. Here the benchmark is the twenty ﬁve most statisti-cally signiﬁcant alignments obtained by Smith-Waterman.We measure the percentage coverage by each programagainst this benchmark. In this way, we assess both com-plete and partial matches.2. ResultsTo verify the DASH algorithm, we performed a compar-ison of both speed and sensitivity, using Smith-Watermanas a benchmark. Two hundred random complete sequenceswere selected from a draft of the Human Genome[10] fornucleotide, and the same number from genpept[11] for pro-tein. The Smith-Waterman results were obtained using theseqaln[12] program, our DDASH distribution framework,and approximately 300 CPU days of the APAC and SAPACsuper computer facilities. BLAST 2.2.6 results were alsoobtained for comparison. In all cases, the databases wereloaded completely into RAM.It is clear that for both protein and nucleotide searchingDASH is capable of searching an order of magnitude fasterthan BLAST, while actually improving sensitivity (table 1and ﬁgures 3, 4, 5 and 6, modes 1 and 2). For the DASHmodes which approach the run time of BLAST (i.e. modes3 and 4), even greater improvements in sensitivity are ob-served. Figure 7 shows the sensitivity of BLAST and DASHusing the alternative metric, demonstrating that the perfor-mance of DASH is not dependent on a single measure.BLAST frequently misses alignments as soon as theﬁrst or second rank, particularly for certain classes of nu-cleotide searches. This is in part due to the query sequenceﬁltering employed by BLAST in its default mode of op-eration. When the ﬁltering excludes signiﬁcant portionsof the query sequence BLAST tends to fragment align-ments which would cover those regions. In light of this, wealso considered the 130 cases where BLAST did not ﬁlter.BLAST then achieves considerably higher scores (medianof 10 versus 2, and mean 19.17 versus 12.11).DASH in contrast, pushes the median scores beyond thetwentieth to twenty-ﬁfth ranks. This represents a markedimprovement over existing sequence ﬁltering algorithms,such as [13] - [17].Finally, ﬁgure 8 reveals that the speedup of DASH overBLAST depends on the query sequence length. Investiga-tion reveals that this is related to a large static time compo-nent (around 7 seconds for nucleotide). It is our understand-ing that a large fraction of searches submitted fall into thesub 500 residue size range, suggesting that typical speedup in a production environment will be much more thanten times. A similar relationship exists for protein, how-ever with greater variability due to the increased DP loadinvolved.Proceedings of the 2004 IEEE Computational Systems Bioinformatics Conference (CSB 2004) 0-7695-2194-0/04 $20.00 © 2004 IEEEProgram Min Med. Mean Max.BLAST-N 7.48 9.40 9.79 21.19DASH-N 1 0.20 0.82 0.93 4.95BLAST-P 0.01 21.76 25.23 138.70DASH-P 1 0.01 1.99 3.22 23.93Table 1. Summary Statistics for DASH mode 1 andBLAST Run Time (seconds)Blast 2.2.6 mode 1 mode 2 mode 3 mode 4DASH−P ModeRun time in seconds0.5252010020050101Figure 3. Speed Comparison for ProteinBlastN 2.2.6 mode 1 mode 2 mode 3 mode 4DASH−N ModeRun time in seconds0.10.5105010051Figure 4. Speed Comparison for NucleotideBlastP 2.2.6 mode 1 mode 2 mode 3 mode 4DASH−P ModeRank of First Search Omission020406080100Figure 5. Sensitivity Comparison for ProteinBlastN 2.2.6 mode 1 mode 2 mode 3 mode 4Rank of First Search Omission020406080100DASH−N ModeFigure 6. Sensitivity Comparison for NucleotideBlastN 2.2.6 mode 1 mode 2 mode 3 mode 4DASH−N Mode20406080100Figure 7. % Coverage of Top 25 Smith-WatermanHits for NucleotideProceedings of the 2004 IEEE Computational Systems Bioinformatics Conference (CSB 2004) 0-7695-2194-0/04 $20.00 © 2004 IEEE500 1000 1500 2000 2500 3000 3500010203040Query Sequence LengthSpeed−up ofDashN over BlastNFigure 8. Query Sequence Length Versus Speedupof DASH-N over BLAST-N3. Conclusions & Future DirectionsIn the light of these results, we conclude that DASH isfaster than BLAST by an order of magnitude. DASH simul-taneously offers greatly improved sensitivity for nucleotideand, signiﬁcantly improved sensitivity for protein search-ing. The localised query ﬁltering regime does not sufferfrom the problems introduced by existing popular ﬁlteringalgorithms.We also plan to continue work on our distributed version(DDASH). This environment has been designed to utiliseheterogeneous computing nodes connected by commoditynetworks. This applies to distributed networks of UNIX,Linux, Windows, MacOS-X as well as dedicated clusters.We plan continue to reﬁning our already light weight com-munication protocol, to further reduce latency induced over-heads. We expect to obtain near linear speed up with num-ber of processors. This is an area for future research.4. AcknowledgementsWe wish to acknowledge the support of the AustralianCo-Operative Research Centre for Sensor, Signal and Infor-mation Processing (CSSIP) Firmware for Genomics project,and the Australian and South Australian Partnerships forAdvanced Computing (APAC & SAPAC).References[1] S. F. Altschol et al, “Gapped BLAST and PSI-BLAST: anew generation of protein database search programs,” Nu-cleic Acids Res, vol. 25, pp. 3389-3402, 1997.[2] W. R. Pearson, “Rapid and sensitive sequence comparisonwith FASTP and FASTA,” Methods in Enzymology, vol. 183,pp. 63-98, 1990.[3] T.F. Smith and M. S. Waterman, “Identiﬁcation of commonmolecular subsequences,” The Journal of Molecular Biology,No. 147(1), pp. 195-197, March 1981.[4] G. Knowles, and P. M. Gardner-Stephen, “A new hard-ware architecture for genomic and proteomic sequencealignments,” presented at 3rd IEEE Computational SystemsBioinformatics Conference, Stanford, USA, 2004.[5] S. F. Altschul, W. Gish, W. Miller, E. W. Myers, D. J. andLipman, “Basic local alignment search tool” The Journal ofMolecular Biology, vol. 215, pp. 403-410, 1990.[6] A. Califano and I. Rigoutsos, “FLASH: A fast look-up algo-rithm for string homology,” in International Conference onIntelligent Systems for Molecular Biology, 1993, pp. 56-64.[7] H. E. Williams J. Zobel, “Indexing and retrieval for genomicdatabases,” IEEE Transactions on Knowledge and Data En-gineering archive, vol. 14, Issue 1, pp. 63 - 78, January 2002.[8] F. Galisson, “The FASTA and BLAST programs,” July 2000,http://bioweb.pasteur.fr/seqanal/blast/blast_fasta-uk.ps[9] P. M. Gardner-Stephen and G. Knowles, "DASH: A NewHigh Speed Genomic Search and Alignment Tool," in 4thInternational Conference on Mathematics and Computers inBiology and Chemistry, 2003, vol. 1, pp. 121-127.[10] National Centre for Biotechnology Information,“Human Genome, Working Draft,” June 2002,ftp://ftp.ncbi.nih.gov/repository/UniGene/Hs.seq.all.gz.[11] National Centre for Biotechnology Informa-tion, “Protein translations of GenBank CDEs,”ftp://www.ncbi.nih.gov/.../genpept.fsa.[12] P. Hardy and M. Waterman, “The Sequence Align-ment Software Library at USC,” 1997, http://www-hto.usc.edu/software/seqaln/.[13] H. E. Williams, “Effective Query Filtering For Fast Homol-ogy Searching," in 4th Paciﬁc Symposium on Biocomputing,1999, pp. 214-225.[14] J. C. Wootton and S. Federhen, “Statistics of local complex-ity in amino acid sequences and sequence databases,” Com-puters in Chemistry, vol. 17, pp. 149-163, 1993.[15] J. C. Wootton and S. Federhen, “Analysis of composition-ally biased regions in sequence databases,” Methods in En-zymology, vol. 266, pp. 554-571, 1996.[16] J. M. Hancock and J. S. Armstrong, “SIMPLE34: an im-proved and enhanced implementation for VAX and Sun com-puters of the SIMPLE algorithm for analysis of clusteredrepetitive motifs in nucleotide sequences,” Comput ApplBiosci, vol. 10, pp. 67-70, 1994.[17] J-M. Claverie and D. J. States, “Information enhancementmethods in large scale sequence analysis,” Computers inChemistry, vol. 17, pp. 191-201, 1993.Proceedings of the 2004 IEEE Computational Systems Bioinformatics Conference (CSB 2004) 0-7695-2194-0/04 $20.00 © 2004 IEEE

DASH: localising dynamic programming for order of
            magnitude faster, accurate sequence alignment

http://dspace.flinders.edu.au/xmlui/bitstream/handle/2328/9786/2004053825.pdf?sequence=1

DASH: localising dynamic programming for order of magnitude faster, accurate sequence alignment

Abstract

Similar works

Full text

Available Versions

CiteSeerX

Flinders Academic Commons