Space Efficient Sequence Alignment for SRAM-Based Computing: X-Drop on the Graphcore IPU

Dedicated accelerator hardware has become essential for processing AI-based workloads, leading to the rise of novel accelerator architectures. Furthermore, fundamental differences in memory architecture and parallelism have made these accelerators targets for scientific computing. The sequence alignment problem is fundamental in bioinformatics; we have implemented the $X$-Drop algorithm, a heuristic method for pairwise alignment that reduces search space, on the Graphcore Intelligence Processor Unit (IPU) accelerator. The $X$-Drop algorithm has an irregular computational pattern, which makes it difficult to accelerate due to load balancing. Here, we introduce a graph-based partitioning and queue-based batch system to improve load balancing. Our implementation achieves $10\times$ speedup over a state-of-the-art GPU implementation and up to $4.65\times$ compared to CPU. In addition, we introduce a memory-restricted $X$-Drop algorithm that reduces memory footprint by $55\times$ and efficiently uses the IPU's limited low-latency SRAM. This optimization further improves the strong scaling performance by $3.6\times$.Comment: 12 pages, 7 figures, 2 table

bihealth/mehari: v0.14.2

<h2><a href="https://github.com/bihealth/mehari/compare/v0.14.1...v0.14.2">0.14.2</a> (2023-10-21)</h2> <h3>Bug Fixes</h3> <ul> <li>make distance to next exon correct (<a href="https://github.com/bihealth/mehari/issues/222">#222</a>) (<a href="https://github.com/bihealth/mehari/issues/223">#223</a>) (<a href="https://github.com/bihealth/mehari/commit/cad307fe9c2c150d254d471f508601dda01df363">cad307f</a>)</li> <li>switching from unmaintained tempdir to tempfile (<a href="https://github.com/bihealth/mehari/issues/227">#227</a>) (<a href="https://github.com/bihealth/mehari/commit/a7b0e08f69b5a3527f78119bc0cf220ddca64597">a7b0e08</a>)</li> </ul&gt

bihealth/mehari: v0.15.0

<h2><a href="https://github.com/bihealth/mehari/compare/v0.14.3...v0.15.0">0.15.0</a> (2023-10-23)</h2> <h3>Features</h3> <ul> <li>adding async I/O, espec. for SV caller guessing (<a href="https://github.com/bihealth/mehari/issues/230">#230</a>) (<a href="https://github.com/bihealth/mehari/commit/33c0e8efec783d3f3d504cd0619ab4ac234ce4d6">33c0e8e</a>)</li> </ul&gt

bihealth/mehari: v0.18.0

<h2><a href="https://github.com/bihealth/mehari/compare/v0.17.1...v0.18.0">0.18.0</a> (2023-11-19)</h2> <h3>Features</h3> <ul> <li>properly handle selenoprotein import from cdot (<a href="https://github.com/bihealth/mehari/issues/224">#224</a>) (<a href="https://github.com/bihealth/mehari/issues/265">#265</a>) (<a href="https://github.com/bihealth/mehari/commit/20137adff0d9f575885c3ee1f6691353a3d6d5b2">20137ad</a>)</li> </ul&gt

bihealth/mehari: v0.14.3

<h2><a href="https://github.com/bihealth/mehari/compare/v0.14.2...v0.14.3">0.14.3</a> (2023-10-22)</h2> <h3>Bug Fixes</h3> <ul> <li>index out of bounds error in stop_retained variant (<a href="https://github.com/bihealth/mehari/issues/226">#226</a>) (<a href="https://github.com/bihealth/mehari/issues/228">#228</a>) (<a href="https://github.com/bihealth/mehari/commit/cc8d4405f1c96c095fc7fa7d2e6ca08c3386563a">cc8d440</a>)</li> </ul&gt

bihealth/mehari: v0.15.2

<h2><a href="https://github.com/bihealth/mehari/compare/v0.15.1...v0.15.2">0.15.2</a> (2023-10-28)</h2> <h3>Bug Fixes</h3> <ul> <li>remove second async runtime creation (<a href="https://github.com/bihealth/mehari/issues/234">#234</a>) (<a href="https://github.com/bihealth/mehari/commit/a6c3e063242d9fc6abbd4887d3902c03ad821776">a6c3e06</a>)</li> </ul&gt

bihealth/mehari: v0.18.1

<h2><a href="https://github.com/bihealth/mehari/compare/v0.18.0...v0.18.1">0.18.1</a> (2023-11-19)</h2> <h3>Miscellaneous Chores</h3> <ul> <li>update dependencies (<a href="https://github.com/bihealth/mehari/issues/268">#268</a>) (<a href="https://github.com/bihealth/mehari/commit/a3c379ea80d6931aa77ddc93941b1c8a042ff120">a3c379e</a>)</li> </ul&gt

bihealth/mehari: v0.17.1

<h2><a href="https://github.com/bihealth/mehari/compare/v0.17.0...v0.17.1">0.17.1</a> (2023-11-15)</h2> <h3>Bug Fixes</h3> <ul> <li>fixing implementation of Provider::get_tx_for_gene (<a href="https://github.com/bihealth/mehari/issues/262">#262</a>) (<a href="https://github.com/bihealth/mehari/commit/42945dd3d08d4921e7ea5ab00525b07a192ad3c8">42945dd</a>)</li> </ul&gt

bihealth/mehari: v0.19.0

<h2><a href="https://github.com/bihealth/mehari/compare/v0.18.1...v0.19.0">0.19.0</a> (2023-11-21)</h2> <h3>⚠ BREAKING CHANGES</h3> <ul> <li>serializing protobufs pbjson-build (<a href="https://github.com/bihealth/mehari/issues/272">#272</a>) (#273)</li> </ul> <h3>Features</h3> <ul> <li>serializing protobufs pbjson-build (<a href="https://github.com/bihealth/mehari/issues/272">#272</a>) (<a href="https://github.com/bihealth/mehari/issues/273">#273</a>) (<a href="https://github.com/bihealth/mehari/commit/0f948f7e1b1fd45ca92f1dd7010ca36d951f88cd">0f948f7</a>)</li> </ul&gt

bihealth/mehari: v0.17.0

<h2><a href="https://github.com/bihealth/mehari/compare/v0.16.0...v0.17.0">0.17.0</a> (2023-11-10)</h2> <h3>⚠ BREAKING CHANGES</h3> <ul> <li>make --path-input-ped for seqvars annotation required (<a href="https://github.com/bihealth/mehari/issues/194">#194</a>) (#255)</li> </ul> <h3>Features</h3> <ul> <li>adding helper script to fix FreeBayes VCF (<a href="https://github.com/bihealth/mehari/issues/252">#252</a>) (<a href="https://github.com/bihealth/mehari/issues/254">#254</a>) (<a href="https://github.com/bihealth/mehari/commit/4bd5461bdbabb1124e17c735bad0c1b282fa2712">4bd5461</a>)</li> <li>adding support for ClinCNV (<a href="https://github.com/bihealth/mehari/issues/253">#253</a>) (<a href="https://github.com/bihealth/mehari/issues/257">#257</a>) (<a href="https://github.com/bihealth/mehari/commit/aba47c93f21aa8752d762e72a4635df19215f643">aba47c9</a>)</li> </ul> <h3>Bug Fixes</h3> <ul> <li>make --path-input-ped for seqvars annotation required (<a href="https://github.com/bihealth/mehari/issues/194">#194</a>) (<a href="https://github.com/bihealth/mehari/issues/255">#255</a>) (<a href="https://github.com/bihealth/mehari/commit/de832b22ec6a55429e63c365c3bb787ea2e94209">de832b2</a>)</li> </ul&gt