Bogoychev, Nikolay

Lopez, Adam

For many applications, the query speed of N-gram language models is a computational bottleneck. Although massively parallel hardware like GPUs offer a potential solution to this bottleneck, exploiting this hardware requires a careful rethinking of basic algorithms and data structures. We present the first language model designed for such hardware, using B-trees to maximize data parallelism and minimizememory footprint and latency. Comparedwith a single-threaded instance of KenLM (Heafield, 2011), a highly optimized CPU based language model, our GPU implementation produces identical results with a smaller memory footprint and a sixfold increase in throughput on a batch query task. When we saturate both devices, the GPU delivers nearly twice the throughput per hardware dollar even when the CPU implementation uses faster data structures

Edinburgh Research Explorer

N-gram language models for massively parallel devices

https://www.pure.ed.ac.uk/ws/files/25518236/n_gram_language_1.pdf

N-gram language models for massively parallel devices

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer