Burrows-Wheeler transform (BWT) is an invertible text transformation that,
given a text T of length n, permutes its symbols according to the
lexicographic order of suffixes of T. BWT is one of the most heavily studied
algorithms in data compression with numerous applications in indexing, sequence
analysis, and bioinformatics. Its construction is a bottleneck in many
scenarios, and settling the complexity of this task is one of the most
important unsolved problems in sequence analysis that has remained open for 25
years. Given a binary string of length n, occupying O(n/logn) machine
words, the BWT construction algorithm due to Hon et al. (SIAM J. Comput., 2009)
runs in O(n) time and O(n/logn) space. Recent advancements (Belazzougui,
STOC 2014, and Munro et al., SODA 2017) focus on removing the alphabet-size
dependency in the time complexity, but they still require Ω(n) time.
In this paper, we propose the first algorithm that breaks the O(n)-time
barrier for BWT construction. Given a binary string of length n, our
procedure builds the Burrows-Wheeler transform in O(n/logn) time and
O(n/logn) space. We complement this result with a conditional lower bound
proving that any further progress in the time complexity of BWT construction
would yield faster algorithms for the very well studied problem of counting
inversions: it would improve the state-of-the-art O(mlogm)-time
solution by Chan and P\v{a}tra\c{s}cu (SODA 2010). Our algorithm is based on a
novel concept of string synchronizing sets, which is of independent interest.
As one of the applications, we show that this technique lets us design a data
structure of the optimal size O(n/logn) that answers Longest Common
Extension queries (LCE queries) in O(1) time and, furthermore, can be
deterministically constructed in the optimal O(n/logn) time.Comment: Full version of a paper accepted to STOC 201