Data structures and compression algorithms for high-throughput sequencing technologies

<p>Abstract</p> <p>Background</p> <p>High-throughput sequencing (HTS) technologies play important roles in the life sciences by allowing the rapid parallel sequencing of very large numbers of relatively short nucleotide sequences, in applications ranging from genome sequencing and resequencing to digital microarrays and ChIP-Seq experiments. As experiments scale up, HTS technologies create new bioinformatics challenges for the storage and sharing of HTS data.</p> <p>Results</p> <p>We develop data structures and compression algorithms for HTS data. A processing stage maps short sequences to a reference genome or a large table of sequences. Then the integers representing the short sequence absolute or relative addresses, their length, and the substitutions they may contain are compressed and stored using various entropy coding algorithms, including both old and new fixed codes (e.g Golomb, Elias Gamma, MOV) and variable codes (e.g. Huffman). The general methodology is illustrated and applied to several HTS data sets. Results show that the information contained in HTS files can be compressed by a factor of 10 or more, depending on the statistical properties of the data sets and various other choices and constraints. Our algorithms fair well against general purpose compression programs such as gzip, bzip2 and 7zip; timing results show that our algorithms are consistently faster than the best general purpose compression programs.</p> <p>Conclusions</p> <p>It is not likely that exactly one encoding strategy will be optimal for all types of HTS data. Different experimental conditions are going to generate various data distributions whereby one encoding strategy can be more effective than another. We have implemented some of our encoding algorithms into the software package GenCompress which is available upon request from the authors. With the advent of HTS technology and increasingly new experimental protocols for using the technology, sequence databases are expected to continue rising in size. The methodology we have proposed is general, and these advanced compression techniques should allow researchers to manage and share their HTS data in a more timely fashion.</p

The political economy of natural disaster damage

Economic damage from natural hazards can sometimes be prevented and always mitigated. However, private individuals tend to underinvest in such measures due to problems of collective action, information asymmetry and myopic behavior. Governments, which can in principle correct these market failures, themselves face incentives to underinvest in costly disaster prevention policies and damage mitigation regulations. Yet, disaster damage varies greatly across countries. We argue that rational actors will invest more in trying to prevent and mitigate damage the larger a country's propensity to experience frequent and strong natural hazards. Accordingly, economic loss from an actually occurring disaster will be smaller the larger a country's disaster propensity – holding everything else equal, such as hazard magnitude, the country's total wealth and per capita income. At the same time, damage is not entirely preventable and smaller losses tend to be random. Disaster propensity will therefore have a larger marginal effect on larger predicted damages than on smaller ones. We employ quantile regression analysis in a global sample to test these predictions, focusing on the three disaster types causing the vast majority of damage worldwide: earthquakes, floods and tropical cyclones

Self or other: Directors’ attitudes towards policy initiatives for external board evaluation

Recurrent crises in corporate governance have board practice and created policy pressure to assess the effectiveness of boards. Since the 1990s boards have faced calls to undertake regular, formal evaluation. Since 2010, the UK Corporate Governance Code has urged large corporations to engage outside parties to conduct them at least every three years, a move that other jurisdictions have copied. Despite this policy importance, little research has been conducted into processes or outcomes of board evaluation. This study explores the attitudes of directors on evaluation, whether self-administered or facilitated by others. We find acceptance of the principle but reservations about the value and even honesty in questionnaire-based approaches. We find scepticism about, but also acknowledgement of, the benefits of using outside facilitators, especially for their objectivity and because their interviewing elicits insights into board dynamics. As this practice expands beyond listed companies to non-listed ones, charities, and even governance branches of government, our findings point to a need to professionalise outside facilitation

Data structures and compression algorithms for high-throughput sequencing technologies

BackgroundHigh-throughput sequencing (HTS) technologies play important roles in the life sciences by allowing the rapid parallel sequencing of very large numbers of relatively short nucleotide sequences, in applications ranging from genome sequencing and resequencing to digital microarrays and ChIP-Seq experiments. As experiments scale up, HTS technologies create new bioinformatics challenges for the storage and sharing of HTS data.ResultsWe develop data structures and compression algorithms for HTS data. A processing stage maps short sequences to a reference genome or a large table of sequences. Then the integers representing the short sequence absolute or relative addresses, their length, and the substitutions they may contain are compressed and stored using various entropy coding algorithms, including both old and new fixed codes (e.g Golomb, Elias Gamma, MOV) and variable codes (e.g. Huffman). The general methodology is illustrated and applied to several HTS data sets. Results show that the information contained in HTS files can be compressed by a factor of 10 or more, depending on the statistical properties of the data sets and various other choices and constraints. Our algorithms fair well against general purpose compression programs such as gzip, bzip2 and 7zip; timing results show that our algorithms are consistently faster than the best general purpose compression programs.ConclusionsIt is not likely that exactly one encoding strategy will be optimal for all types of HTS data. Different experimental conditions are going to generate various data distributions whereby one encoding strategy can be more effective than another. We have implemented some of our encoding algorithms into the software package GenCompress which is available upon request from the authors. With the advent of HTS technology and increasingly new experimental protocols for using the technology, sequence databases are expected to continue rising in size. The methodology we have proposed is general, and these advanced compression techniques should allow researchers to manage and share their HTS data in a more timely fashion