Biological Data Compression Algorithms

Abstract

The exponential growth of biological data from genome sequencing, transcriptomics, proteomics, and imaging has created storage and transmission bottlenecks that general-purpose compression cannot adequately address, and domain-specific compression algorithms that exploit the statistical structure of biological data achieve substantially higher compression ratios, yet adoption varies across data types and algorithm categories. We evaluated 216 biological data compression programmes across centres in Estonia and France between 2018 and 2022, spanning five algorithm categories: reference-based genomic sequence compression, quality score compression and lossy encoding, alignment file compression formats, single-cell expression matrix compression, and biological image compression. A Biological Data Compression Index (BDCI) was constructed from five sub-scores -- compression ratio achieved, decompression speed, information preservation fidelity, downstream analysis compatibility, and ecosystem integration breadth -- with weights from regression against sustained institutional adoption. BDCI correlated with adoption at r = +0.84 and discriminated adopted from non-adopted algorithms with an AUC of 0.884. Alignment file compression scored highest (mean BDCI 0.826), while biological image compression trailed at 0.600. Only 35.6 percent exceeded the 0.75 threshold. Compression ratio carried the largest weight (beta = +0.278), followed by downstream compatibility (beta = +0.230).