Scalable, Real-Time, and Information-Rich Sequence Search Over SRA
Compact, exact, and incrementally updatable indexes for searching petabyte-scale sequencing data.
The Sequence Read Archive (SRA) holds petabytes of raw sequencing data, but searching it for a given sequence has long been impractical at scale. This project builds compact, exact, and information-rich indexes that make large-scale sequence search fast and space-efficient.
We introduced Squeakr (Pandey et al., 2017) for fast and compact $k$-mer counting, and the Mantis system (Pandey et al., 2018; Almodaresi et al., 2019) for exact, large-scale sequence search. These build on succinct colored de Bruijn graph representations such as deBGR (Pandey et al., 2017) and Rainbowfish (Almodaresi et al., 2017). We made the search index incrementally updatable (Almodaresi et al., 2022), extended the approach to genomic variant search with VariantStore (Pandey et al., 2021), and most recently scaled colored compacted de Bruijn graph construction to massive datasets with Cuttlefish 3 (Khan et al., 2026). We continue to push toward faster, distributed, and noise-robust search: accelerating streaming $k$-mer operations with a dynamic, concurrent, cache-efficient hash table (Khan et al., 2026), balancing load across distributed genomic indices (Galbraith et al., 2026), and enabling noise-robust DNA search at SRA scale via local-alignment embeddings with LOCALE (Synk et al., 2026).
References
2026
- bioRxiv 2026kache-hash: A Dynamic, Concurrent, and Cache-Efficient Hash Table for Streaming k-mer OperationsbioRxiv, 2026
- BioDMS 2026Similarity-Based Load-Balancing for Distributed Genomic IndicesIn VLDB 2026 Workshop on Biomedical Data Management Systems (BioDMS), 2026
- VecDB 2026LOCALE: Local-Alignment Embeddings for Noise-Robust DNA Search at SRA ScaleIn VLDB 2026 Workshop: The 2nd Workshop on Vector Databases (VecDB), 2026
2022
2021
2019
- RECOMB 2019, JCB 2020