A practical guide to using bigram_index in Manticore Search to accelerate phrase queries. Covers all four modes: default (no bigrams), all (every adjacent token pair indexed), first_freq (pair stored when first token is in a frequent-word list), and both_freq (pair stored only when both tokens are frequent). A 1M-document benchmark shows bigram_index='all' delivers ~2.9x QPS improvement and ~3.2x latency reduction for phrase queries, at the cost of slower indexing (~17k vs ~45k docs/sec). Includes reproducible manticore-load benchmark commands and guidance on choosing the right mode for different workloads.
Table of contents
TL;DRWhat bigram indexing actually doesImportant caveat: bigrams work at tokenization levelMode 1: Default behaviorMode 2: allMode 3: first_freqMode 4: both_freqWhich performance mode should you choose?Benchmark: does bigram indexing really speed up phrase search?Final takeawaySort: