A production case study on using Bloom filters to optimize a recommendation feed service handling 18,000 requests/second. The post covers Bloom filter mechanics (bit array, hash functions, insertion, membership queries), a full Go implementation with packed uint64 bit arrays, the math for tuning parameters m (bit array size) and k (hash count), and practical lessons from production deployment. Results showed p95 latency dropping from ~140ms to ~96ms, exact history lookups reduced by ~80%, and backend read traffic down ~65-70%. Includes guidance on hash function selection, lifecycle management, and operational monitoring signals.

19m read timeFrom infoq.com
Post cover image
Table of contents
IntroductionNaive Solution: Exact Lookup for Every CandidateWhy Exact Lookup Alone Was Not Good EnoughThe Solution: Bloom FiltersBloom Filters in PracticeImplementing a Bloom Filter in GoThe Math Behind Bloom FiltersResults SnapshotPractical Considerations & Best PracticesConclusionAppendix Dive: The Math Behind Bloom FiltersAbout the Author

Sort: