Optimizing a Go service that streams data from Postgres replication slots to Elasticsearch requires careful memory management and performance tuning. Key techniques include switching from encoding/json to jsoniter for faster serialization, using sync.Pool to reduce allocations in hot paths, and tuning garbage collection behavior. The challenge lies in balancing three competing forces: Elasticsearch backpressure, continuous database changes, and Go's memory allocator. By controlling allocations, optimizing JSON handling, and carefully tuning GC settings, services can maintain stable latency and memory usage under sustained high volume while preventing Postgres WAL buildup.
5 Comments
Sort: