How we contributed two features to Debezium to solve WAL growth and enable safer logical replication for our event streaming infrastructure

Zalando Tech Blog is a platform where Zalando's engineering team shares insights, experiences, and best practices on technology and innovation. Covering topics such as software architecture, data science, and machine learning, Zalando's Tech Blog offers resources for developers and tech enthusiasts. Developers can learn about Zalando's engineering culture, projects, and technology stack through their blog posts and articles.

Zalando

Zalando contributed two features to Debezium to solve WAL growth issues in PostgreSQL logical replication at scale. The first contribution introduced an opt-in `lsn.flush.mode` configuration to prevent WAL accumulation on low-activity databases. The second added `offset.mismatch.strategy` to handle conflicts between stored offsets and replication slot positions, enabling safer operation with durable offset stores. These features allow users to trust the replication slot position when appropriate, preventing unnecessary full re-syncs and enabling recovery from corrupted WAL segments.

Contributing to Debezium: Fixing Logical Replication at Scale

The Real Problem: When the Slot and Offset Disagree