Meta released OpenZL, an open source lossless compression framework for structured data that achieves format-specific compression performance while maintaining a single universal decompressor. The framework applies configurable transformation sequences based on data structure descriptions, uses an offline trainer to optimize compression plans, and supports runtime adaptation without requiring decoder updates. OpenZL demonstrates significant improvements over general-purpose compressors like Zstandard and XZ on structured datasets, offering better compression ratios while maintaining or improving speed. The system includes a Simple Data Description Language (SDDL) for defining data shapes and integrates with Meta's Managed Compression infrastructure for automated retraining as data evolves.
Table of contents
A Decade of LessonsMake the Structure ExplicitAn Example Compression Using OpenZLGenerate a Compressor AutomaticallyEmbracing Changes: Re-Training and In-Flight ControlThe Advantages of the Universal DecoderResults With OpenZLGetting Started With OpenZLWhere We’re GoingSort: