How does Netflix ensure the data quality for thousands of Apache Iceberg tables?

Netflix employs the Write-Audit-Publish (WAP) pattern using Apache Iceberg to maintain high data quality across thousands of tables. The WAP pattern involves writing data to a hidden snapshot, auditing it, and publishing it only if it passes quality checks. This approach is analogous to CI/CD workflows, ensuring validated data is exposed to downstream consumers. Apache Iceberg's structure, including manifest files, metadata files, and catalog, supports efficient snapshot management and branching, similar to version control in Git.