After writing about the Apache Iceberg file format overview in this article, I decided to spend more time understanding its internals. This article includes all the lessons I learned after hours of…

Data Engineer Things

This post delves into the internals of the Apache Iceberg file format, covering its data and metadata layers, manifest files, and how it manages read and write operations. It includes details on compaction, hidden partitioning, sorting, and row-level updates with both copy-on-write and merge-on-read modes. The goal is to offer a comprehensive understanding of Iceberg's capabilities and optimizations for managing large datasets efficiently.

I spent 7 hours diving deep into Apache Iceberg

<p>I’ve aggregated a large list of Apache Iceberg tutorials and resources for Apache Iceberg:</p>
<p><a href="https://datalakehousehub.com/blog/2024-10-ultimate-directory-of-apache-iceberg-resources/" target="_blank" rel="noopener nofollow">Iceberg Resource List</a></p>
