Explore how Redpanda fits into the popular Medallion Architecture to seamlessly integrate batch and real-time analytics while preserving data quality.

Redpanda

The Medallion Architecture structures data storage into three progressive layers: Bronze (raw ingestion), Silver (cleansed and transformed), and Gold (analytics-ready). Redpanda's integration with Apache Iceberg enables seamless implementation by automatically converting row-oriented topic data into columnar Iceberg tables in object storage, eliminating traditional ETL processes. This integration preserves data lineage while supporting both batch and real-time analytics. Stream processing tools like Apache Flink can perform near real-time transformations between layers, reducing time-to-insight compared to scheduled batch jobs. The architecture leverages open file formats (Parquet, ORC) and table formats (Iceberg) for efficient storage, schema evolution, and ACID compliance across data lakehouse implementations.

Implementing the Medallion Architecture with Redpanda

The role of open table formats and file formats #

How Redpanda fits into the Medallion Architecture #