A step-by-step guide to building a lakehouse architecture using Snowflake Postgres, the pg_lake PostgreSQL extension, and Apache Iceberg tables stored on AWS S3. The setup separates OLTP workloads (handled by Snowflake Postgres) from OLAP analytics (handled by Snowflake), using Iceberg as the open format bridge. The tutorial covers spinning up a Snowflake Postgres instance, configuring AWS IAM roles and storage integrations, creating Iceberg tables via pg_lake, registering them in Snowflake using external volumes and catalog integrations, and automating metadata refresh with Snowflake Tasks and Streams. A Python simulation script demonstrates bulk-inserting retail transactions and syncing them to Iceberg tables.

18m read timeFrom medium.com
Post cover image
Table of contents
Get Reza Brianca’s stories in your inboxSetup in SnowflakeSimulate the TransactionClosing

Sort: