This post discusses a data engineer project that involves transforming data from a PostgreSQL database to an S3 datalake and then to a PostgreSQL data warehouse. The author shares their experience and methodology, as well as the tools used. They also highlight the problems they faced during the project.

12m read timeFrom medium.com
Post cover image
Table of contents
Data Engineer Project: Postgres (Database) to S3 (DataLake) to Postgres (Data Warehouse) | Prep Coffee Inventory ManagementWhy do I do thisWhat I want to seeTools and MethodologyPostgres (Database) to S3 (Datalake) to Postgres (Data Warehouse)Extract-Transform-load code1st Dag: database_to_datalake2nd Dags: datalake_to_datawarehouse

Sort: