Learnings from optimising 22 of our most expensive Snowflake pipelines

A team at Medium worked to reduce Snowflake costs by optimizing 22 expensive pipelines, implementing 56 changes. They focused on pragmatic solutions like eliminating unused services, reducing unnecessary data columns, adjusting pipeline schedules, improving query efficiency through explicit filtering and simpler predicates, and using time windows to avoid exponential data growth. They also modularized code for reuse and adjusted warehouse sizes for cost efficiency. Future plans include monitoring cost-efficiency tools and experimenting with advanced Snowflake features like Snowpipe Streaming.