Snowflake's Error Logging feature, now integrated with the Snowpipe Streaming high-performance architecture, provides a SQL-queryable dead-letter queue built directly into Snowflake. When enabled via a single table setting (ERROR_LOGGING = TRUE), every row that fails ingestion is automatically captured in a dedicated error table with the original payload and detailed metadata. The post covers how server-side processing enables systematic error capture, explains the error table schema and streaming-specific metadata fields, and walks through a complete hands-on workflow: enabling error logging, querying failures, fixing and re-inserting bad rows, and automating the entire process using Snowflake streams, tasks, and alerts. Cost impact is minimal — only standard storage charges apply for error table data, retained up to 14 days.

9m read timeFrom medium.com
Post cover image
Table of contents
IntroductionSnowpipe Streaming High-Performance ArchitectureError Logging: What and WhyHands-On WalkthroughGet Andrey Zagrebin’s stories in your inboxWhat about cost?ConclusionGet Started TodayLearn More

Sort: