Snowflake is announcing improved Task-based orchestration for containerized Jobs, targeting ML Jobs and Snowpark Container Services customers. The update enables native scheduling and event-triggering of containerized runs, chaining them into DAG-based pipelines without external schedulers. Key improvements include serverless Task execution (no Warehouse needed for long-running GPU workloads), task context propagation via Python APIs, ML Job definitions inside DAGTask(), and enhanced Snowsight observability with visual graph views, run history, and drill-down into SPCS telemetry.

2m read timeFrom medium.com
Post cover image

Sort: