The post discusses migrating a task scheduling system from a simple Go service using SQLite to a more complex SaaS platform. The new challenges of dynamic scheduling and concurrent execution were solved using Redis's sorted set for task prioritization and Postgres for handling distributed writes. Evaluations of various scheduling libraries indicated rq_scheduler in Python as the most suitable. The result was an efficient architecture capable of scalable and reliable task scheduling and execution.
Table of contents
Challenges with replicating the previous setup:How did we arrive at the solution?Implementation:Sort: