I spent 4 hours learning Apache Spark Resource Allocation

An overview of Apache Spark's resource allocation mechanisms and scheduling modes. It covers static and dynamic resource allocation, highlighting how dynamic allocation uses heuristics for acquiring and removing executors. It also compares FIFO and fair scheduling, explaining how the latter ensures equal resource sharing among jobs. Additionally, considerations for gracefully decommissioning executors and the usage of an external shuffle service are discussed.