An analysis of NYC subway arrival data collected via the MTA real-time API over several weeks. The author explores the distribution of train intervals, waiting times by subway line and time of day, and the 'sunk cost' question of when to give up waiting. Key finding: after about 5 minutes of waiting during daytime hours, the longer you've waited, the longer you'll likely still have to wait. The analysis connects to probability concepts like memorylessness and fat-tailed distributions, and suggests an optimal give-up threshold of around 11 minutes for a 90th-percentile 30-minute budget.
Table of contents
Let’s do some cool stuff with this data!Erik please digress and talk about the relationship between the two curvesWaiting time by lineWaiting time by time of dayWaiting for subway and sunk costSort: