Best of Data Science — November 2025

1. 1
   Article

   Spotify Labs·28w

   Shuffle: Making Random Feel More Human

   Spotify redesigned its shuffle feature to balance statistical randomness with user perception. While the previous implementation used pure randomization (Mersenne Twister), users complained about repetitive patterns. The new 'Fewer Repeats' system generates multiple random sequences, scores them based on listening history and recency, then selects the freshest option. This approach maintains mathematical randomness while reducing perceived repetition. Premium users now get this as default, with classic random shuffle still available as 'Standard Shuffle'.

   86

   5
2. 2
   Article

   InfluxData·27w

   Introducing the New Cloud Dedicated Admin UI

   InfluxData has released a major update to the Cloud Dedicated Admin UI, introducing live cluster observability dashboards with CPU, memory, and request rate metrics. The update includes redesigned navigation for quick access to databases, tables, and tokens, plus enhanced table schema browsing with column type filtering. Users can now monitor cluster performance across different time periods and switch between multiple accounts and clusters directly from the interface.

   42

   1
3. 3
   Article

   ByteByteGo·29w

   How Datadog Built a Custom Database to Ingest Billions of Metrics Per Second

   Datadog built Monocle, a custom time-series database in Rust, to handle billions of metrics per second. The system uses Kafka for data distribution and replication, separates metadata storage from time-series data, and employs a thread-per-core architecture with LSM-tree storage. Key optimizations include arena allocators, time-based file pruning, and cost-based query scheduling. The platform splits storage into real-time (24 hours) and long-term systems, with the real-time database handling 99% of queries. Future plans include dynamic load balancing and merging separate databases into a unified columnar format.

   37