Best of Storage — February 2025

1. 1
   Article

   ByteByteGo·1y

   How Amazon S3 Stores 350 Trillion Objects with 11 Nines of Durability

   Amazon S3 is a highly scalable and durable object storage service, launched in 2006, which has revolutionized cloud storage with 99.999999999% durability. Initially reactive, S3 has evolved to predictive scaling and threat modeling. It uses a complex microservices architecture to manage over 350 trillion objects, ensuring high availability and security. Key features include multiple storage classes, automatic scalability, encryption, and integration with other AWS services. Notable milestones include the introduction of regional storage, Glacier, and intelligent-tiering. S3's indexing and partitioning strategies enable rapid data retrieval and fault tolerance.

   66
2. 2
   Article

   Hacker News·1y

   deepseek-ai/3FS: A high-performance distributed file system designed to address the challenges of AI training and inference workloads.

   The Fire-Flyer File System (3FS) is a high-performance distributed file system tailored for AI training and inference workloads. It utilizes modern SSDs and RDMA networks to offer a shared storage layer that simplifies the development of distributed applications. Key features include strong consistency, stateless metadata services, efficient data preparation, random access to training samples, high-throughput checkpointing, and cost-effective KVCache for inference. Performance tests show impressive read throughput and sorting efficiency, making 3FS an optimal choice for large-scale AI tasks.

   57
3. 3
   Article

   Watercooler·1y

   Uh oh

   13

   10