Best of Storage — 2025

1. 1
   Article

   ByteByteGo·1y

   How Amazon S3 Stores 350 Trillion Objects with 11 Nines of Durability

   Amazon S3 is a highly scalable and durable object storage service, launched in 2006, which has revolutionized cloud storage with 99.999999999% durability. Initially reactive, S3 has evolved to predictive scaling and threat modeling. It uses a complex microservices architecture to manage over 350 trillion objects, ensuring high availability and security. Key features include multiple storage classes, automatic scalability, encryption, and integration with other AWS services. Notable milestones include the introduction of regional storage, Glacier, and intelligent-tiering. S3's indexing and partitioning strategies enable rapid data retrieval and fault tolerance.

   66
2. 2
   Article

   gitconnected·1y

   Why Knowing TOAST in PostgreSQL is necessary for Backend Devs!

   TOAST (The Oversized-Attribute Storage Technique) in PostgreSQL helps manage large data values exceeding the 8kB page size limit using compression and out-of-line storage. It supports various data types and uses four storage strategies: PLAIN, EXTERNAL, EXTENDED, and MAIN, each with different methods for managing oversized data. Understanding TOAST strategies can help optimize database performance.

   65

   1
3. 3
   Article

   Hacker News·1y

   deepseek-ai/3FS: A high-performance distributed file system designed to address the challenges of AI training and inference workloads.

   The Fire-Flyer File System (3FS) is a high-performance distributed file system tailored for AI training and inference workloads. It utilizes modern SSDs and RDMA networks to offer a shared storage layer that simplifies the development of distributed applications. Key features include strong consistency, stateless metadata services, efficient data preparation, random access to training samples, high-throughput checkpointing, and cost-effective KVCache for inference. Performance tests show impressive read throughput and sorting efficiency, making 3FS an optimal choice for large-scale AI tasks.

   57
4. 4
   Article

   Jeff Geerling·45w

   Upgrading an M4 Pro Mac mini's storage for half the price

   The M4 Pro Mac mini now has third-party storage upgrade options available. A 4TB upgrade costs $699 compared to Apple's $1,200 option, offering significant savings. The upgrade process involves removing the rear cover, replacing the proprietary SSD module, and performing a DFU restore using another Mac. Performance testing shows the upgraded 4TB module delivers better write speeds than the stock 512GB drive due to more flash chips distributing write activity. The internal storage consistently outperforms external Thunderbolt 5 drives for sustained transfers.

   56
5. 5
   Article

   The Verge·28w

   Sandisk’s new 1TB USB-C SSD is so small you never need to take it out

   Sandisk launched the Extreme Fit, claiming it's the world's smallest 1TB USB-C flash drive. The L-shaped device is designed to stay plugged into laptops permanently, protruding vertically rather than horizontally to remain flush with the device. It offers capacities from 64GB to 1TB with transfer speeds up to 400MB/s using USB 3.2 Gen 1. Pricing ranges from $15.99 for 64GB to $117.99 for the 1TB model, now available in the US and other markets.

   54

   10
6. 6
   Article

   Jeff Geerling·1y

   Don't pay $800 for Apple's 2TB SSD upgrade

   Apple charges $800 to upgrade the Mac mini from 256 GB to 2 TB of storage. Recently, more affordable third-party upgrade kits became available for under $300, offering significant savings. The process involves opening the Mac mini and entering DFU mode using another Apple Silicon Mac. Performance improvements are modest, but the savings are substantial. These upgrade kits are sourced from the same manufacturer and may have delayed shipping during the Chinese New Year.

   44

   3
7. 7
   Article

   Redis·49w

   Redis to Manage Storage Replication

   A freelancer shares how Redis transformed their file system replication architecture by replacing a complex SQL-based solution with Redis pub/sub messaging. The new approach eliminated scheduled batch jobs in favor of real-time replication, reduced operational overhead, and leveraged Redis data structures for analytics. The case study demonstrates Redis's versatility beyond caching, showing its effectiveness for messaging, data storage, and system coordination.

   25

   2
8. 8
   Article

   Lobsters·28w

   Release v0.12 · perkeep/perkeep

   Perkeep 0.12 ships after a 5-year hiatus with major modernization: upgraded to Go 1.25, integrated Tailscale's tsnet library for private server deployment without public endpoints, migrated encryption from GPG to age, completed UI transition from GopherJS to native JavaScript, unified cloud storage backends (S3, B2, GCS), added container image distribution, expanded CI to macOS and Windows, and introduced comprehensive linting and testing improvements. The release includes contributions from 27 new contributors and removes deprecated features like AppEngine support and macOS FUSE.

   13
9. 9
   Article

   Watercooler·1y

   Uh oh

   13

   10