Best of Storage — May 2024
- 1
- 2
Community Picks·2y
The State of Data Engineering 2024
The 2024 State of Data Engineering report discusses the influence of GenAI on software infrastructure, the expansion of product offerings due to the economic downturn, and the impact of open table formats and their catalogs in the data lake industry. It also highlights the importance of data version control and observability in AI/ML systems.
- 3
- 4
Substack·2yS3 Is Showing Its Age
S3 is falling behind its competitors in terms of features such as compare-and-swap operation, multi-region buckets, and object appends. These missing features are becoming a bigger problem as new infrastructure relies on object storage as the primary persistence layer. The decision to abandon or embrace S3 depends on the system's use cases and design goals. Preconditions on S3 and a unified S3Express One Zone API would make this decision easier.
- 5
