Skein sits in front of services like AWS S3, Cloudflare R2, Backblaze B2, Wasabi, MinIO, and other S3-compatible providers, and exposes them as a single S3-compatible endpoint for applications.
The goal is to connect it to any mutiple s3 compatible object storage api providers and let it handle:
<ul>
<li>routing</li>
<li>redundancy</li>
<li>caching</li>
<li>usage tracking</li>
<li>operational visibility</li>
</ul>
Basically, you may use multiple s3-compatible api providers like r2,s3 etc at the same time and store data simultaneously. You can control which type of data to flow in which s3 provider. Like video in s3, photos in r3 and pdf in wsabi etc. Or you can store same data into multiple providers but this doesnt make sense until a provider offers 99999999% durabilty but still a feature.
Moreover, it has caching functionality and it caches the objects in locally or in redis up to x size(configurable), it is useful for egress, when you do a lot of pull/fetch common data from s3 and you providers charge you for that…
Current features include:
<ul>
<li>multiple routing strategies:</li>
<li>file-size and content-type based rules</li>
<li>redundancy support</li>
<li>disk and Redis caching</li>
<li>live dashboard</li>
<li>REST management API</li>
<li>compatibility with standard S3 clients</li>
</ul>
I’m not a professional Go developer, so this project was also part of my effort to learn Go by building something practical. It may not be the most idiomatic Go codebase written, but it is functional, tested, and already learned a lot. There’s might be still work to do. Multi-account migration is implemented, but I still want to improve its performance and overall polish. I would like to improve also the cache feature for identifying the common objects fetched cache it up to x time.
The project started from a simple need: managing multiple free tier object storage providers and create “unique” space, all different apps talk to skein directly instead of different object stoage providers. But now it has some real cases such as improving fetch efficiency with caching, and reducing egress for frequently requested objects.
And yes, part of the inspiration was also seeing how far I could stretch free tiers across different cloud providers.
GitHub: <a href="https://github.com/techgonia-devjio/skein" target="_blank" rel="noopener nofollow">https://github.com/techgonia-devjio/skein</a>
Feedback is welcome, especially from people with experience in Go, storage systems, or infrastructure tooling.

The official Open Source daily.dev Squad. 

Open Source

Skein is a self-hosted S3 gateway written in Go that sits in front of multiple S3-compatible object storage providers (AWS S3, Cloudflare R2, Backblaze B2, Wasabi, MinIO) and exposes them as a single unified S3-compatible endpoint. It supports content-type and file-size based routing rules, redundancy across providers, disk and Redis caching to reduce egress costs, a live dashboard, and a REST management API. The project was built as a learning exercise in Go and was motivated by the desire to aggregate multiple free-tier storage providers into one logical storage space.

Introducing Skein: a self hosted s3 "gateway"