<p><strong>TLDR:</strong> Someone imported the entire Linux kernel git history into PostgreSQL and it fits in 6.6 GB with full SQL query support. Pinterest quietly fixed a persistent Spark reliability problem, cutting OOM failures by 96% through config tuning and automatic retries. AWS launched a native file system interface on S3, and PlanetScale published a detailed breakdown of how their Postgres-based query traffic control actually works.</p>
<hr>
<h2>pgit: the Linux kernel history as a queryable Postgres database</h2>
<p>Someone built a Git-like CLI called pgit that stores repository history in PostgreSQL using delta compression, then pointed it at the Linux kernel. The result: 1,428,882 commits, 24.4 million file versions, and 20 years of history imported in 2 hours on a Hetzner dedicated server, landing in a 6.6 GB database (2.7 GB actual data). From there you can run SQL across the full history in seconds — things like identifying the most tightly coupled subsystems (Intel i915 and Btrfs), or finding that David S. Miller merged 7.9% of all commits. The 25:1 contributor-to-committer ratio and the fact that 90% of commits touch 5 or fewer files are the kind of numbers that are hard to get any other way. Worth knowing: bcachefs took 13 years to merge into mainline, and there are exactly 7 f-bombs in commit messages, from 2 people.</p>
<h2>Pinterest cut Spark OOM failures by 96%</h2>
<p>Pinterest reduced out-of-memory failures in Apache Spark by 96% across recommendation systems and large-scale data processing pipelines. The fix came from three directions: better observability to actually see what was failing, configuration tuning, and automatic memory retry mechanisms. No exotic infrastructure changes — mostly instrumentation and letting the system retry intelligently instead of dying. For teams running Spark at scale, the observability piece is the one worth stealing first.</p>
<h2>AWS S3 Files: object storage with a file system interface</h2>
<p>AWS launched S3 Files, a native NFS v4.1+ interface on top of S3 that exposes objects as files and directories. It supports standard file operations including permissions, locking, and incremental updates, and is accessible from any AWS compute instance, container, or function. The pitch is that you no longer have to choose between S3’s cost and the interactivity of something like EFS — and you don’t need FUSE-based workarounds to get there. The target workloads are AI agent pipelines, ML training, and multi-node applications that need concurrent read/write without moving data out of S3. Generally available now.</p>
<h2>How PlanetScale’s query traffic control actually works</h2>
<p>PlanetScale published a detailed technical breakdown of their Database Traffic Control feature, which is a Postgres extension that intercepts queries before execution and applies four checks: rule matching, concurrency limits, per-query cost estimates, and cumulative resource usage via a reverse leaky-bucket algorithm. Cost prediction uses exponential moving averages to map planner estimates to real CPU time per query pattern. Rules are stored as Postgres config parameters, which means propagation to all replicas takes 1-2 seconds without server restarts. The custom O(1) rule set data structure is the interesting part — evaluating many rules per query with minimal overhead is the kind of problem that’s easy to get wrong at scale.</p>
<hr>
<h2>Also notable</h2>
<ul>
<li><strong>Supabase Agent Skills:</strong> Supabase released a ~100-line <a href="http://SKILL.md" target="_blank" rel="noopener nofollow">SKILL.md</a> file that teaches AI coding agents to use Supabase correctly; task completion rates in testing went from 42-71% to 67-88% when using MCP plus the skill, with the main gains coming from agents knowing <em>when</em> to apply security_invoker and RLS, not just how.</li>
<li><strong>PostgREST:</strong> A practical walkthrough of using PostgREST to expose a Postgres database as a REST API with full CRUD, filtering, and pagination — the tradeoff is heavier database load from RLS and complex logic pushed into SQL functions or views.</li>
<li><strong>Snowflake DCM Projects:</strong> Snowflake’s Database Change Management Projects hit public preview, offering a declarative YAML-based framework for managing database objects across environments via the Snowflake CLI, with dry-run previews before applying changes.</li>
<li><strong>OpenSearch on Graviton4:</strong> Amazon OpenSearch Service now supports i8ge instances on Graviton4, claiming 60% better compute performance over Graviton2-based Im4gn instances, 55% better storage performance per TB, and 60% lower I/O latency.</li>
<li><strong>Polyglot database tax:</strong> A concrete breakdown of the compounding costs of running five specialized databases for one application — network latency per hop, broken transactions requiring saga patterns, fragmented security, and high cognitive load — with the argument that multimodal engines reduce all of it.</li>
<li><strong>EF Core batch ops:</strong> EF Core 7+ ExecuteUpdate and ExecuteDelete bypass the change tracker and run a single SQL statement regardless of row count; benchmarks against 1M rows show 2-4x speedup over the naive load-modify-save pattern, with EF Core 10 still lacking a native INSERT…SELECT equivalent.</li>
<li><strong>Iceberg REST Catalog in CI:</strong> A walkthrough of collapsing a multi-service Iceberg stack (HDFS, Hive Metastore, MinIO, REST Catalog) into a single Docker image under 5GB for CI pipelines where docker-compose isn’t available.</li>
<li><strong>AI bot cache disruption:</strong> Cloudflare and ETH Zurich published findings that AI crawler traffic — now over 10 billion requests per week — is breaking CDN and database caching strategies, with 70-100% unique URL access ratios causing high miss rates and displacing human-oriented cached content.</li>
<li><strong>OpenEBS vs Longhorn:</strong> A hands-on Kubernetes storage comparison found Longhorn’s ~1.5GB memory overhead and ~10x higher latency versus local PVs unjustifiable for small clusters; OpenEBS with the hostpath provisioner matched raw local PV performance at ~180MB memory overhead.</li>
<li><strong>InfluxDB 3.9:</strong> InfluxData’s Q1 recap includes InfluxDB Enterprise 1.12.3 with 100x faster retention enforcement, 30% lower CPU during compaction, and 5x faster backups, plus new Processing Engine plugins for Kafka, MQTT, and AMQP ingestion.</li>
</ul>


Query optimization, schema design, and data store coverage across PostgreSQL, MongoDB, Redis, and more.

Databases Digest

A roundup of notable database and data engineering developments: pgit imports the entire Linux kernel git history (1.4M commits, 20 years) into PostgreSQL in 6.6 GB with full SQL query support. Pinterest cut Apache Spark OOM failures by 96% through observability improvements, config tuning, and automatic memory retries. AWS launched S3 Files, a native NFS v4.1+ interface on S3 eliminating the need for FUSE workarounds, targeting AI/ML and multi-node workloads. PlanetScale detailed how their Postgres-based query traffic control works, using rule matching, concurrency limits, cost estimation via exponential moving averages, and a reverse leaky-bucket algorithm. Also covered: Supabase agent skills improving AI coding task completion rates, PostgREST for REST APIs over Postgres, Snowflake DCM Projects in public preview, EF Core batch operations benchmarks, Iceberg REST Catalog in CI, and findings on AI crawler traffic disrupting CDN caching strategies.

pgit loads Linux kernel history into Postgres, Pinterest cuts Spark OOM failures by 96%