<p><strong>TLDR:</strong> Uber open-sourced the architecture behind IngestionNext, a streaming-first data lake ingestion system that cuts end-to-end latency and compute costs by 25%. Snowflake moved INTERVAL to a native column type in public preview, which is a quiet but meaningful win for anyone doing date arithmetic in SQL. There’s also a solid war story on pgvector benchmarks lying to you at small scale, and SQL Server has a silent rename bug worth knowing about before it bites you in production.</p>
<hr>
<h2>Uber’s IngestionNext: streaming-first data lake ingestion</h2>
<p>Uber shipped IngestionNext, a system that flips the default assumption from batch-first to streaming-first for data lake ingestion. The result is a 25% reduction in both end-to-end latency and compute costs. That’s a meaningful number at Uber’s scale, and the architectural direction — treating streaming as the primary path rather than a bolt-on — is worth paying attention to if you’re still running batch-heavy pipelines.</p>
<h2>Snowflake native INTERVAL type hits public preview</h2>
<p>Snowflake now supports INTERVAL as an actual column data type, following the ANSI SQL standard with two variants: INTERVAL YEAR TO MONTH and INTERVAL DAY TO SECOND. Until now, storing durations meant reaching for integers or strings and doing the conversion yourself. This lets you store time spans directly, filter and join on them, and write cleaner date arithmetic without workarounds. It’s in public preview, so worth testing before you commit to it in production schemas.</p>
<h2>pgvector benchmarks at small scale don’t predict production behavior</h2>
<p>If you’ve been benchmarking pgvector on 10k vectors at 128 dimensions and feeling good about it, this post is a useful reality check. At 5M vectors and 1536 dimensions, the operational picture changes significantly. The practical advice here is solid: choose between IVFFlat and HNSW deliberately based on workload, tune ef_search and M, use WHERE clause filtering to narrow candidate sets before the vector scan, partition on query-filter fields, and run pg_prewarm after deployments to avoid cold-cache penalties. pgvector can work in production, but it needs the same rigor you’d give any serious Postgres workload.</p>
<h2>SQL Server silently renames users on ALTER with login</h2>
<p>This one is easy to miss: running ALTER USER … WITH LOGIN without explicitly specifying a NAME causes SQL Server to silently rename the database user to match the login name. The first run of an idempotent script renames the user; the second run can’t find it and fails. The fix is straightforward — always include WITH NAME = [original_username] — but the silent behavior is the kind of thing that causes confusing failures in automated deployments.</p>
<hr>
<h2>Also notable</h2>
<ul>
<li><strong>Snowflake Cortex Code GA:</strong> Cortex Code is now generally available in Snowsight, with Windows CLI support, Agent Teams for parallel multi-agent orchestration, and new skills covering ML lifecycle, cost intelligence, and Cortex AI functions.</li>
<li><strong>Snowflake AI_PARSE_DOCUMENT:</strong> New Cortex SQL function extracts text, structure, and images from PDFs, DOCX, PPTX, and image formats, with four modes (OCR, LAYOUT, page filtering, image extraction) and composable with AI_EXTRACT and AI_COMPLETE.</li>
<li><strong>Halodoc RAG retrieval architecture:</strong> Halodoc built a hybrid semantic + keyword retrieval layer on OpenSearch using HNSW k-NN with 768-dimensional embeddings, validating ~21K prescriptions and driving ~32K bookings from their Q&amp;A assistant.</li>
<li><strong>jOOQ 3.21 redacted columns:</strong> Commercial distributions now support marking specific columns as redacted in code generation settings, masking values with asterisks in debug logs to prevent sensitive data leakage in production.</li>
<li><strong>Greenmask 0.2.9-0.2.17:</strong> PostgreSQL anonymization tool adds PostgreSQL 18 support, a new RandomCompany transformer, postcode masking, dynamic transformer parameters, and fixes for composite foreign key introspection and ACL restoration ordering.</li>
<li><strong>pgvector + PostgreSQL stack dominance:</strong> A survey of 200 profitable startup founders found Next.js, Tailwind, and PostgreSQL as the most common stack — not surprising, but a useful data point if you’re making infrastructure decisions.</li>
<li><strong>Mongo Lens mobile client:</strong> Android app and VS Code extension for browsing and editing MongoDB from a phone, built on Next.js and React Native, with graph/tree views and fingerprint auth.</li>
</ul>


Query optimization, schema design, and data store coverage across PostgreSQL, MongoDB, Redis, and more.

Databases Digest

A curated roundup of data engineering news: Uber open-sourced IngestionNext, a streaming-first data lake ingestion system achieving 25% lower latency and compute costs. Snowflake added native INTERVAL column type in public preview, enabling cleaner date arithmetic per ANSI SQL. A practical deep-dive warns that pgvector benchmarks at small scale (10k vectors) don't reflect production behavior at 5M+ vectors, with tuning advice for IVFFlat/HNSW, ef_search, partitioning, and pg_prewarm. SQL Server has a silent rename bug when using ALTER USER ... WITH LOGIN without specifying NAME. Also covered: Snowflake Cortex Code GA, AI_PARSE_DOCUMENT for PDFs, Halodoc's RAG retrieval on OpenSearch, jOOQ 3.21 redacted columns, and Greenmask 0.2.9-0.2.17 PostgreSQL anonymization updates.

Uber's IngestionNext cuts latency 25%, Snowflake ships native INTERVAL type