<p><strong>TLDR:</strong> Apache Flink CDC 3.6.0 landed today with a meaningful connector expansion, including a new Oracle source and Apache Hudi sink. PostgreSQL 19 is adding native JSON format support to COPY TO, which is faster and lighter than the old to_json() workaround. ProxySQL is splitting into three release tracks, with an experimental v4 line that puts AI at the proxy layer. There’s also a solid deep-dive on category theory and dataframe algebra if you’re into that kind of thing.</p>
<hr>
<h2>Apache Flink CDC 3.6.0 release</h2>
<p>Flink CDC 3.6.0 extends support to Flink 1.20.x and 2.2.x, bumps the minimum JDK to 11, and adds two new pipeline connectors: an Oracle source and an Apache Hudi sink. PostgreSQL users get schema evolution support, and the Transform framework now handles a VARIANT type with JSON parsing. On the connector side, Iceberg picks up AWS Glue and BigQuery catalog support, StarRocks gets DDL support, and Kafka gets better complex type serialization. There are also targeted bug fixes for MySQL 8.0+ binlog edge cases and data loss scenarios that are worth reviewing if you’re running MySQL CDC in production.</p>
<h2>PostgreSQL 19 COPY TO with JSON format</h2>
<p>PostgreSQL 19 will let you run <code>COPY TO</code> with a <code>FORMAT JSON</code> option, outputting NDJSON directly — one JSON object per line. There’s an optional <code>force_array</code> flag if you need a JSON array instead. The native implementation is faster and uses less memory than the existing workaround of wrapping a SELECT with <code>to_json()</code>. Worth noting: it’s incompatible with the <code>HEADER</code>, <code>NULL</code>, and <code>DELIMITER</code> options, so it’s a clean break from the text/CSV path.</p>
<h2>ProxySQL splits into three release tracks</h2>
<p>ProxySQL 3.0.6 ships alongside a new tiered release strategy. The 3.0.x stable track is for production. The 3.1.x innovative track introduces early features including an embedded time series database and a traffic observer. The 4.0.x AI track is explicitly experimental, targeting RAG pipelines and natural language querying at the proxy layer without requiring schema changes or new infrastructure. The idea of putting AI intelligence at the proxy rather than in the app or database is interesting in principle — whether v4 delivers on that is a different question.</p>
<h2>Category theory and dataframe algebra</h2>
<p>A developer building a typed Haskell dataframe library traced 200+ pandas operations back to 15 algebraic operators from Petersohn et al., then found those reduce further to three categorical patterns: Delta (projection/restructuring), Sigma (groupby/merge), and Pi (join/pairing). These map to an adjoint triple from Fong and Spivak’s category theory work, which turns out to explain why those three operations compose correctly. The practical payoff is compile-time schema verification and safe query optimizations like predicate pushdown and filter fusion.</p>
<hr>
<h2>Also notable</h2>
<ul>
<li><strong>Data warehouse market overview:</strong> A broad explainer covering the subject-oriented, integrated, time-variant, non-volatile characteristics of data warehouses, with a market projection of USD 30 billion by 2025 — mostly useful as a reference piece rather than new signal.</li>
</ul>


Query optimization, schema design, and data store coverage across PostgreSQL, MongoDB, Redis, and more.

Databases Digest

Apache Flink CDC 3.6.0 is out with Oracle source and Apache Hudi sink connectors, Flink 1.20/2.2 support, PostgreSQL schema evolution, and bug fixes for MySQL 8.0+ CDC edge cases. PostgreSQL 19 adds native NDJSON export via COPY TO FORMAT JSON, offering better performance than the to_json() workaround. ProxySQL introduces a three-track release strategy: stable 3.0.x, innovative 3.1.x with an embedded time series DB, and experimental 4.0.x targeting AI/RAG at the proxy layer. A deep-dive also explores how 200+ pandas operations reduce to three categorical patterns, enabling compile-time schema verification and safe query optimizations in a typed Haskell dataframe library.

Flink CDC 3.6 ships Oracle and Hudi connectors, PostgreSQL 19 gets native NDJSON export