Hussain Sultan, co-founder of Xorq, discusses the fragmented DataFrame ecosystem and how his company addresses it with a multi-engine DataFrame library. The conversation covers Xorq's compute catalog approach, which creates portable, reusable expressions that work across different engines like DuckDB, DataFusion, and Pandas. Sultan explains how the system handles caching, cache invalidation, and plans for streaming support, while also touching on AI/ML workloads and the technical architecture built on Apache Arrow, Ibis, and DataFusion.
Sort: