Building Databases over a Weekend

Databases are ubiquitous yet often viewed as complex systems, typically developed by specialized experts. Despite their complexity, innovation in database technology continues, with tools like Apache DataFusion simplifying the process for developers. DataFusion allows developers to build custom databases by extending or replacing various layers, particularly useful for creating bespoke query engines. This guide demonstrates how to implement a window operator for stream processing applications using DataFusion, detailing the integration into the physical and logical planning stages and optimizing the custom operator.