Metadata is useful for data warehouses, streaming systems, humans, and machines. We don't need different catalogs for each use case, though. Catalog convergence is here.

Materialized View

Metadata is crucial for various use cases, but having separate catalogs for each is unnecessary. The trend towards catalog convergence shows promise in making systems more integrated. Projects like Recap, which offers a single schema definition language, illustrate this shift. Recap has been adopted by Gable.ai, and similar functionalities are being incorporated into platforms like Backstage and Datahub. Ultimately, the idea is to streamline metadata management across different systems.

These Aren’t the Catalogs You’re Looking For