pathwaycom/pathway: Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Pathway is a Python ETL framework designed for stream processing, real-time analytics, and building AI pipelines. It integrates easily with Python ML libraries and utilizes a scalable Rust engine for efficient multithreading and distributed computation. With a variety of connectors to data sources and robust features like persistence and stateful transformations, Pathway is suited for both batch and streaming data environments. It is deployable through Docker and Kubernetes, making it a scalable solution for enterprise-level applications.
