How Agoda Enhanced the Uptime and Consistency of Financial Metrics by Warot Jongboondee Introduction At Agoda, financial metrics, including sales, cost, revenue, and margin, are among our most …

Agoda Engineering Blog offers insights, technical articles, and updates on building scalable and high-performance travel technology platforms. Covering topics such as backend development, distributed systems, and machine learning applications, Agoda Engineering Blog provides resources for software engineers and data scientists working in the travel and hospitality industry, helping them deliver exceptional user experiences and drive business growth through technology innovation.

agoda

Agoda consolidated multiple financial data pipelines into a single centralized system called FINUDP to address data inconsistencies, quality issues, and reliability concerns. The unified pipeline processes millions of daily financial data points using Apache Spark, achieving 95.6% uptime with a goal of 99.5%. Key improvements include hourly data updates, automated quality checks using tools like Quilliup and GoFresh, shadow testing, staging environments, and data contracts with upstream teams. The centralization reduced pipeline runtime from five hours to 30 minutes while establishing a single source of truth for financial metrics across the organization.

How Agoda Enhanced the Uptime and Consistency of Financial Metrics

The Challenges of Multiple Financial Data Pipelines

What Happens When Pipeline Inconsistencies Go Unchecked

The Solution: Centralized Financial Data Pipeline (FINUDP)

Key challenges encountered while building FINUDP

How can we ensure data quality and uptime with a centralized data pipeline?

Architectural Trade-offs in Centralizing Financial Data Pipelines