With cross-platform monitoring, a common problem is the difficulty in getting comprehensive and in-depth views on metrics, making it tough to see the big picture. Read to find out how the Data Tech team ideated Iris to turn observations into actionable insights for enhanced decision-making.

Grab is a leading technology company in Southeast Asia, offering a wide range of services, including ride-hailing, food delivery, and digital payments. Through its platform, Grab provides convenience, accessibility, and reliability to millions of users across the region. From their blog developers can learn from Grab's innovative approach to technology and business, gaining insights into building scalable, customer-centric platforms that address real-world challenges and improve people's lives.

Grab Tech Blog

Iris is an observability platform for ETL jobs that captures critical job metrics, provides real-time monitoring and offline analytics, and offers actionable insights. It uses metrics collector, Kafka queue, and TIG stack for its platform architecture and collects data using JVM Profiler and sparkMeasure. Iris visualizes real-time data using Grafana and routes metrics to InfluxDB for offline analysis. It also employs K-means clustering for job classification and enhances accuracy in calculating Databricks infrastructure costs. Case studies demonstrate how Iris improves performance and cost efficiency. Future plans include developing APIs, adding a Presto listener, and creating a feedback loop for continuous improvement.

Iris - Turning observations into actionable insights for enhanced decision making