Say What You See: Building Multimodal Content Signals at Scale

Agoda built a Multimodal Content System that bridges the gap between hotel images and guest reviews by introducing a shared topic taxonomy. The system clusters images and reviews under common topics (e.g., Breakfast, Pool, Room Quality), enabling a unified view with relevant images, review snippets, and sentiment scores. At scale, it processes 450M+ images and millions of multilingual reviews daily using PySpark jobs orchestrated via Kubeflow, with results served from Couchbase. Three core pipelines handle image curation, review snippet extraction, and sentiment aggregation per topic. A/B tests showed doubled click rates on multimodal highlights on mobile and consistently positive engagement metrics across platforms. The next step is consolidating all content signals through LLMs into a single unified model.