Say What You See: Building Multimodal Content Signals at Scale by Neeraj Barthwal “Say what you see.” It’s a simple concept, but this phrase highlights a fundamental disconnect in how travel …

Agoda Engineering Blog offers insights, technical articles, and updates on building scalable and high-performance travel technology platforms. Covering topics such as backend development, distributed systems, and machine learning applications, Agoda Engineering Blog provides resources for software engineers and data scientists working in the travel and hospitality industry, helping them deliver exceptional user experiences and drive business growth through technology innovation.

agoda

Agoda built a Multimodal Content System that bridges the gap between hotel images and guest reviews by introducing a shared topic taxonomy. The system clusters images and reviews under common topics (e.g., Breakfast, Pool, Room Quality), enabling a unified view with relevant images, review snippets, and sentiment scores. At scale, it processes 450M+ images and millions of multilingual reviews daily using PySpark jobs orchestrated via Kubeflow, with results served from Couchbase. Three core pipelines handle image curation, review snippet extraction, and sentiment aggregation per topic. A/B tests showed doubled click rates on multimodal highlights on mobile and consistently positive engagement metrics across platforms. The next step is consolidating all content signals through LLMs into a single unified model.

Say What You See: Building Multimodal Content Signals at Scale

The User Journey: Manual Cross-Referencing

Get Agoda Engineering’s stories in your inbox

Technical Deep-Dive: Topic-Based Image Curation

Technical Deep-Dive: Topic-Based Review Snippets

Technical Deep-Dive: Sentiment Aggregation

Putting It Together: The Multimodal Content Package