How we improved Canva’s private design search while respecting the privacy of our community.

Canva offers insights into graphic design, visual communication, and creative inspiration, providing articles, tutorials, and design templates to help individuals and businesses create stunning visuals for various purposes. By exploring Canva's curated content, designers and non-designers alike can learn about typography, color theory, and layout principles for crafting compelling designs for social media, marketing materials, presentations, and more. Whether you're designing logos, posters, or infographics, Canva offers resources to unleash your creativity and bring your ideas to life.

Canva Engineering

In October 2024, Canva reached 200 million monthly active users, highlighting the need for an effective search function for user designs. Traditional search evaluations use sampled user queries and expert labeling, but Canva prioritizes privacy, avoiding the use of real user data. Instead, they leverage generative AI to create synthetic content, enabling them to test search improvements without compromising privacy. The post discusses building a labeled evaluation dataset using GPT-4, issues encountered, and the development of an evaluation tool for rapid, reproducible offline testing. This allows engineers to confidently assess changes locally before online experimentation, enhancing iteration speed and result reliability.

How to improve search without looking at queries or results

Generating realistic private search datasets