Discover how daily.dev uses Project Sauron, a powerful two-tower retrieval model, to deliver personalized content to over half a million developers. Learn how this innovative recommendation system leverages deep learning to improve user engagement and enrich developer experience with relevant and diverse posts.

daily.dev

Project Sauron by daily.dev uses two-tower retrieval models to deliver personalized content to developers, significantly boosting engagement metrics. The model employs deep learning to process user and post features, creating highly relevant recommendations. Efforts are ongoing to improve the model's accuracy and address concerns such as diversity in recommendations and cold-start user issues.

Project Sauron: building a two-tower retrieval model for personalized recommendations at daily.dev

How to separate the wheat from the chaff?

What’s next for the Sauron recommender?

<p>This work has had a huge impact on the quality of my feed.</p>


<p>This is why I keep coming back to daily.dev .</p>
<p>Also, tldr for lazy or busy people (made with Orbit by Mozilla):</p>
<h2>Summary</h2>
<p>Project Sauron is a personalized recommendation system developed by daily.dev, which utilizes two-tower retrieval models to deliver highly relevant posts to users in “My Feed” and Digest emails. The system processes various user and content features to enhance relevance and diversity, leading to increased user engagement. Daily.dev faced the challenge of filtering through thousands of daily posts to deliver the most relevant content to over half a million users, each with unique preferences.Two-tower retrieval models are an evolution of matrix factorization models, which learn embedding representations of users and posts in a shared space. These models use deep learning to process multi-modal features from both users and posts, capturing complex relationships between them. The architecture consists of two separate networks, called towers, which process user and post features to create the embedding representations.Project Sauron, named after the Two Towers in Lord of the Rings, has achieved great results using a simple feature set, including post titles, summaries, tags, source, author, post type, content type, post age, and post language. The user features include the user ID. The model is implemented using TensorFlow and the retrieval model from the tensorflow-recommenders package.The impact of Project Sauron has been significant, with improvements across all testing metrics, including a 5% increase in significant users interacting with the feed, 11% in unique posts clicked, 5% in upvotes, and 44% in unique bookmarked posts. The model is also used to generate recommendations for Digest emails, resulting in a 2% increase in clicks for inactive users and 17% and 11% increase in clicks for old and new registered users, respectively.Future plans for the Sauron recommender include addressing the user-coldstart issue using two-tower models and maintaining feed diversity to avoid echo chambers. Project Sauron has been a success for daily.dev, improving the platform as a tool for developers and contributing to their personal growth.</p>
