This talk is intended for data scientists and engineers interested in building a large-scale recommendation system, with a specific focus on real-time delivery. Building a real-time recommendation system for publishers poses distinct challenges and requirements, including managing a constantly changing item space, balancing contextual relevance with article performance, and accommodating custom curation of recommendations while handling tens of thousands of requests per second.

We will introduce you to our approach for addressing these architectural challenges. Beginning with a comprehensive overview of our architecture, which incorporates a batch-processing component for generating candidates and a real-time component for serving recommendations, we will then zero in on the architectural decisions and lessons learned from our experience scaling up the system.

We will delve into a detailed exploration of how we utilize the Web-Queue-Worker architecture to generate recommendations for each request while simultaneously maintaining a service level objective of sub-one-second request latency. Additionally, we will guide you through our iterative development process, which relies on regular load testing to meet the demands of ever-increasing traffic. We will share the metrics employed for tracking during load testing and demonstrate how these metrics aided us in pinpointing performance bottlenecks and selecting the appropriate technology and approach to overcome design challenges.