Abstract/Proposal

Server-driven recommendation systems (RecSys) face significant challenges when it comes to scaling to handle large data volumes and providing real-time recommendations. At Glance, we serve personalized recommendations to over millions of users, prioritizing response time and data privacy. To tackle these challenges, we have turned to edge machine learning (ML). By deploying ML models closer to the data source, edge ML offers a solution that reduces latency, conserves network bandwidth, and enhances data privacy for our customers.

In this talk, we will explore the potential of edge ML in scaling Recommendation Systems. We will delve into how Glance is utilizing edge computing to deliver user-personalized content recommendations at scale, resulting in reduced latencies, enhanced user experiences, and lower server costs. Moreover, we will discuss the capabilities enabled by this approach, such as real-time personalization for third-party integrations without their data ever leaving the edge device. We will also talk about how we enable large number of A/B experiments in our release cycle and also how we implement a server-side RecSys as an augmentation that we use with our Edge ML flow.

However, achieving these results was not without its challenges. We will address the obstacles we encountered during the development and deployment of this architecture, particularly focusing on our implementation along with a roadmap. While we made some strides in solving some of the challenges, we are still on the path of our initial journey of complete autonomous federated edge ML systems. So, apart from the solved obstacles, we will also cover some of the ongoing challenges.

By the end of this talk, attendees will gain a deeper insight into the practical application of edge ML Recommendation Systems. We will emphasize the advantages it offers, such as reduced latency, improved user experiences, enhanced privacy, and cost-effectiveness. Moreover, we will touch upon the future potential and broader implications of edge ML in the field.

Talk Outline

• Why Edge ML is even needed in a server-driven space?
• How Edge ML is beneficial for Glance?
• Our journey from POC to full-scale production
• How we do fast experimentation in a slower app release cycle?
• How we are handling some of the challenges of running and architecting the system at scale.