Anthill Inside 2019

Search and product recommendations are typically served using CF, MF, FM techniques, content/context/sequence based methods or using learning to rank framework which is more generic than rest. Evolving from traditional classification and regression modeling methods, the loss functions, gradients, computation tricks have evolved to suit ranking problems (point wise to pair wise to list wise solutions) in this ambit. Learning to rank models such as LambdaRank and LambdaMart models (developed by Microsoft research group) are very competitive and have shown success in many instances (Kaggle, Yandex, Yahoo rankings challenges). More recently, multi-item group wise scoring functions were proposed by google research group (2018). This talk would focus on our team’s journey (what worked and what didn’t) in using these methods for ranking, importance of sampling, importance of measuring performance using multiple metrics (not just NDCG), importance of offline and online evaluations for ranking, and compare Ranknet, LambdaRank/LambdaMART, Listnet and multi-item group-wise scoring functions.

Outline

1. Introduction to learning to rank framework.
2. Metrics for evaluation.
3. Description of the methods (Ranknet, LambdaRank, lamdaMART, multi-item group-wise scoring functions)
4. Integrating diverse feature types, feature engineering, transformations and sampling of items.
5. Scaling up model training and deployment infrastructure
6. Results, Summary
7. Future work

Speaker bio

SPEAKER: Arpit Katiyar: Arpit Katiyar is currently working as Lead Data Science Engineer at MakeMyTrip(MMT). He is part of user personalization team with focus on delivering relevant content to each MMT customer. He has a total work experience of 5 years in building machine learning based solution for biometric and large scale telecom data with companies Samsung and Mobileum, respectively. Arpit holds B. Tech and M. Tech degree in Computer Science from IIT Delhi with thesis work published in ACM - ICS(International Conference on Supercomputing).

CONTRIBUTORS:
Pulkit Bansal: Pulkit is currently Lead Data Scientist at MakeMyTrip. His focus at MakeMyTrip has been on problems relating to developing personalized hotel recommendations using ML models. He has also worked on feature store engineering and customer-loan propensity projects. He has total 5.5+ years of experience, and has previously worked on problems in quantitative finance and algorithmic trading at Goldman Sachs and WorldQuant. He has also worked on optimization problems in online advertising at Adobe. Pulkit has graduated with a BS-MS Dual Degree in Mathematics and Scientific Computing at IIT Kanpur, where he graduated at the top of his class.

Narasimha Medeme: Director, data science at MakeMyTrip. He oversees Hotel ranking, user personalization, multi LOB initiatives such as cross-sell, personalized notifications, and hotel dynamic pricing. He has ~15 years experience working with many fortune 500 company clients (previously at Kantar data science), multiple domains and business problems, solving diverse data science and optimization problems. While he is applying machine-learning/deep-learning models for personalization, he is also experimenting with deep reinforcement learning models/practices in off-line learning environments. Narasimha has MSc from University of Missouri, BTech from IIT Madras.

Links

• http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.180.634&rep=rep1&type=pdf
• https://arxiv.org/pdf/1811.04415.pdf
• https://ai.googleblog.com/2018/12/tf-ranking-scalable-tensorflow-library.html

Slides

https://docs.google.com/presentation/d/1KxUPdaybA0kSqBcKy9vpGMT_O8_qp8ML-q9G-Qh7lU8/edit#slide=id.p1