Anthill Inside 2019

A conference on AI and Deep Learning

Tickets Propose a session

Learning to Rank framework for product recommendation - Ranknet to LambdaMART to Groupwise scoring functions - experiments

Submitted by narasimha m (@6544) on Wednesday, 1 May 2019

Preview video

Session type: Full talk of 40 mins


Search and product recommendations are typically served using CF, MF, FM techniques, content/context/sequence based methods or using learning to rank framework which is more generic than rest. Evolving from traditional classification and regression modeling methods, the loss functions, gradients, computation tricks have evolved to suit ranking problems (point wise to pair wise to list wise solutions) in this ambit. Learning to rank models such as LambdaRank and LambdaMart models (developed by Microsoft research group) are very competitive and have shown success in many instances (Kaggle, Yandex, Yahoo rankings challenges). More recently, multi-item group wise scoring functions were proposed by google research group (2018). This talk would focus on our team’s journey (what worked and what didn’t) in using these methods for ranking, importance of sampling, importance of measuring performance using multiple metrics (not just NDCG), importance of offline and online evaluations for ranking, and compare Ranknet, LambdaRank/LambdaMART, Listnet and multi-item group-wise scoring functions.


  1. Introduction to learning to rank framework.
  2. Metrics for evaluation.
  3. Description of the methods (Ranknet, LambdaRank, lamdaMART, multi-item group-wise scoring functions)
  4. Integrating diverse feature types, feature engineering, transformations and sampling of items.
  5. Scaling up model training and deployment infrastructure
  6. Results, Summary
  7. Future work

Speaker bio

SPEAKER: Arpit Katiyar: Arpit Katiyar is currently working as Lead Data Science Engineer at MakeMyTrip(MMT). He is part of user personalization team with focus on delivering relevant content to each MMT customer. He has a total work experience of 5 years in building machine learning based solution for biometric and large scale telecom data with companies Samsung and Mobileum, respectively. Arpit holds B. Tech and M. Tech degree in Computer Science from IIT Delhi with thesis work published in ACM - ICS(International Conference on Supercomputing).

Pulkit Bansal: Pulkit is currently Lead Data Scientist at MakeMyTrip. His focus at MakeMyTrip has been on problems relating to developing personalized hotel recommendations using ML models. He has also worked on feature store engineering and customer-loan propensity projects. He has total 5.5+ years of experience, and has previously worked on problems in quantitative finance and algorithmic trading at Goldman Sachs and WorldQuant. He has also worked on optimization problems in online advertising at Adobe. Pulkit has graduated with a BS-MS Dual Degree in Mathematics and Scientific Computing at IIT Kanpur, where he graduated at the top of his class.

Narasimha Medeme: Director, data science at MakeMyTrip. He oversees Hotel ranking, user personalization, multi LOB initiatives such as cross-sell, personalized notifications, and hotel dynamic pricing. He has ~15 years experience working with many fortune 500 company clients (previously at Kantar data science), multiple domains and business problems, solving diverse data science and optimization problems. While he is applying machine-learning/deep-learning models for personalization, he is also experimenting with deep reinforcement learning models/practices in off-line learning environments. Narasimha has MSc from University of Missouri, BTech from IIT Madras.



Preview video


  • Abhishek Balaji (@booleanbalaji) Reviewer 5 months ago

    Hello Pulkit/Arpit/Narasimha,

    Thank you for submitting a proposal. As per the speaker policy for The Fifth Elephant, we only allow one presenter on stage per session. Please decide who among you would be presenting and update the bio to reflect the same. To proceed with evaluation, we need to see detailed slides and a preview video for your proposal. Ensure your slides must cover the following:

    • Problem statement/context, which the audience can relate to and understand. The problem statement has to be a problem (based on this context) that can be generalized for all.
    • What were the tools/options available in the market to solve this problem? How did you evaluate alternatives, and what metrics did you use for the evaluation?
    • Why did you pick the option that you did?
    • Explain how the situation was before the solution you picked/built and how it changed after implementing the solution you picked and built? Show before-after scenario comparisons & metrics.
    • What compromises/trade-offs did you have to make in this process?
    • What is the one takeaway that you want participants to go back with at the end of this talk? What is it that participants should learn/be cautious about when solving similar problems?
    • Is the tool free/open-source? If not, what can the audience takeaway from the talk?

    We need to see the updated slides on or before 21 May in order to close the decision on your proposal. If we do not receive an update by 21 May we’ll move the proposal for consideration at a future event.

  • Abhishek Balaji (@booleanbalaji) Reviewer 4 months ago

    Moving to rejected, since proposers havent responded to comments/feedback.

  • narasimha m (@6544) Proposer 4 months ago

    hi Abhishek,
    I had already uploaded the slides and preview video above with detailed comments. This update was made before the deadline (21st May).

    Please take a look and let me know.

  • narasimha m (@6544) Proposer 4 months ago

    hi Abhishek,

    We had also updated the description of who will be the speaker. Arpit is speaker, others have contributed to the project/applied-research. So these details too have been updated in the slides AND in speaker bio section above. Before 21st May.


Login with Twitter or Google to leave a comment