The Fifth Elephant 2019

The eighth edition of India's best data conference

Participate

Model interpretability

Submitted by Namrata Hanspal (@namrata4) on Sunday, 14 April 2019

Session type: Short talk of 20 mins

Abstract

The choice we make

Complex machine learning models work very well at prediction and classification tasks but become really hard to interpret. On the other hand simpler models are easier to interpret but less accurate and hence oftentimes we are made to take a call between interpretability and accuracy.

The need to interpret

We live in exciting progressive times, decision making algorithms are everywhere right from influencing our social experiences, the news we see, our finances to our career opportunities . Turning decision making entirely to algorithms raises concerns. How can we be sure the algorithmically curated news doesn’t have a political party bias or job listings don’t reflect a gender or racial bias. This paves a path to fully understand the algorithm in and out.

Outline:

  • What is model interpretability.
  • Why model interpretability is important.
  • Different types of model interpretability
  • Trade off between interpretability and accuracy.
  • Popular model-agnostic libraries and how do they differ
  • Key takeaways

Outline

The choice we make

Complex machine learning models work very well at prediction and classification tasks but become really hard to interpret. On the other hand simpler models are easier to interpret but less accurate and hence oftentimes we are made to take a call between interpretability and accuracy.

The need to interpret

We live in exciting progressive times, decision making algorithms are everywhere right from influencing our social experiences, the news we see, our finances to our career opportunities . Turning decision making entirely to algorithms raises concerns. How can we be sure the algorithmically curated news doesn’t have a political party bias or job listings don’t reflect a gender or racial bias. This paves a path to fully understand the algorithm in and out.

Outline:

  • What is model interpretability.
  • Why model interpretability is important.
  • Different types of model interpretability
  • Trade off between interpretability and accuracy.
  • Popular model-agnostic libraries and how do they differ
  • Key takeaways

Speaker bio

Data Scientist, avid reader & listener, loves engaging in meaningful conversations over a cup of coffee.

https://www.linkedin.com/in/namrata-hanspal-9613b0132/

Slides

https://docs.google.com/presentation/d/1oXIavmxMeKVXrbiaIVou2y7ftzmYhh6dZhErgC9625w/edit?usp=sharing

Comments

  • Zainab Bawa (@zainabbawa) Reviewer 2 months ago

    The slides are unclear, Namrata. Here are some questions I have:

    1. If model intepretability is importamt, is this a talk about the tools/libraries one can use for model intepretability?
    2. At what stage of the model’s evolution (or at the inception itself) do libraries play a role?
    3. What is the reliability of this approach?
    4. Will you show a comparison of Shap, Lime, DeepLIFT, relevance prop and path expectation for model intepretability? If yes, what are the criteria for comparison?
    5. What will participants at The Fifth Elephant learn from this talk? Are you trying to say something new? Are you demonstrating a new approach? Are you making participants aware of an anti-pattern?
    • Ashwin (@trds) a month ago

      hi Namrata
      in addition to the above questions, it is important to give an approach to modelling that includes a thinking-with-interpretability.

      it will be good to have an example that quickly runs through the most common variants, LIME, Shapley, ICE, Surrogate models.

      also mention what are the equivalents for a deep learning model/neural net. e.g. attention based models.

      ~ashwin

      • Namrata Hanspal (@namrata4) Proposer a month ago

        Sure Ashwin, I’ll update the details ASAP.

      • Namrata Hanspal (@namrata4) Proposer a month ago (edited a month ago)

        Hi Ashwin

        Certainly, there will be examples to showcase the popular libraries but not all.

        Libraries like SHAP are used for deep learning networks as well, I will be covering this in the talk. Attention based models like BERT are fairly new and hence the model interpretibility libraries haven’t updated to include but there is other visualisations which can be used to interpret the model.

    • Namrata Hanspal (@namrata4) Proposer a month ago (edited a month ago)

      Hi Zainab, will update the slides. Below are the answers to your questions:

      1.This talk intends to help build an intuition around why model interpretability is important and how we can interpret complex models which were blackbox to us until recently.
      2. Model building is an iterative process, very rarely do we feel satisfied with the first iteration. As we iterate apart from measuring model performance using metrics/methods like precision, recall, F1 score, ROC curve etc., it is also important to understand what has the model learnt. Because there can be a situation where the model is performing well but has learnt wrong features because of the nature of training data. The talk will explain this with examples(inception for example)
      Another very important significance of model interpretability is that we can explain feature importance which is very useful for specific domains/usecases.
      3.Model interpretibility works by modeling the results and interpreting the feature importance. Like all models, this also doesn’t gaurantee 100% accurate results but gives us good idea about what the model is learning.
      4.The talk aims to introduce difference libraries but the focus will be on explaining and showcasing how model interpretibility can help us build better models.
      5.Model interpretibility is a fairly new concept. Not all data scientist dig deep into understanding what their model is learning and are mostly focused on metric numbers. The talk aims to bust the supreme importance we give to performance metrics.

  • Venkateshan (@venky1729) a month ago

    Hi Namrata,

    While this is definitely an important topic, we need to distinguish between (a) model interpretability in the narrow sense of the relevance of input features (however defined; for example, local additive model) on the accuracy of model output and (b) interpretability in the sense of the broader context in which the model is being developed, what sort of data it is being trained on and the questions it is seeking to answer and the interpretation of those outputs. Ex: what sort of issues exist with using model that predicts probability of defaulting on loan while using race/religion/marital status as an input feature?

    If I am not mistaken all the methods you have proposed address (a) but they would have no bearing on grappling with (b). For instance, none of these methods would be able to shed light on causal associations (for the simple reason that statistical dependencies are what these models ultimately capture).

    Please be aware of this distinction when referring to interpretability.

    Venky

    • Namrata Hanspal (@namrata4) Proposer a month ago

      Hi Venkatesh,

      Sure, will take care of this distinction.

      • Abhishek Balaji (@booleanbalaji) Reviewer 16 days ago

        Hi Namrata,

        Please add the distintion in your slides as well. The overall feedback that we’ve received on this proposal is that it that the content is too thin. As next steps, I’m parking your proposal for evaluation under a future event if you choose to work on your slides. Unfortunately, we cannot consider this for The Fifth Elephant 2019 in Bangalore.

        We’re hosting a Birds of a Feather session on model interpretability at the conference. Will keep you posted on this.

Login with Twitter or Google to leave a comment