Anthill Inside 2019

On infrastructure for AI and ML: from managing training data to data storage, cloud strategy and costs of developing ML models

Propose a session

Opening the Black Box: How to Interpret Machine Learning models; techniques, tools, and takeaways

Submitted by Farhat Habib (@distantfedora) on Tuesday, 30 April 2019


Preview video

Section: Workshops Technical level: Intermediate Session type: Workshop

Abstract

Interpretability of a model is the degree to which a human can consistently predict the model’s result. The higher the interpretability of a machine learning model, the easier it is to comprehend why certain decisions or predictions have been made. While interpretability is not important in low-risk domains and black box models abound, in domains such as medicine or finance, or high-risk domains such as self-driving cars, or weapons systems model interpretability is a strong requirement. As privacy preserving legislation such as GDPR becomes a norm across the globe, interpretability of models is important for explaining how particular recommendations or decisions were made. The more an ML model decision affects a person’s life, the more important it is for it to be interpretable. The training data fed to a model may have biases, inconsistencies, and other artifacts. Interpretability is a useful debugging tool for detecting bias in machine learning models.

Various interpretation methods are explained in depth. How do they work under the hood and their strengths and weaknesses? How can their outputs be interpreted? We will start with models that are interpretable easily such as linear regression and decision trees and then go on to interpretation methods that are model agnostic such as feature importance, local surrogate (LIME), and Shapley values (SHAP). In the traditionally inscrutable domain of deep learning, we will look at gradient based and attention based methods for interpreting deep neural nets.

Outline

  1. Importance of interpretability
  2. Stories of uninterpretable model failures
  3. Evaluation of interpretability
  4. Human-friendly explanations
  5. Interpretable models
    1. Linear and Logistic regression
    2. GLM and GAM
    3. Decision trees
  6. Model Agnostic Methods
    1. Partial Dependence Plots
    2. Feature Interaction
    3. Feature Importance
    4. Global and local surrogate (LIME)
    5. Shapley Values
  7. Interpretability of Deep Learning Models
    1. Gradient based methods
    2. Attention based methods
  8. Counterfactual explanations
  9. Adversarial examples

Requirements

  1. Basic knowledge of Machine Learning and Deep Learning
  2. Basic familiarity with Python and Jupyter notebooks
  3. User should have a working Jupyter setup on their laptops with internet access

Speaker bio

Farhat Habib is a Director in Data Sciences at Inmobi currently working on anti-fraud and improving creatives. Farhat has a PhD and MS in Physics from The Ohio State University and has been doing data science since before it was cool. Prior to Inmobi he worked on solving logistics challenges at Locus.sh. Before that he was at Inmobi working on improving ad targeting on mobile devices and prior to that he was at Indian Institute of Science Education and Research, Pune leading research on computational biology and genomic sequence analysis. Farhat enjoys working on a wide range of domains where solutions can be found by the application of machine learning and data science.

Aditya Patel is Director, Data Science at InMobi Glance. Previously he was head of data science at Stasis and has 7+ years of experience spanning over the fields of Machine Learning and Signal Processing. He graduated with Dual Master’s degree in Biomedical and Electrical Engineering from the University of Southern California. He has presented his work in Machine learning at multiple peer reviewed conference. He also contributed to first generation “Artificial Pancreas” project in Medtronic, Los Angeles.

Links

Preview video

https://photos.app.goo.gl/pmL1ESuBCJUySWBQ8

Comments

  • Abhishek Balaji (@booleanbalaji) Reviewer 2 months ago

    Hello Farhat,

    Thank you for submitting a proposal. To proceed with evaluation, we need to see detailed slides and a preview video for your proposal. Your slides must cover the following:

    • Problem statement/context, which the audience can relate to and understand. The problem statement has to be a problem (based on this context) that can be generalized for all.
    • What were the tools/options available in the market to solve this problem? How did you evaluate alternatives, and what metrics did you use for the evaluation?
    • Why did you pick the option that you did?
    • Explain how the situation was before the solution you picked/built and how it changed after implementing the solution you picked and built? Show before-after scenario comparisons & metrics.
    • What compromises/trade-offs did you have to make in this process?
    • What is the one takeaway that you want participants to go back with at the end of this talk? What is it that participants should learn/be cautious about when solving similar problems?
    • Is the tool free/open-source? If not, what can the audience takeaway from the talk?

    We need to see the updated slides on or before 21 May in order to close the decision on your proposal. If we do not receive an update by 21 May we’ll move the proposal for consideration at a future event.

    • Farhat Habib (@distantfedora) Proposer a month ago

      Hello Abhishek,

      Problem statement/context, which the audience can relate to and understand. The problem statement has to be a problem (based on this context) that can be generalized for all.

      Model interpretability is an important consideration in many fields. It is also starting to be a legal requirement in many regions especially where ML models are used to take decisions significantly affecting individuals such as crime prediction systems, financial aid and loans, medical decisions among others. Even more generally, understanding how a model works can point out issues with biases in training data, or provide ideas on which features may have a causal relationship to the target.

      Why did you pick the option that you did?

      We will cover a range of options to interpret models here, spending on some time on models that are easily interpretable, such as linear models or decision trees, and progressively going toward models that are harder to interpret such as random forests, gradient boosting, and finally deep neural nets. We will cover techniques such as LIME (Local Interpretable Model-agnostic Explanations) and the more recent SHAP (SHapley Additive exPlanations) which are model agnostic.

      Explain how the situation was before the solution you picked/built and how it changed after implementing the solution you picked and built? Show before-after scenario comparisons & metrics.

      N/A. We will talk about some cases where not understanding how the model worked led to some famous failures when they went into production or widespread use (many medical cases).

      What compromises/trade-offs did you have to make in this process?

      We will talk about tradeoffs between accuracy and interpretability and where it may be useful to trade one for the other.

      What is the one takeaway that you want participants to go back with at the end of this talk? What is it that participants should learn/be cautious about when solving similar problems?

      The participants should have a reasonably good idea of how their model is making predictions. In many domains it is not good enough to throw large amounts of data at a number of different ML techniques, tune, and pick the one with the best performance. Participants should be cautious about completely black box models even if they are accurate on the test dataset as they may fail in unforeseen ways.

      Is the tool free/open-source? If not, what can the audience takeaway from the talk?

      We plan to have a Jupyter notebook that the audience can work with us on as we go through the workshop. All datasets and code are free to use. We will attempt to have it working on Google colab so people can work with us even in the absence of a working Jupyter installation on their laptops. The current state of the notebook can be seen at https://github.com/Farhat/Model-interpretability-Anthill-Inside-2019

Login with Twitter or Google to leave a comment