The Fifth Elephant 2019

Gathering of 1000+ practitioners from the data ecosystem

Tickets

The final stage of grief (about bad data) is acceptance

Submitted by Chris Stucchio (@stucchio) on Friday, 10 May 2019

Session type: Full talk of 40 mins

View proposal in schedule

Abstract

Over the course of my career I’ve gone through the many stages of grief; I’ve become angry at the poor quality of my data, I’ve attempted to bargain with engineering/PMs/etc for better data, and I became depressed over the issue. Now I’ve reached the final stage; I accept that my data is bad. Given that my data is bad, I then attempt to model it’s badness, and use that model to correct for the biases introduced.

In this talk I’ll discuss how I approach bad data; I accept that I cannot fix it and instead try to model where it came from. This usually involves getting a more detailed grasp of the data generating process and writing down a formal model.

In many cases this enables me to use the data model to correct and enhance my predictive model, as well as provide useful measurements and insights for improving and repairing the data collection process.

Outline

This talk is about bad data, and how to deal with it. It is NOT about improving data collection, correcting broken data, monitoring, etc.

To start with I’ll discuss the problem of data that is collected incorrectly, and focus on a couple of examples:

  • Data that is randomly missing, perhaps due to a malfunctioning tracking pixel.
  • Data that is mislabelled, perhaps due to data collection partners who use slightly different processes to collect it.

I’ll discuss particular fixes for these, both to correct for biases introduced by incorrect data, and to understand how bad the data collection is.

Then I’ll move on to data which is fundamentally bad.

The first example I’ll cover is delayed reactions. When measuring ad clicks, the time between display and click is nearly instantaneous (minutes at most). When measuring clicks on links contained in an email, the time can be quite significant (days). The same is true for many relevant scenarios, including debt collection (e.g. at Simpl it takes 30 days to know if a user is delinquent).

I’ll discuss a technique for modeling and correcting for the bias introduced by delays, which comes by modeling the delay via survival models.

The second example I’ll cover is selection bias caused by using your model. In particular, I’ll show discuss why height appears to be uncorrelated with player performance in the National Basketball Association and why GRE does not seem to predict academic performance among students admitted to graduate school.

The conclusion I want everyone to take away from this is that bad data is not a show stopper. It’s also not something you’re helpless to do anything about. Rather, it’s an obstacle, but one that can be overcome (or at least mitigated) with careful modeling.

Requirements

It would be useful to be familiar with Bayes rule and a bit of linear algebra.

Speaker bio

Chris is currently the head of data science at Simpl, India’s top Pay Later platform. In past lives he’s been a physicist, a high frequency stock trader, an automated marketer, a bodyguard and a nootropic drug courier. He’s a strong believer in correct statistics, clean code, and putting skin in the game to demonstrate your beliefs.

Links

Slides

https://www.chrisstucchio.com/pubs/slides/fifth_elephant_2019/bad_data.pdf

Preview video

https://youtu.be/BVq7uOulOWc

Comments

  •   Squad tech (@geeksquad21) 7 months ago

    This is really an amazing website.i want to be the participant of this blog.Keep in touch.
    if you have any queries related to your hardware devices like more mobiles, laptop then easily contact our page Geek squad support we provide the best solution for our customer.
    visit our website:
    https://squadtechsupport.us/

  •   emma jackson 3 months ago (edited 3 months ago)

    Its a great pleasure reading your article post.Its full of information I am looking for and I love to post a comment that “The content of your post is awesome” Great work.
    note: https://bit.ly/32bPue6

  •   Alessia jackson (@quickbooks01) 2 months ago

    Good post. I learn something totally new and challenging on blogs
    Thanks for such a good articles it really helped me after reading your posts.
    visit: http://quicksupports.co.uk

  •   norton setup (@nortonsetup90) a month ago

    we are impressed by the details that you have on your site
    There’s certainly a great deal to know about this topic.
    notes: https://i-nortonsetup.info/

  •   Harry jones (@harry01) a month ago

    Its a very valuable Knowledge On this topic.Mark this Post to future reference. https://mcafee-activate-product-key.org/

  •   sando john (@officesetup11) a month ago

    All of the information shared by you is unique for me and I know you have many different ideas to share with us. https://gooff.org/

  •   Shirley world (@quickbooks2247) a month ago

    we are impressed by the details that you have on your site
    There’s certainly a great deal to know about this topic. https://247qb.org/

  •   jones williamson (@geeksquad565) a month ago

    Hey, Excellent work. I must appreciate the way you manage your article’s content and the look & feel of your website. you are really doing a great job, carry on the hard work. http://mygeeky.org/

  •   Quickbook Support (@productactivate) 24 days ago

    There’s certainly a great deal to know about this topic.
    we are impressed by the details that you have on your site.
    https://www.product-activate.com/

  •   quickbookssupport (@quickbookssupport14) 8 days ago

    Thanks for such a good articles it really helped me after reading your posts. https://goaccounts.co/quickbooks-desktop-support/

Login with Twitter or Google to leave a comment