Anthill Inside 2019

On infrastructure for AI and ML: from managing training data to data storage, cloud strategy and costs of developing ML models

Using AI for improving performance and design of ad creatives

Submitted by Farhat Habib (@distantfedora) on Tuesday, 30 April 2019


Preview video

Technical level: Intermediate Session type: Lecture Section: Crisp talk

Abstract

The ad creative is the image of the ad that the user sees on their device. It includes information such as the image dimensions, its placement on the screen or within the app, the native resolution of the creative and the mobile screen it is displayed on and similar parameters. Why exactly a particular ad is visually appealing and another not is hard to determine. Using data from the performance of hundreds of thousands of creatives that Inmobi displays, we model the performance of an ad creative based on the objects in a creative and their relative positions and sizes. We use state of the art convolutional neural nets to detect objects, text, and logos in creatives and their positions. Our model is able to provide actionable insights to creative designers on how to modify the creative to improve its performance. In general, our approach would be useful for understanding how the composition of an image relates to any engagement metric chosen by a researcher or business.

Outline

  1. Description of a creative
  2. Examples
  3. Measuring performance of a creative
  4. Object detection using CNNs
  5. Various object detection models
  6. Insights from data exploration
  7. Modeling
  8. Interpreting the model
  9. Take home messages

Speaker bio

Farhat Habib is a Director in Data Sciences at Inmobi currently working on improving creatives and anti-fraud. Farhat has a PhD and MS in Physics from The Ohio State University and has been doing data science before it was cool. Prior to Inmobi he worked on solving logistics challenges at Locus.sh. Before that he was at Inmobi working on improving ad targeting on mobile devices and prior to that he was at Indian Institute of Science Education and Research, Pune leading research on genomic sequence analysis and computational biology and bioinformatics. Farhat enjoys working on a wide range of domains where solutions can be found by the application of machine learning and data science.

Slides

https://www.slideshare.net/secret/44rMgC8dKciwFd

Preview video

https://photos.app.goo.gl/pmL1ESuBCJUySWBQ8

Comments

  • Abhishek Balaji (@booleanbalaji) Reviewer a month ago

    Hello Farhat,

    Thank you for submitting a proposal. To proceed with evaluation, we need to see detailed slides and a preview video for your proposal. Your slides must cover the following:

    • Problem statement/context, which the audience can relate to and understand. The problem statement has to be a problem (based on this context) that can be generalized for all.
    • What were the tools/options available in the market to solve this problem? How did you evaluate alternatives, and what metrics did you use for the evaluation?
    • Why did you pick the option that you did?
    • Explain how the situation was before the solution you picked/built and how it changed after implementing the solution you picked and built? Show before-after scenario comparisons & metrics.
    • What compromises/trade-offs did you have to make in this process?
    • What is the one takeaway that you want participants to go back with at the end of this talk? What is it that participants should learn/be cautious about when solving similar problems?
    • Is the tool free/open-source? If not, what can the audience takeaway from the talk?

    We need to see the updated slides on or before 21 May in order to close the decision on your proposal. If we do not receive an update by 21 May we’ll move the proposal for consideration at a future event.

  • Farhat Habib (@distantfedora) Proposer 29 days ago

    Hello Abhishek,

    Problem statement/context, which the audience can relate to and understand. The problem statement has to be a problem (based on this context) that can be generalized for all.

    In this talk we will show how AI can be used to improve the design of creatives. A creative is the actual ad image that a user sees on their mobile. Inmobi has performance data from hundreds of thousands of creatives. Can we derive insights about what works best for different users? Are there regional, cultural differences in how creative performance varies?

    What were the tools/options available in the market to solve this problem? How did you evaluate alternatives, and what metrics did you use for the evaluation?

    We used pretrained models such as resnet50 for object detection in creatives. Others such as YOLOv3 were also considered. For modelling, multiple approaches were considered but interpretability was an important consideration so we worked with tree based methods. Will go into some of the approaches that did not work as well.

    Why did you pick the option that you did?

    For object detection speed/compute resources was a concern as we had over a hundred thousand creatives. Also, coverage in terms of number of objects detected. We manually looked at a number of cases where no objects were detected or a high number of objects were detected to see if object detection is working correctly. For the model, interpretability was important as we needed to communicate to creative designers and take their feedback regarding the findings.

    Explain how the situation was before the solution you picked/built and how it changed after implementing the solution you picked and built? Show before-after scenario comparisons & metrics.

    Creative design is, by its nature, not considered very amenable to ML approaches. Designers have strong opinions on what should go where. Moreover, brands often restrict how they may be placed. With data driven insights, we could speed up generation of creatives for smaller advertisers that do not have in house design teams. Also, given a creative, we could estimate its performance before putting it online. This saves time and budget that is spent learning how a creative performs before it is scaled to a larger user base.

    What compromises/trade-offs did you have to make in this process?

    Number of objects that could be reliably detected was an issue especially for smaller creatives. We ended up restricting object types, to humans and non-human objects. For human objects, further work is underway to see how the person placement affects creative performance. For animated gifs, further compromises had to be made as due to file size restrictions, they tended to not have as much detail, and multiple frames made it necessary to deal with them differently.

    What is the one takeaway that you want participants to go back with at the end of this talk? What is it that participants should learn/be cautious about when solving similar problems?

    I would like the participants to look at image analysis in a more holistic manner. The response of an image (however quantified) varies with who is looking at it, from which medium (same creative could have a different performance based on whether someone is on a tablet, high end mobile phone, or the phone is held in portrait or landscape model). On caution, I would say look at anomalous data points, visualize features as much as you can. You will find things that standard approaches miss. We found mobiles being considered as remotes and other misclassifications. Consider what you would do about cases such as objects shown inside a TV in a TV ad being detected independently.

    Is the tool free/open-source? If not, what can the audience takeaway from the talk?
    The tools used for analysis are largely open source. All data is proprietary to Inmobi.

    • Abhishek Balaji (@booleanbalaji) Reviewer 21 days ago (edited 21 days ago)

      Hi Farhat,

      Thanks for the responses. These points need to be incorporated into your slides. Make sure that your slides cover the above points.

      • Farhat Habib (@distantfedora) Proposer 20 days ago

        Hi Abhishek,

        Definitely. All points will be included in the slides and the talk.

Login with Twitter or Google to leave a comment