Meta Refresh 2016

For someone new to product development, it looks like most products are built on a hunch. Someone in the office who happened to be the one calling shots said that the delivery address should be taken after the screen where the customer can increase/decrease order quantity. Is a hunch enough here? Can product folks do better than take hunches or copying what their biggest rival is doing? Data driven product development is the answer.

By default, most projects work in an iterative development cycle. It’s difficult to launch a product with the 100% of features that you have planned for it. After all, if you are not embarrassed by the first version of your product, you’ve launched too late.

So you launch with the basic 3 features, let’s say, and then start working on 2 new features for the next version. In between you get bug reports that you need to fix. You dig in deep into your app logs and figure out what’s the issue. Also, turns out that the 3rd feature you had released is not producing the expected results. So you now dig into your analytics systems to see how are people using the feature, at which point are they deviating from expected path etc. You make necessary changes in the feature and release v2. But this release was buggy. The latest updates rendered one route slightly more error prone and this got detected by your error reporting and alerting tool. You open up vim, fix the issue, run your deployment process and voila, you have v3 out in the market.

Take any product out there and the above will be true. Right from Prisma that just released an app for Android and has already updated it multiple times to Tesla releasing and iterating on its Autopilot feature.

While I have mentioned a bunch of tools, this talk will be more inclined towards how you, as a product developer, can start asking the right questions which can then be answered by leveraging data. I will walk you through the process of tracking efficiently what your users are doing and how are they using a feature. The talk will also include topics like coffee shop testing and how to do it the right way.

Outline

• Pick a feature, like Instagram Stories or maybe a feature from one of Razorpay’s product
• Define what would be right product questions to ask for a feature like this. For eg:
  • How many people use this feature?
  • Does this feature work as expected?
  • Have we even defined what would be classified as success for this feature?
  • How much of a success is it really? Can it be quantified?
  • What would make it a failed feature?
  • What information would you need to debug the feature if it doesn’t work as expected?
  • etc.
• Define what would be the best metrics to track
• How to analyse the data?
• How to find patterns in data?
• Stepping out of data into the real world
  • Office wide testing
  • Coffee shop testing
• Iterate, track, modify, iterate

Speaker bio

Shashank is a core team member at Razorpay. Primarily a techie, he is the go to guy for all things product. Lately he has been focussing on driving product decisions through data, which also happens to be the premise of his talk. In the process he ended setting up verbose data logging systems to the extent that Razorpay can pinpoint exactly where the customer dropped off in a payment flow.

Links

• https://shashankmehta.in
• https://twitter.com/leostatic
• https://github.com/shashankmehta