Customer Driven Development : What, Why & How?

Submitted Feb 6, 2019

Technical level: Advanced

Think of this talk as a crash course for Mobile Developers to become Product Owners.

It will teach developers how to efficiently write & setup experiments (A/B Tests) and hence showcase a way to create apps that keep the customers at the centre of everything.
If you attend this talk you will learn how to frame the hypothesis, calculate runtime of an experiment, set up metrics to measure success so that you can act as a product owner yourself in future!

Having run over a 100 experiments in the last 2 years at Booking.com (World’s biggest OTA running massive number of A/B Tests at scale) with an enormously high success rate, I’ll present some hard to believe facts related to user experience and design that developers often overlook while shipping features.

Outline

How often do you change the colour of a button on your app? How often do you change the layout of a part of the app? How do you measure the effect of these changes? Of course you do that with a hypothesis in mind related to some business goal you want to achieve. Do you actually always validate your hypothesis?
This talk will tell you how you can convert users into customers and customers into decision makers of your product vision.
By validating all your decisions in your product through data you can be sure what’s working and what’s not. Digging a little deeper you’ll also always find a cause about why it’s working and why it’s not. Taking a step further you can easily derive what is smallest next step you can take to ship a new feature/change in your product that might positively impact your customers.

The talk’s overview:

Introduction
About Me
Agenda of the talk
Introduction to Customer Driven Development
What is A/B testing?
Examples of A/B Tests
Basic Fundamentals of setting up a valuable A/B Test
What is a hypothesis in this context?
Basic formula for writing a hypothesis
Example : Share personal template that I use for crafting hypothesis.
What are Primary & Secondary metrics?
Examples of metrics
Choosing the (correct) primary metric
Examples of a metric that will work
Examples of a metric that will fail
Choosing appropriate supporting metrics
Examples of supporting metrics
What is run time of an experiment?
How to calculate the runtime of an experiment (Demo)?
Why is it important to having a runtime before starting an experiment?
What is noise in an experiment?
Example of an experiment’s data that had a lot of noise
How to eliminate noise by correctly tracking stages in your experiment?
Example of the same experiment that was run after a noise removal fix.
How to interpret results?
Avoiding Statistical Ghosts
How to plan follow-ups?
Questions & Answers
The End (Thank you)

Speaker bio

Ishan is a passionate product enthusiast and self-taught developer who loves open source technologies, tech conferences, and hackathons.
Currently working as a Senior Android Engineer at Getaround (Amsterdam, Netherlands) before that he spent 2 years crafting experiments for Booking.com on their Android Apps. He is also PMC member at Apache Fineract Project and a Maintainer at Mifos Initiative. He successfully graduated as a Google Summer of Code Intern in 2014 under Mifos and in 2015 under XMPP Standards Foundation and has been mentoring students at Google Summer of Code ever since.
https://www.linkedin.com/in/ishankhanna/
https://twitter.com/droidchef
https://github.com/droidchef

Call for round the year submissions for Fragments in 2019