Think of this talk as a crash course for Mobile Developers to become Product Owners.
It will teach developers how to efficiently write & setup experiments (A/B Tests) and hence showcase a way to create apps that keep the customers at the centre of everything.
If you attend this talk you will learn how to frame the hypothesis, calculate runtime of an experiment, set up metrics to measure success so that you can act as a product owner yourself in future!
Having run over a 100 experiments in the last 2 years at Booking.com (World’s biggest OTA running massive number of A/B Tests at scale) with an enormously high success rate, I’ll present some hard to believe facts related to user experience and design that developers often overlook while shipping features.
How often do you change the colour of a button on your app? How often do you change the layout of a part of the app? How do you measure the effect of these changes? Of course you do that with a hypothesis in mind related to some business goal you want to achieve. Do you actually always validate your hypothesis?
This talk will tell you how you can convert users into customers and customers into decision makers of your product vision.
By validating all your decisions in your product through data you can be sure what’s working and what’s not. Digging a little deeper you’ll also always find a cause about why it’s working and why it’s not. Taking a step further you can easily derive what is smallest next step you can take to ship a new feature/change in your product that might positively impact your customers.
The talk’s overview:
- Introduction
- About Me
- Agenda of the talk
- Introduction to Customer Driven Development
- What is A/B testing?
- Examples of A/B Tests
- Basic Fundamentals of setting up a valuable A/B Test
- What is a hypothesis in this context?
- Basic formula for writing a hypothesis
- Example : Share personal template that I use for crafting hypothesis.
- What are Primary & Secondary metrics?
- Examples of metrics
- Choosing the (correct) primary metric
- Examples of a metric that will work
- Examples of a metric that will fail
- Choosing appropriate supporting metrics
- Examples of supporting metrics
- What is run time of an experiment?
- How to calculate the runtime of an experiment (Demo)?
- Why is it important to having a runtime before starting an experiment?
- What is noise in an experiment?
- Example of an experiment’s data that had a lot of noise
- How to eliminate noise by correctly tracking stages in your experiment?
- Example of the same experiment that was run after a noise removal fix.
- How to interpret results?
- Avoiding Statistical Ghosts
- How to plan follow-ups?
- Questions & Answers
- The End (Thank you)
Ishan is a passionate product enthusiast and self-taught developer who loves open source technologies, tech conferences, and hackathons.
Currently working as a Senior Android Engineer at Getaround (Amsterdam, Netherlands) before that he spent 2 years crafting experiments for Booking.com on their Android Apps. He is also PMC member at Apache Fineract Project and a Maintainer at Mifos Initiative. He successfully graduated as a Google Summer of Code Intern in 2014 under Mifos and in 2015 under XMPP Standards Foundation and has been mentoring students at Google Summer of Code ever since.
https://www.linkedin.com/in/ishankhanna/
https://twitter.com/droidchef
https://github.com/droidchef
- Speaking Experience:
-
- GDG DevFest in Vienna, Austria (November, 2018)
-
- AppsConf in Moscow, Russia (October, 2018) https://youtu.be/WF5Iigx_wQo
-
- DROIDKAIGI in Tokyo, Japan (February, 2018) https://youtu.be/-flI_-NvvTI
-
- DROIDCON in Boston, USA (April, 2017) https://news.realm.io/news/roasting-google-apis-on-a-firebase-to-create-uberritos/
-
- DROIDCON in Turin, Italy (April, 2017) - https://youtu.be/2dt-bK1KQ60
-
- FOSSASIA in Singapore, Singapore (March, 2016) - https://youtu.be/8ndhN-9k38k
-
- Mifos Tech Days in Amsterdam, Netherlands (March, 2016)
-
- GSOC Meetup by Google Developers Group in Delhi, India (March, 2016)
-
- GSOC Meetup at NSIT, Delhi, India (January, 2016),
-
- DROIDCON in Bangalore,India (December, 2015)
-
- Global Mifos Summit in Sharjah, U.A.E (March, 2015)
-
- GSOC Meetup in Delhi Technological University (January, 2015)
-
- GSOC Meetup in Jamia Milia Islamia and Jaypee Institute of Information Technology (2014)
{{ gettext('Login to leave a comment') }}
{{ gettext('Post a comment…') }}{{ errorMsg }}
{{ gettext('No comments posted yet') }}