7 ways you are doing your A/B testing wrong by Côme Courteault

14
#TFWORKSHOP 7 WAYS YOU ARE DOING YOUR A/B TESTING WRONG CÔME COURTEAULT GROWTH HACKER AT THEFAMILY @C2PRODS

Transcript of 7 ways you are doing your A/B testing wrong by Côme Courteault

Page 1: 7 ways you are doing your A/B testing wrong by Côme Courteault

#TFWORKSHOP

7 WAYS YOU ARE DOING YOUR A/B TESTING WRONG

CÔME COURTEAULT GROWTH HACKER AT THEFAMILY

@C2PRODS

Page 2: 7 ways you are doing your A/B testing wrong by Côme Courteault

A/B TESTING

Page 3: 7 ways you are doing your A/B testing wrong by Côme Courteault

A/B TESTING

Page 4: 7 ways you are doing your A/B testing wrong by Côme Courteault

#1 NOT ENOUGH PEOPLE

You must A/B test on a lot of people

Check the χ² test(evanmiller.org/ab-testing/chi-squared.html)

Page 5: 7 ways you are doing your A/B testing wrong by Côme Courteault

GOOD

BAD

Page 6: 7 ways you are doing your A/B testing wrong by Côme Courteault

#2 NOT BEING RADICAL

Don’t start by testing shades ofred for the uppermost button!In the beginning, focus on

changes that have impact.

Page 7: 7 ways you are doing your A/B testing wrong by Côme Courteault

#3 TIME 2 days are not enough. 2 months are too much.

Run your tests for one week.

Page 8: 7 ways you are doing your A/B testing wrong by Côme Courteault

#4 NOT PAYING ATTENTION TO THE CONTEXT

Shit happens in the world. Holidays, too.Pay attention to what’s going on, this can

have an effect on the way your users behave.

Page 9: 7 ways you are doing your A/B testing wrong by Côme Courteault

#5 NOT HAVING AN HYPOTHESIS

Don’t run tests just because you can.Think of an hypothesis. Why are

you running this test?

Page 10: 7 ways you are doing your A/B testing wrong by Côme Courteault

#6 NO FOLLOW-UP TESTS

Double down on what works.Kill what doesn’t.

A/B testing is never over!

Page 11: 7 ways you are doing your A/B testing wrong by Côme Courteault

#7 IGNORING SMALL GAINS

A/B testing doesn’t always bring a 1000% increase!

Even a 10% gain on 100 000 users can prove worthy ;-)

Page 12: 7 ways you are doing your A/B testing wrong by Côme Courteault

PLEASE Build an awesome product before

running A/B tests.But after, use A/B testing to improve

your onboarding and your retention.Run multivariate tests only when you can (i.e. when you have bandwidth)

Don’t forget to care personally!

Page 13: 7 ways you are doing your A/B testing wrong by Côme Courteault

LET’S TALK!

Page 14: 7 ways you are doing your A/B testing wrong by Côme Courteault

SEE YOU SOON! LOVE.

THEFAMILY.CO ___

___