How to Beat Facebook's A/B Testing Algorithms

At the end of 2019, Consumer Acquisition ran a test of 60 original, fully-animated videos for a gaming client. All 60 failed to beat the control’s installs per mille (IPM). Concerned, the folks at Consumer Acquisition ran an A/A test to determine if their growth hacking methodology was valid. The results might surprise you.

In this webinar, Consumer Acquisition shares some critical discoveries regarding creative testing on Facebook; most significantly, that Facebook maintains creative history for control videos, meaning controls included in A/B testing have an innate advantage and will outperform others. Since Facebook’s algorithm visually groups similar concepts, creative that looks different from the control does not share its history, resulting in lower IPMs.

Based on these revelations, Consumer Acquisition made some changes to its testing methodology. Here are the three phases the team implemented to improve creative success by 300%:

Phase One: IPM Testing — Consumer Acquisition shares several tips for updating your IPM testing, including the importance of clean test campaigns, the money-saving potential of test-proxies, culling in-app ad titles, and picking IPM winners.
Phase Two: Initial ROAS Testing — Once winning ads are selected, it’s time to test the return on ad spend (ROAS). Consumer Acquisition advises creating a new campaign with either Facebook app event optimization (AEO) or value optimization (VO), and once more keeping the winning control video out of the mix. Watch the webinar for more on selecting audience proxies, lifetime budget, and potential ROAS concerns.
Phase Three: ROAS Scale — To determine whether or not your ad spend will scale, Consumer Acquisition encourages strong campaign budget optimization (CBO). The webinar walks you through testing Phase 2 winners, oscillating the CBO budget between groups, and allowing an ad to exit the learning phase before challenging the control video.

For more insights into A/B testing on Facebook and actionable tactics designed to improve your success rates, check out the full Consumer Acquisition Summer Update.