Your First Brightback Experiment

Brightback offers a powerful platform with the ability to test retention offers and target users based on your canceler's user profile. One common question we get when new customers are starting is: "How should I go about setting up my first experiment?". It's a great question and, in short, the answer is to start simple. Starting with an overly complex and intricate setup can leave you scratching your head when it becomes time to analyze your phase 1 results. Increased complexity can also lead to delays in getting started as you'll need to curate and build content for each layer of complexity you add.

"How should I go about setting up my first experiment?

-anonymous Brightback customer

This article breaks out the learnings and best practices we've compiled from our many successful customer implementations. Let's start with a few dos and don'ts.

Dos and Don'ts

1. Create a single page experience

Use dynamic elements like the user's name and usage metrics such as activity or total orders, but don't worry about breaking out into more complex audience groups just yet.

Do: Create a single page to use as a template for all other pages. Use dynamic field insertion for personalization and appropriate messaging and artwork that is on-brand. Use this as your template which you can easily duplicate later when it's time to go live.

Don't: Create unique and specific static content that requires multiple different pages with curated content for highly segmented audiences.

2. Create 2-4 modal offers

While some thought can be put in the context of how the offer is presented in terms of messaging, we have found that it's best to test the offers against the profile of the canceller rather than trying to respond to the reason they select.

Do: Come up with 2-4 offers you are comfortable showing to all of your users for all reasons.

Don't: Create overly specific offers targeted based on the reason the user selects.

3. Add a loss aversion card offer

The loss aversion section is your first opportunity to present an offer to the user. Our recommended approach is to place an offer here with less incentive than the modal offers presented when reasons are clicked.

Do: Come up with a single lower incentive offer to place in the loss aversion section of your pages. e.g. A link to support, skip an order, etc.

Don't: Test multiple Loss aversion offers in combination with multiple modal offers. To make that work you'll have to increase the number of pages you have exponentially and dilute your traffic so much that results will take much longer to ascertain.

4. Setup your simple test and go live (don't forget your holdout page!)

Now all you need to do is set up your experiment by duplicating your template and placing the offers and loss aversion cards in the placement editor and target traffic.

Do: Duplicate your template as many times as needed based on how many offers you have created. Give them a meaningful full name to track them on the experience performance report. Copy the page one more time to use as a holdout.

Do: Place your loss aversion offer card on all of the pages you created.

Do: Edit the placement of your modal offers by applying them to all of the reasons on the page that matches the offer.

Do: Edit your targeting page to distribute traffic to your final set of experiences.

Don't: Leave out the no offer holdout page!

Now you have a simple set of offers and experiences you can track and analyze once you start sending traffic. For this final step head over to our testing and targeting page to learn how the routing tree and traffic split works.