Creating A/B Tests

A/B Testing in Playbooks requires 2 Salesforce fields, 2 Plays, and 2 Robots. The four parts below breakdown the order and how the different components relate to each other. Setting up an A/B Test requires some effort at the beginning, but once launched the process is automatic.

Part 1: Auto-Number Records

The first step of A/B Testing is to auto-number records. This can be done by either using the existing Salesforce record ID or creating a new custom field that auto numbers records. This field will be referenced in Part 2. A XANT Help Center article is available with instructions on how to set up a new custom Auto-Number field.

Part 2: Create an “AB Test” Formula Field in Salesforce 

The next step is to create a formula field in Salesforce that labels records as either A or B based on whether the ID number (part 1) is odd or even. This field will be referenced in part 4. The instructions and formula for this field can be copied from the same XANT Help Center article mentioned above.

Part 3: Create the A/B Plays in Playbooks

Set up Plays to compare the different elements you would like to test (email templates, Play Steps, phone call talk tracks, etc.). Then label these Plays to indicate which is which (i.e., Inbound A/B Test – Play A and Inbound A/B Test – Play B). The Plays will be referenced in Part 4. Be sure to only change one thing between the two Plays. Having too many changes between the Plays will muddle the data making it impossible to determine which change is responsible for changing the outcome.

Part 4: Create the A/B Robots in Playbooks

Finally, create 2 Robots.  In the filter criteria of each Robot, use the new “AB Test” formula field you created in Part 2 to identify A records or B records. Then, enroll records into the corresponding Plays that were created in Part 3.

This whole process is ‘behind-the-scenes’ and should not be visible to Reps. Reps should continue working as usual through Play tasks and marking Success in Playbooks when the objective of the Play is met. The next lesson will discuss how success is measured and reports that can validate the testing data.

Related Article: Playbooks Solution for A/B Testing
We’ll cover the overall process to create an A/B Test in this course, but the full steps to complete the process and easy copy formulas are in this article. We suggest having this article open to follow along.