Creating an Experiment (A/B Test)

TNL allows you to run experiments (A/B tests) on any content within your app. You can test page designs, block variations, and measure performance based on defined goals.

Follow the steps below to set up your first experiment.

Step 1: Navigate to Experiments

Go to the Marketing menu
Select Experiments
Click Create Experiment

Step 2: Enter Basic Information

Provide the following details:

Experiment Name
Description (What are you testing?)

If running multiple experiments, it’s recommended to include a hypothesis, such as:

We believe changing the hero image will increase conversions because it better highlights our primary offer.

Documenting a hypothesis helps guide analysis and future decisions.

Step 3: Choose Experiment Type

You have two options:

Page-Level Experiment

Test different full page designs.

Select the page you want to test
Any variants created will replace this page for users in the experiment

Click Next after selecting your target page.

Block-Level Experiment

Test variations of a specific block within a page.

This is useful for testing:

Headlines
Images
CTAs
Layout variations

Step 4: Configure Control & Variants

Control

Add a description explaining the current version

Variants

Select the page or block variation you want to test
Add a description explaining what is different

By default:

Traffic is split equally between variants
If you add more variants, traffic is distributed evenly

Step 5: Define Goals

Set measurable goals to determine experiment success.

Goal Types

Choose from:

Conversion
Engagement
Revenue
Custom Event

Event Types

Depending on goal type, available events may include:

Page view
Button click
Scroll depth
Time on page
Conversion event
Custom event

You may create multiple goals.

Select one goal as the Primary Goal to determine the winning variant.

Step 6: Audience Targeting

Choose who should see the experiment.

Targeting Options

All users
New users only
Returning users
Logged-in users
Anonymous users

Platform Targeting

iOS
Android
Both

Step 7: Traffic Allocation

Determine how traffic is distributed across variants.

Options include:

Equal split (default)
Custom weights
Multi-Arm Bandit (auto-optimizes allocation toward higher-performing variants)

Step 8: Traffic Percentage

Set the percentage of eligible users to include in the experiment.

Example:

100% = all eligible users
50% = half of eligible users

Step 9: Statistical Settings

Configure experiment thresholds:

Minimum Sample Size
Confidence Level (percentage required to declare a winner)

You may also enable:

Auto Promotion

Automatically promote the winning variant once statistical criteria are met.

Step 10: Review & Launch

On the final screen:

Review all experiment details
Confirm configuration
Click Create Experiment

Your experiment will now begin running according to your defined settings.