Run an A/B test

An A/B test shows one version of some content to part of your visitors and another version to the rest, eg two hero banners or two product names. In Vox, an experiment is a context type: its members are the variants, each variant holds only the values it changes, and Vox splits visitors between the variants. When you have a winner, you promote it into the published content.

Before you start

You need a Vox application with the entity types you want to test, eg a campaign page or a product, and a site or app that reads them through Vox. Measuring which variant wins is done in your analytics tool.

How it works

A variant is a member of a concurrent context type. The variants of one experiment form a group, and a visitor is in at most one variant of each experiment. Several experiments can run at the same time.
Each visitor gets a bucket from 1 to 100 per experiment, calculated from the visitor's id. A variant's selection criteria pick a range of buckets, eg 1–50 for hero-a and 51–100 for hero-b.
A variant's values are cells keyed by the variant, like the values of a market, eg {Variant=hero-b} or {Variant=hero-b|Language=sv}. A visitor in hero-b sees those values, and the published values for everything the variant doesn't set.
Promoting the winner writes its values into the published content in one transaction and deletes the variant.

Set up an experiment

Add the context type. A variant is an entity that implements IVoxContext, with the id of its experiment:

Variant.cs
public class Variant : IVoxEntity, IVoxContext
{
    // Built-in entity properties omitted.
    public required string Id { get; set; }            // eg "hero-b"
    public required string ExperimentId { get; set; }  // eg "hero"
}

Configure it as concurrent and selected by criteria. The group is the experiment, so the variants of one experiment share the visitor's bucket:
Program.cs
```
services.AddVox(options =>
{
    options.ConfigureContextType<Variant>(type => type
        .Inherit()
        .SelectedByCriteria()
        .Concurrent(variant => variant.ExperimentId));
});
```
Inherit() means an entity exists in every variant unless the variant says otherwise, so a variant only has to hold the values it changes.
Create the variants and split the visitors. Create one variant per version and give each a range of buckets in its selection criteria. In the admin you can edit the criteria as conditions or as JSON and test them. In JSON, the range ends before to:
```
{ "between": { "input": "bucket", "from": 1, "to": 51 } }
```
```
{ "between": { "input": "bucket", "from": 51, "to": 101 } }
```
The criteria can also read the request, the time and the other selected contexts, eg to run the experiment only in Sweden. See Portable contexts.
Set the values of each variant. Select the variant in the admin and edit the entities as usual, eg a new headline. For entity types with drafts, you edit the variant in a draft and can preview it on your site before it goes live. Only the values you change are stored at the variant.
Send visitors through Vox. Vox reads the visitor's id from the vox-visitor cookie and, when it's missing, sets it with a new id for a year. The id decides the visitor's bucket, so a visitor keeps seeing the same variant. A system that selects contexts outside Vox, eg a CDN, can select the same variants from the context manifest, see Portable contexts.

Promote the winner

When your analytics tell you which variant won, promote it with Promote on the variant in the admin, or in code:

await promoter.PromoteAsync<Variant>("hero-b", expectedVersion: variant.Version, cancellationToken);

In one transaction, Vox writes the values of the variant where they'd be without it, eg {Variant=hero-b} into the published entity and {Variant=hero-b|Language=sv} into its Swedish values, and deletes the variant. Delete the losing variants to delete their values.

A variant with unpublished drafts of its values isn't promoted until they're published or reverted.

How it works​

Set up an experiment​

Promote the winner​

Next steps​

How it works

Set up an experiment

Promote the winner

Next steps