Skip to main content
A/B Testing lets you compare your live cart against a draft cart variant with real shopper traffic. Upcart automatically splits incoming traffic between the two variants based on your chosen allocation, tracks per-variant analytics, and lets you declare a winner when you’re ready.

​
How It Works

Each A/B test is made up of:
  • Variant A - your current live cart (automatically selected, cannot be changed)
  • Variant B - one draft cart of your choosing
  • Traffic allocation - a configurable split between the two variants (1-99% each, must sum to 100%)
  • Test name - up to 255 characters; use something descriptive like “Free Shipping Bar - March Test”
Once the test is published, incoming shoppers are routed to one of the two variants. Each variant collects its own impressions, conversion, and revenue data. When you’re ready, you declare a winner and that variant becomes the new live cart.

​
Setting Up a Test

  1. Go to Upcart > Carts
  2. Make sure you have at least one live cart and one draft cart. The Create Test button only appears when both exist.
  3. Click Create Test
  4. Enter a test name
  5. Select the draft cart you want to test against your live cart
  6. Set your traffic allocation (defaults to 50/50)
  7. Click Publish to start the test
The traffic allocation and cart settings are locked once the test is live. If you need to make changes, end the current test and start a new one.

​
Viewing Test Analytics

Once your test is running, per-variant analytics are available in the Analytics tab. The four KPIs reported for each variant are:
Analytics are aggregated by a daily job. If you just started a test, you will see dashes instead of numbers for the first ~24 hours. This is expected and not a bug. Numbers will appear within 24 hours of the test starting.
A progress bar in the Analytics tab shows how close you are to the recommended number of impressions for statistical significance. Once you reach that threshold, you have enough data to confidently declare a winner.

​
Ending a Test and Declaring a Winner

  1. Go to Upcart > Carts and click End Test
  2. Select the winning variant
  3. Click Confirm
The winning variant is published as the new live cart with 100% of traffic. The losing variant remains as a draft.
There is no undo on a winner declaration. If you select the wrong variant, you will need to manually republish the previous cart from your drafts.

​
Frequently Asked Questions

No. Only one active test is allowed per store at a time. You must end the current test before starting a new one.
No. A/B Testing always tests your live cart against one draft cart. The live cart is automatically selected and cannot be changed.
No. A/B Testing supports exactly two variants. Multivariate testing is not currently supported in Upcart.
No. The traffic allocation is locked once the test is published. To change it, end the current test and start a new one with the desired split.
No. Both the test configuration and the cart editor settings are locked while a test is live. End the test to make changes.
Analytics are aggregated by a daily job. Brand new tests show dashes until the first aggregation run completes, which typically happens within 24 hours of the test starting. This is expected behavior and not a bug.
No. Each shopper is assigned to one variant and stays in that variant for the duration of their browser session.
No. Variant assignment only matters when the cart is shown. Shoppers who never trigger the cart are not counted in impressions.
No. Revenue is summed across all transaction currencies without conversion. If your store sells in multiple currencies, interpret the Added Revenue number accordingly.
No. Upcart shows per-variant metrics and a progress bar toward statistical significance, but you make the final call. There is no automatic winner selection.
No. Once a winner is declared, that variant becomes the live cart immediately. To revert, you would need to manually republish the previous cart from your drafts.
The losing variant remains as a draft cart. It is not deleted.
Yes. The Analytics tab shows both your current live test and previous completed tests side by side.
Need help? If you have questions about setting up or running an A/B test, reach out to our support team via the in-app chat.