Split Testing for E-Sellers


With almost 500 million products on Amazon, it’s hard to stand out from the crowd. Split testing is one of the most effective means to optimize your listings and increase engagement and sales on your products.


What split testing is

If you’re new to split testing (also called A/B testing), it’s essentially an experiment between two or more variations to see which variation in the set performs best. These variations might be different product names, featured photos, or product descriptions


A traditional split test directs a portion of your traffic to one test variation and another portion of your traffic to a different variation. Did one set of visitors stay on your listing for longer? Did one set convert at a higher rate?


How to run split tests

On your own e-commerce website, you can split traffic to all your variations simultaneously. You can write your own code to direct traffic to the correct pages, or you can use landing page software such as Optimizely or Instapage, among others.


Another method is to split traffic using pay-per-click ads. You can run an A/B test in Facebook Ads or Google Ads so that clicks go to different destination URLs. If you use this method, use identical ad creative to eliminate the ad content as a variable. Make sure your campaign is set to rotate the creative evenly; otherwise, the algorithms will favor one ad and serve it up the most.


On a marketplace like Amazon, splitting traffic is trickier to do. You’ll need to rotate the variants over time and compare the results. You might, for example, test Version A for the first three days of the week. Then at midnight on the third day, you update your listing to Version B and monitor it for the next three days of the week. Of course, this example reveals a limitation of this method: shoppers may behave differently on different days of the week. You’ll need to take factors like this into account when you analyze your results. Other factors might include inventory levels, sales or promotions, and traffic levels. 


If you don’t want to rotate your variations manually, you can use Amazon A/B testing software to automate the rotation. Each program has its own method of rotating and determining a winner, so try to understand how the algorithms are calculated before you begin testing. 


Split Testing Seller's Choice


What to split test

For e-sellers, the main elements to split test are

1. The product title

2. The listing’s featured photo

3. The product description


Using live tests, you can also experiment with back-end keywords and pricing strategies to see whether they affect sales or rankings.


Limitations of traditional split tests

We already discussed one limitation of traditional split tests, which is that shoppers might behave differently on different days of the week. But there are other shortcomings as well. 

  • In order to gather enough data to make an informed decision, you have to accrue a significant amount of traffic. 
  • Usually, this means you’ll need to pay to promote your products, which can become expensive. 
  • Depending on your traffic levels, tests may take days or even weeks. Often, the tests are inconclusive. 
  • And finally, because you’re using live traffic for your test, you risk an adverse effect on sales or ranking should one of your test variations perform worse than your current listing.


This is not to say that traditional split tests don’t have their value. They do. However, traditional split tests are not the only means to garner valuable insights and improvements on your e-commerce listings. 


An alternative to traditional split tests

PickFu is instant polling software that enables you to quickly test elements of your listing with audiences that match your customer profile and target demographics. On PickFu, it takes less than a minute to ask a market research question, choose the kind of people who you’d like to respond to your question, and PickFu brings 50, 100, even 500 respondents to your poll. Think of it as an online focus group. 


Examples of PickFu split tests

Testing product names

When one e-seller was developing a gardening product for apartment dwellers, two product names were tested using PickFu: PatioHarvest and PorchFarmer. When the results came in, the difference between the words patio and porch emerged as a major factor in why 76% of respondents preferred PatioHarvest. In addition, the word farmer didn’t resonate with the audience.

Optimizing product names this way helps you understand what shoppers will be more likely to buy. Polling also helps uncover problems you might not have realized were there, such as unintended word associations, difficulty in pronunciation, and unflattering brand perceptions. 


Testing product photos

One e-seller, Bumblebee Linens, asked 50 female respondents which picture of linen napkins they preferred. In less than 20 minutes, the results showed a new photo was preferred by a 3 to 1 margin over the old one. When the listing was updated with the winning photo, sales of the item improved 209%.  


Testing product descriptions Seller's Choice


Testing product descriptions

In a PickFu poll for a spherical ice tray, 50 Amazon Prime members were asked which product description they preferred. The two descriptions were similar in length and used a bulleted format to lay out the features. But one version used capital letters to introduce each bullet point, such as “PERFECT FOR THE WHISKEY DRINKER” and “PRESERVES THE FLAVOR OF YOUR DRINK.” Respondents found this description easier to skim and more attention-grabbing.


The winning description can now be seen on Amazon, where the product has earned the coveted “Amazon’s Choice” distinction. 



Screen shot from Amazon, Tovolo Sphere



You can’t simply list your product and expect sales to come in. By continually optimizing your listings, you’ll help improve your sales, rankings, and seller reputation.


Traditional split tests can be valuable but have limitations. These tests can be time-consuming, expensive, and too often are inconclusive. However, traditional split tests are valuable for subtle changes that humans aren’t good at discerning (for example, seeing whether changing the order of color, size, and fabric in your product title has an effect). 


For many tests, however, using PickFu is a fast and simple way to optimize product names, featured photos, and product descriptions.


This is a guest post by Kim Kohatsu, Director of Marketing at PickFu, an instant polling service that Amazon sellers use to optimize product listings. With PickFu, e-sellers can quickly test product photos, description copy, and UI layouts with shoppers who reflect their target demographic, including Amazon Prime members, those within a certain income bracket, or people based on traits such as homeownership, marital status, or exercise habits.

e-commerce tech stack ebook



Gain access to our exclusive list of top rated events for e-commerce sellers


Get E-Commerce News Right To Your FB Inbox!