If you routinely send email or bulk mail to a large number of people, you should really consider using Randomized Testing in your next campaign. While it may sound like something better left to marketing "experts," and large corporations, there are some very important advantages to this style of testing that can have real benefits for you, personally.


What is Randomized Testing?
Randomized testing is a method of data mining and analysis that allows you to predict the outcome of a campaign before you send it. The concept is simple: using a large list of message recipients, you choose a subset completely randomly, and send the message just to those people. Based on their response, you can take a good guess at how the rest of the people will respond.

Why does it have to be Random?
If you pick and choose data from your list in an orderly fashion, you are very likely to choose groups that are non-representative of your list as a whole, statistically. A perfect example would be to use the first 250 people in your list as a test group. Here are some potential problems:

• The oldest items in your list are most likely to contain expired data, such as bad addresses.


• Clients in a certain area of your list were probably all gathered in a similar way, such as personal correspondence, forms filled out at a trade show, or responses to a mailer during a particular season. Because of this, they can skew in many different ways, by race, gender, and socioeconomic status.


• Names in a list region have likely been contacted a similar amount of times in the past. Older and newer list members may have different preconceptions about your company.

Why should YOU randomize?
The simplest reason is that you simply may not have the tools or expertise to handle multivariate analysis of data. And, moreover, you may not have any data to analyze. If all you are collecting from your clients are their name, address, and email, you really know almost nothing about them as individuals.

Just as importantly, random testing can keep you from making costly mistakes in front of a large audience. A smaller, more representative audience will help you weed out problems early on, and without huge advertising expenditures.


What are some strategies?
Randomized testing can give you a good idea of how almost any new method can work in general. And, even better, it can help you decide on a method to follow, if you have several choices to choose from. Here are a few examples to inspire you:

• Headline Testing - On your next mailer, test out 3 or 4 different headlines to see which was most successful, based on the number of opens you monitor.


• Offer Testing - Thinking about offering a free service, coupon, discount, or rebate, but you don't know how it will affect business? Try all of your ideas on a small scale and see what happens


• Earthshaking ideas - You have the idea that is going to save your company, but your boss thinks it's too risky! Randomized testing can show him if you're right, and it won't ruin you if you're wrong.

How can you get started?
If you can open your database in Excel, you have everything you need to get started. The simplest way is to add a new column to the right side of your data called "Test Group." In that column, use the following equation to generate a random value:

=ROUND(RAND()*SOMENUMBER,0)

To find the value of SOMENUMBER, divide the total number of names in your list by the size of the test you'd like to send. Then, round down to the nearest integer. So, if you wanted to send out 250 emails out of a 10,000 email list, 10,000/250=40, so SOMENUMBER = 40.

From there, copy and paste the equation into all rows. This can be done by selecting all of the rows at once before pasting, saving you hours of hitting ctrl-V. After that is done, select ALL values in that column, copy, and use Paste Special. Change the setting to "Paste Values." Voila, you have a randomized list. Simply sort by "Test Group" and you'll have many sublists of approximately the right size. If the list is a little big or a little small, that is to be expected, based on probability.




For a fun read regarding the value of randomized testing, please check out Super Crunchers by Ian Ayres.