Website Testing and Optimization (CRO) Guide

Digital marketing makes it possible to easily create tests to learn what works best in order to improve the results of your marketing programs. The nature of digital marketing means you can test almost anything about your campaign. Testing offers, key messages, ad copy, graphic design, headlines, and every other element is not only possible, but easier and much less expensive than with traditional media.

Just because you can test anything, however, does not mean you should, and what you test and how you test will determine how effective you are at improving your results. After all, no one has an unlimited testing budget, and even if you did, it’s important to test the variables that can have the largest impact first in order to use that information to get better results faster.

Here are the steps to creating an effective testing program.

Test Strategy

Before you begin creating your first test asset or installing testing software, you need to identify clear goals for your tests. Do you want to improve click-through rates, lead conversion rates, or increase actual sales revenue? Are you trying to learn which messaging is most effective or drive more leads? Once you identify your goals, you’ll be able to create a test plan that achieves your goals. As you do this, you should also do the math to see the impact that you might make with your test plan. In other words, how much more money would you make if you improve your conversion rate by 25%, and is this worth the cost of the test?

Test Ideas Hypothesis

When testing, you must start with a test hypothesis, which is a question you want to answer. If you can’t frame your test into a meaningful question, then you can’t really develop the test properly. The test hypothesis might include tests of the design, structure, copy, or offers on your website or the ads you are running. The goal of this exercise is to identify the most likely changes that are going to have the greatest impact so you can test them first.

What Not to Test

The first thing you need to understand when building a test plan is what not to test. Don’t test things that are unlikely to make a difference. Should your banner be red or blue? Should the submit button be round or rectangular? These changes are unlikely to produce significant results, so test them after you test the biggest, most important questions you have.

Another place to avoid testing is an area we call “best practices.” If other people have already tested something and published data that tells you it works, then use their experience and don’t make these things the first thing you test. An example of this would be landing page design. A best-practice landing page should have a good offer with a strong call to action, the form above the fold at the top of the page, and a limited amount of copy.

Don’t start by testing a bad offer, with no call to action, a form buried deep down the page and thousands of words of copy against the good page. You’ll just be wasting your time.

By leveraging experts in design and direct marketing you can save yourself tens of thousands of dollars and many hours, and arrive at the answer much faster. So do your homework first and leverage all of the expert advice and best practice experience that you can.

After you’ve done all of this, you’ll still find that there are many things you’ll want to test.

Everyone Gets an Opinion—Then Test

A good place to start building a test plan is the disagreements you have with your team on what will make a strong campaign. One person will often feel strongly that one topic or offer is best. Another person might strongly believe that one media type is best. Still another person will feel that certain creative designs will perform best.

Many times these arguments can be diffused through testing. People often believe strongly that their ideas are best because they’ve had experience doing something that’s worked or not worked, even though they have not really tested it scientifically. These ideas are often good, and it’s fine to build them into your test plan.

Good Things to Test

In my experience, the best things to test are the things that are likely to make the most difference the fastest. In most programs, this includes the offer itself, the advertising media, the targeting variables, and the creative.

The Offer

The offer is almost always the most important element of the campaign. What are you asking the user to do? What is their incentive to do it? If you are running a B2B campaign and offering a white paper in exchange for the user’s contact information, then it’s critical to offer content that is valuable, timely, and important. An offer can be early-stage content such as a white paper, mid-stage content such as a case study or demo, or a late-stage offer such as a sales meeting.

The Media

The media you choose to use for your campaign is one of the most important decisions you will make. If you choose to advertise on billboards when newspaper ads, tradeshow ads, or Internet banner ads would have been more effective, you can waste a lot of money quickly.

The problem with media decisions is that they are often relatively expensive and any one item can take a large portion of your budget, so unless you have a very large budget, you can only choose a few items to test. It can also take a lot of effort to evaluate and then manage each media purchase, and then collect and analyze the data.

When choosing media, it’s important to evaluate each option on similar criteria, and then make decisions based on your ultimate goal. This will make it easier to set up the tracking mechanisms you need to evaluate the results once the campaign is over.

If your goal is to drive qualified leads for your sales team, then create a simple table to evaluate all of your media choices. The table will include the name, cost, number of impressions, clicks or contacts expected, leads expected, and cost per lead expected. By collecting the data across your media options based on your goals, you’ll be able to rank order the use of media based on which will produce the most leads or which will produce the lowest cost per lead.

Paid vs. Earned Media

Today, paid media are only a small portion of your media options. If you are active with your website or social media, you’ll also need to manage your investments in time and energy in non-paid media such as social networking sites, your own website, micro sites, partner sites, and press activity. These activities may take very little money, but they all require time, and it’s important to evaluate these investments in the same way.

If one of your marketing people is spending eight hours per week on social media, then allocate the time they are spending to each of the social media activities to make sure you understand the cost per click or cost per lead from these activities, just as you would from other paid media placements.

Once you have a good list of media options, it’s important to make small bets with your limited marketing dollars on new media until you have a solid understanding of the cost per lead you can expect from each type of media.

I’ve seen marketing managers get very excited about a new type of Internet marketing program and allocate 75% of their budget to the media without any experience on how well it would perform. Making small bets is critical to testing. If you need to commit large dollars to one media just to buy in, it’s very difficult to test it against other choices efficiently.

Creative Testing

There are a limitless number of creative options you can test in any given campaign. The concept, headline, graphics, photography, copy, and layout can all be tested.

When testing creative designs online, you can A/B test or multivariate test. A/B testing simply means that you position two fully designed creative options against each other. The critical part of this testing is that the creatives need to be seen by different parts of the same population at the same time in order for the test to be meaningful.

If you send the “A” email on Tuesday morning at 9 a.m. and the “B” email test on Saturday night at 10 p.m., you won’t be sure if it was the time of week or the creative that determined the success of the design. If you show one website ad on the home page and another on an interior product page, the results won’t be meaningful. If you run an ad on one keyword and the test on a different keyword, again, you won’t get meaningful results.

You also need to make sure your test results are statistically significant. To do this, you’ll need to do a little math to make sure you showed both ads enough times and that the results were different enough to tell you that one ad was the winner and the other was the loser.

Building the Plan

Once you have these elements in place, you are ready to develop the plan. The plan should start with a hypothesis that should give the rationale for the test, the test budget, how you plan to run the test, and the outcome you expect. In the hypothesis, I like to add a statement explaining the business impact so that you can start to make decisions about which tests you should run first, second, and so on.

Some tests are very expensive, and can be conducted quickly, and others need more time. For example, creating two competing TV ads might be a very expensive test, but might result in a strong ROI. Other tests can be run on less expensive media to learn something that can be used later on more expensive media.

In other words, you might run a test on your website to see which type of offer, an eBook or a white paper with the same title, is more effective. Based on this test, you can use the outcome to position the winner in an email campaign, a banner ad campaign, or a direct mail campaign, which is more expensive to run.

Record the Test Results

After you run a test, make sure that you document what you learn. A year or two in the future, a new team may be considering more options for testing and you want to make sure that they don’t spend money to repeat a test you’ve already completed. If you are running tests on a regular basis for every campaign, you’ll soon amass a steady list of results that will help you get better and better at producing consistent results.

Email Testing

Email is an important part of almost every online marketing program. Because of spam filters and because of how easy it is to opt out of email campaigns, it’s important to make every email valuable and professional. You don’t want to antagonize your users with miscellaneous testing emails that cause them to opt-out.

A key part of every email creative is the subject line. This is the headline for the email campaign that will either get the reader’s attention or cause them to delete the email before they open it. Because of this, you should test the subject line first. The best way to measure the subject line is to evaluate the open rate. Even though it’s not completely accurate, it’s a good gauge for you to see if the subject line is doing its job.

Another important part of email testing is to evaluate the click-through rate. If they user opens the email, the click-through rate will tell you if the email itself is doing its job to get the user to take an action.

It’s important to test both the open rate and the click-through rate to help you see if the email is effective.

Website Testing

Many organizations don’t think about testing their own website, although this can be the most productive and lowest cost marketing vehicle to test. You should be tracking marketing results on your website with the same metrics you are using to evaluate other media. Impressions, clicks, and conversions can all be easily tracked.

Your website gives you the ability to rapidly create and deploy tests with no media costs. These tests can take the form of creative elements or ads on your homepage or other pages, and you can vary the design, offers, or messages on these ads, just as you do with other media.

Once you have your hypothesis and your ads developed, you can deploy them and get results on the click-through rate in order to see which message or creative is best.

The power of website ad testing is that you can learn quickly and then deploy these tested ads to other paid media.

Testing can drive a very high ROI for your marketing campaigns. If you use your imagination and then deploy the tests in a systematic and disciplined way, you can generate better results.