How to split-test a website with PHP
by Chris Woodford. Last updated: September 19, 2022.
You've shelled out for the best web design company you could afford. You've paid big bucks to the SEO company to shoot you up the rankings. But if your site's designed to make you money—through product sales or advertisements—there's one more thing you must do: optimize it with split-testing. Most people set their sites up in an arbitrary way on the basis of what looks nice or "seems to work." But as web pros know only too well, even apparently insignificant changes to your site's structure, layout, colors, or type can bring big payoffs in click-throughs and orders. Split-testing (sometimes called A/B testing) is a simple way of optimizing your website by comparing the results you get with two different versions of the same page. Let's take a closer look!
Photo: Split-testing web pages is the best way to determine which of various different site layouts gives you the results you want—in sales or advertising revenue. All you need to master is a tiny bit of PHP, like this.
Sponsored links
Why simple testing doesn't always work
The simplest way to test the effectiveness of a web page is to alter your current design, try the new design, and compare the results. Sounds simple—and it is. The only trouble is, it's too simple. If you run design A for a whole week and then run design B for the whole of the next week and compare the results, you won't necessarily establish which design is the best. If you're tuned in to how your web traffic fluctuates from day to day and week to week, you'll know only too well that many things can change from one week to the next. You might hit a school holiday, for example, or a run of really good weather that sees large numbers of people quitting their desks for the beach. Similarly, if you're trying to compare advertising performance, you probably know that many factors can change week by week, day by day, and even hour by hour. There may be more people advertising in the second week than the first, producing a bigger advertising inventory, higher paying ads, and better results—irrespective of which design is running on your site. Someone might flag your site with a social bookmark or mention you on a forum, sending you a sudden, massive batch of very dubious quality traffic that dramatically reduces your site's performance.
All told, running one design for a day, week, or month before alternating with another design is going to give only a crude idea of what works best. You could keep switching back and forth between different designs and use some form of graphical or statistical analysis package to compare your results each time you switch but, the trouble is, it's going to cost you money to do this! One of your designs is almost certainly a bit better than the other. If you change your original layout for the worse, every day you run a less-effective, alternate design is a day that dips into your dollars. That's why you need split-testing.
What is split testing?
Split testing dates back to early "key-code" tests of newspaper advertising devised by Claude Hopkins about 100 years ago. He printed slightly different versions of an advertisement (perhaps one with one headline, one with another), including a coupon with a return address that featured a different mailing box number. By comparing the number of people who replied to reach coupon, he could reliably measure the effectiveness of each ad variant. Testing multiple variants over days or weeks, he would eventually arrive at an optimized ad. Today's split testing of online ads is just an updated, hi-tech version of the Hopkins method.
The method is simple. You create two different versions of a web page and arrange for them to be shown randomly, alternately, to your visitors. You track the performance of each page separately and compare your results from each page after a decent number of impressions (a few thousand is usually enough but a big difference in performance will show up after only a few hundred).
You can do split testing either with javascript (so the two variants you're comparing are included in one HTML page and the user's browser creates a single page on the fly) or with PHP (where one page or the other is built on-the-fly by your web server). Google's Adsense blog has a really simple example of how to do A/B javascript testing—in this case, it's being used to optimize the colors of advertisements. I much prefer using PHP. Even if you have no experience of using it, it's simpler to understand and completely transparent to search engines, which "see" either one complete variant of your page or the other (not some complex, glued together hybrid as with javascript). Using PHP also means you don't have to tinker with any of the advertising code (which is typically prohibited) and you can test much bigger differences between pages. For that reason, I won't discuss javascript testing any further here.
How to do split testing with PHP
PHP, if you don't already know, is a superb program that runs at the web server end of things, building web pages on the fly before they're sent out to a browser. It's immensely powerful and you can spend ages learning how to do all kinds of neat things with it. But, even if you have no experience of using PHP, you can learn just enough to run simple web page tests in about five minutes. Here's how!
1. Make sure PHP is running on your server and enabled with .htaccess
If you're using free, low-budget, or Windows web hosting, you probably don't have PHP. Check with your hosting company first. Once you've confirmed this, ensure PHP is working for your site and that files with the suffix HTM (or HTML) are correctly processed by the PHP processor. For Apache servers, you will probably need to include something like the following line in your .htaccess file:
AddType application/x-httpd-php .htm .html
Exactly what you need in .htaccess will vary according to how your server is set up. On another server I run, I have to use a slightly different variant and explicitly say that I'm using a certain version of PHP:
AddType application/x-httpd-php5 .html
AddHandler application/x-httpd-php5 .html
2. Save your original web page (or site)
You'll be comparing a new version of your web page against your original and possibly modifying the original quite a bit, so do the sensible thing and make a backup of the original to start with.
3. Create a new version of your web page
Now alter your page to create the new, "improved" version you want to test. Maybe you want to test a different ad layout, change the colors, use a different style sheet, rewrite some wording, place a big "Place your order now" button at the very top, or whatever. You can compare versions that are as different as you like, but ideally you want to change only one thing at a time so you can be sure which change made the difference. So maybe change the colors first and test those. Then, when you've found the best color scheme, change the layout. Then change the wording. Then add your "order now button." And so on. Make each change and evaluate it separately before monkeying with anything else. If you change everything all at once, you won't know where you are: you won't know what improved or impaired the response to your page.
4. Create two small pages based on the differences
Chances are the two versions of your web page will differ only slightly. Suppose you want to compare how many click-throughs you get with black text on a white background versus using white text on a black background. You can do that by using two slightly different style sheets so, in this case, the two versions of your web page will differ only in that they need to load different stylesheet files. Similarly if you're assessing the effect of adding a "Place your order now" button, one version of your page will have an extra line or two linking to the image and order page that the other version doesn't have. So now create two small HTML pages including only the code that is different. For example, the first file might be like this. Call it file1.html:
<P>
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
</P>
and the second file (including the order now link) might be like this. Call it file2.html:
<P>
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
</P>
<P><A HREF="ordertest.html">Order now!!</A></P>
5. Add PHP code to your original web page
Now you need to edit your original web page and remove the section you've identified that's going to alternate between the code in file1.html and the code in file2.html. Cut that section out entirely and replace it with this little bit of PHP code:
<?php
srand((double)microtime()*1000000);
if (rand(1,2)==1)
{
include("file1.html");
}
else
{
include("file2.html");
};
?>
See how simple that is? It will randomly include the bit of HTML from either file1.html or file2.html in your web page each time the web server is requested to send the page out to a browser. The browser will receive a page including either the file1 or file2 code, but (unlike in a javascript test) there will be no clue that the page has been built with PHP on the fly.
6. Set up a way to count the results
In this simple example, you could monitor effectiveness simply by seeing how many orders you get through the page ordertest.html compared to your usual ordering page. You could see how many times each page is loaded and how many orders you get as a result. If you want to compare the performance of two different advertisements, you can put one lump of advertising code in file1, another lump in file 2, and set up different channels/counters to monitor the CTR from each.
7. Monitor the results
Now all you have to do is wait for the results to come in. There are still some things to be wary of. The first is that if there is no real difference between the two pages you're testing, any counters or measurements you set up will likely flip back and forth showing first one page in the lead and then the second page, with the position constantly alternating. If that happens, it usually means the pages are just as good as each other, even if one page ultimately gets slightly better results than the other. If one page is constantly ahead of the other page throughout your test, it's obviously a better page. But do be sure to allow enough page impressions to give a reliable result (certainly a few hundred and maybe a few thousand, if you have time). Don't be tempted to draw conclusions from only a few impressions. If you want to be really certain (and scientific), run a chi-square test and check for statistical significance.
8. Don't stop there!
Once you've established one change that works, go back and incorporate it into your entire site and monitor the results. Then work on identifying another change and see if you can improve things a bit more. Keep doing this and, over time, you should systematically and significantly improve your site's performance. Every so often, run a test where you compare your current version of the site with an earlier incarnation to reassure yourself that the cumulative changes you've made are all working to your advantage. That's just a little double-check for your own peace of mind!
