High conversion rate improvement using split-testing is just as achievable by web sites with unique monthly page visitors of less than 4,000, as those sites with 100,000+ visitors – as this trial shows.
This conversion improvement trial was carried out on the web site of a small family-run hotel.
While the use of split testing to improve conversion rate is not new, it is often reported ( Note 1 & 2 ) as being only within the realm of medium to high traffic web sites.
This can create the perception that split testing is generally only viable for medium to high volume traffic sites that have 100’s of thousands of page visitors a week or even a month.
The hotel owner had done much to improve the site and increase traffic through pay per click (PPC) and search engine optimization.
She had this to say:
“We had worked hard to create a very attractive and informative site. Traffic did significantly increase but we were frustrated by the total lack of increased bookings, sales were flat.”
“If more people were viewing the site and everyone liked it why wasn’t that translating into more conversions? We were stumped.”
The web site enabled bookings to be made online for hotel accommodation and acted as a marketing platform for weddings and conferences for which reservations were taken offline.
Although traffic to the website had gone up by almost 97% in the past year, revenue had not increased by the same amount, PPC was not profitable and the conversion of site visitors to hotel guests had stalled at around 0.11%.
And if you can’t get enough free organic search engine traffic, paying for visitors by using pay per click is not necessarily cost-effective, even during a test, as this study shows.
The majority of traffic entered the site through the homepage and although this page was visually appealing it suffered from poorly focused content with unclear navigation and too many links that were a distraction for visitors.
Before making any modifications 35 web pages on the site were chosen for a traffic survey to determine visitor behavior. The survey confirmed concerns about the unfocused site navigation and identified four other key website pages apart from the homepage.
Elements on the homepage selected for testing were:
The original page comprised a large header graphic and a single-center panel. This was modified for the test pages to include a smaller header graphic with left and right side panels and the center panel color was lightened to make the text easier to read.
The header graphic consisted of a very large 650 x 405 pixel image of the hotel which was to be replaced in the test pages by two different versions 775 x 140 pixels.
Two new headlines were created for the test pages. One short headline with a sub-headline and the second a longer main headline with no sub-headline.
Main Body Content
Completely new content was written for the test pages more focused and sectionalized for the different interests of each target group of visitors. A single version of this content was applied to all the new test pages.
The original page navigation was a mixture of text links and graphic images, many of which were not clear to the visitor as even being navigation links.
The text links were not the conventionally accepted blue underlined but instead used the same color as text on the page and were also not underlined. Two text links were even located between the headline and sub-headline.
The image links were also unclear and obscured in the header graphic and a large distracting sitemap with too many unnecessary links was located at the foot of the page.
This was all replaced in the test pages by a 6 image button in the left-hand panel for visitor navigation and text links at the foot of the page for search engine navigation.
To some of the test, pages were added a number of 200 x 160-pixel images that related to the text content alongside and designed to show features of the hotel.
To some of the test pages, testimonials were added to the left and right panels, running the full length of the page.
A simple test was designed in which 5 different versions of the homepage were tested against the original unmodified homepage in a 6 way A/B/C/D/E/F split test. Equal amounts of traffic were then sent to each page in such a way that only page impressions for unique visitors were counted and returning visitors were presented with the same page they first viewed.
Visitors to the site when presented with test page version A (1) would see the original home page and the four original key pages if they used the navigation. Visitors presented with test page versions B(2) to F(6) would see a new homepage variant and new modified key pages if they clicked the on-page navigation.
The 6 way A/B/C/D/E/F split test lasted for 59 days during which time each page was viewed by an average of 1,258 unique visitors and no other modifications were made to the web site.
The results showed that page version 2(B) produced a conversion rate of 0.64% compared to the original homepage version 1(A) with a conversion rate of 0.08%.
Using the exact conversion rate figures for version 2 of 8/1258 and version 1 as 1/1255 gave an increased conversion rate for the homepage of ( 0.63593 – 0.07968 ) / 0.07968 = 698%.
The homepage conversion rate improvement was therefore 698%.
All new homepage versions produced a better conversion rate than the original homepage. Even the worst-performing new homepage had a conversion rate just under 0.4% compared to 0.08% for the original page. However, statistically, there was not a significant enough difference between the new versions to determine an outright winner. The test would need to be run for longer.
The best performing new homepage was 8 times better than the old page and even the worst-performing page was 5 times better.
The average number of online lodging bookings per day taken by the hotel reservation system during the 6 months prior to the conversion improvement work was compared to the average online bookings and value per day taken during the test period.
The results showed the site sales revenue was improved by 181%.
Before the split-test was run it was expected that either test page versions 5 or 6 that included on-page photos and testimonials to the left and right side of the page would be one of the best-converting pages.
In fact whilst the results showed that both of them performed equally well, neither performed as well as version 2 which had no on-page photos or testimonials. This was contrary to what was expected.
The study showed that the conversion rate of a homepage with a low traffic volume of less than 4,000 unique visitors per month could be improved by 698% coupled with an almost tripling of sales revenue.
This demonstrated that web sites with low volumes of traffic are just as capable of achieving high conversion rate improvements through testing as web sites with medium to high volumes of traffic – 100,000+ unique visitors per month.