A/B Testing: Schedule a Campus Tour
We are always on the hunt to optimize our websites and making changes on a consistent basis. I like to call the process "Micro Redesigns (opens new window)" and I have been talking (opens new window) about it more (opens new window) and more this year (opens new window). A lot of the examples I use are from Wayne State. We are lucky enough to control the complete user experience from the bottom up so we have a lot of opportunity to play. And by play I mean optimize, optimize, optimize.
# What is a micro redesign?
It is taking small deliberate steps to reach a larger goal (which may be not 100 percent apparent at the time).
# No perfect website
No matter how long you work on a site and the number of people you test it on there is always room for improvement. For example, our homepage has felt the same for the last few years but has actually changed a lot. Instead of doing a sweeping redesign every two years we decided to focus on one piece of the site every two months and optimize it. As of right now, two years after launch, only 20 percent of the homepage is the same as the original look. We have increased the usability of every piece of the site without interrupting any stakeholders use of the page. It isn't just the homepage either, sub pages have been changing also. More about that in the next week or so.
# Schedule a tour example
This last month we set out to try and improve the number of conversions to our "Schedule a Tour" link on the wayne.edu (opens new window) homepage. We knew the link was getting traffic but we tested our homepage with prospective students asking them to start on the homepage and schedule a tour and they would completely overlook it. We decided to test making it more visual. Below is a comparison between the initial and the proposed visual treatment.
The original (left) design features text and a link to schedule a tour now. The proposed (right) design features a map image with a marker similar to Google Maps with a smaller amount of text.
We decided to put the two versions up against each other for a month to see which one performed the best. We had our idea of which one would perform the best, but we had to have the data to back it up.
# The results
Launched: Jun 22, 2011 | Completed: August 9, 2011
After the first week it was pretty clear which version was performing the best. But we made sure to run the test for at least a month before making a final decision. Just over a month, 1 million visits and 1,000 clicks later, the results were pretty conclusive. The newer, more visual schedule a tour button resulted in a 67 percent increase in the number of click throughs to the schedule a tour form.
# Completed a reservation
I try to follow my own advice as often as possible but sometimes for what ever reason it doesn't happen. This was one of those times. I can't stress enough to test your configuration to ensure you can follow a user through the complete funnel to determine exactly what changes you made resulted in the most overall impact.
# Analysis
One could easily say that giving anything more color or space to click would yield more clicks. But we think these results show something deeper than that. I would agree in some sense that more color and space do yield more clicks but for this case I think it was what we added. The initial design didn't connect the label with what the user was about to do. Adding a visual map and a marker that people were already familiar with (Google Maps), made a connection with an existing construct immediately.
If you look at the graph above you will see the goal to complete a visit is set up correctly and is recording but what isn't recording is what page, or version of the "Schedule a Tour" button they came from that resulted in the goal completion. Google is just supplying "(entrance)" as the referral page and this isn't helpful at all.
One note on the graph above, it is only the tours completed coming from our homepage, it is not a total of all the tours coming in from the entire website.
It's important to have that source page to determine that although more people clicked on the more graphical version of the button, if they didn't ultimately convert then it doesn't matter how many people click it. We need to find which version produced the best ROI overall, not just in the micro sense.
# Dream big, think small
In the end it comes down to the notion of improving the overall experience for your visitors. Don't lose sight that those numbers are not a mass of people coming in hoards, but individuals coming one by one with a goal in mind with no tolerance for getting the run around.