Multivariate Testing and Conversion Tweaking

Back from lunch. That was one yummy chocolate chip cookie! This time we’re at the Multivariate Testing and Conversion Tweaking session with panelists Tom Leung, Glenn Alsup, Philippe Lang and Rand Fishkin. Moderating will be Gillian Muessig, aka Rand’s mommy.

Gillian isn’t wasting any time. She introduces Glenn. Hi Glenn.

Glenn talks about qualitative vs. quantitative data

Qualitative researchers: Reject the idea that social sciences can be studied like natural or physical sciences. They feel that human behavior is always bound to the context in which it occurs. And it’s usually personal and subjective.

Quantitative researches: Argue that both the natural and social sciences are testable and confirmable theories. They also do lots of other stuff I didn’t have time to get down. Sorry. There was a lot of text on that one slide.

Glenn says qualitative is Gods Fuel:

They look at:

Goals
Overview
Description
Scenarios

And get:

Finding
Usability Issues
Effects on Goals
Lessons Learned

It’s effective to look at this as a sales funnel. Identify all your drivers – Offline, Online, Explicit and Extract. Then take the data and group attributes based on filters.

Look at the stages in the funnel. Weight them by the value of that event. Downloading a white paper is more valuable than looking at a banner. Identify when the action took place. When you convert, those drivers are calculated in different ways. You want to increase conversions by identifying the right mix of marketing drivers.

Tom from Google is next to talk about smart testing. I have a feeling he’s going to talk about Google’s Web site optimizer.

Ooo, I’m right! Tom asks: What do you do when the visitor lands on your site and what can you do to get them to convert?

Driving traffic is just the beginning. You invest in search engine optimization and SEM resources for 100 percent of visitors. Your pages may lose more than half their visitors in seconds. Most that do stay choose not convert. (Industry average for converting customers is 2-3 percent.) Why bother bringing more visitors to a site that convert poorly?

Evolve with continuous improvement. Drive the right traffic to your site. Measure & analyze site activity. Test changes and implement winners. Repeat steps 1-3 until conversation rate is 100 percent.

How Testing Works

Visitors arrive on your site. They’re shown a random version of your site and the testing tool will tell you what percentage of users converted based on what version of your page.

He shows an example of a page Google uses their Web site optimizer on. It’s the home page for Picasa. In the first version of the page, it uses "free" a lot and is very picture and action oriented. They found that the second page, on that is cleaner, includes a Try button and states a clear value proposition converted 30 percent more.

How do you set up a test?

Most of the testing tools involve copy and pasting a piece of JavaScript. The control on top tells you that someone is trying to load the page. The tracking script on the bottom tells you that they saw the page, and then you have another code on the conversion page which tells you they converted and what version of the page they were looking at. If you do a test, each version of the page has a unique sticker for you to identify it by. After the test runs for awhile, Google will populate reports for you.

Best Practices in testing

• Test a small number of variations: Rule of thumb is less than 100 conversions per combination.
• Test big changes: If you can’t see difference between two combos in 8 seconds, visitors probably won’t either.
• Consider early indicators if you don’t have enough conversions: If you’re selling a $100k software package or a small business with modest volume, optimize for conversion indicators such as request info, view product details, etc.
• Don’t jump to conclusions: Less than 2 weeks is no good, focus on absolute conversion difference, don’t get too excited by sliver of green.

More Testing Ideas:

Conversion Cocktail: Headline/Image/Call to action. Those are the best three sections to test.
Trust seals?
Which testimonials
Inspirational or fact-based pitch
YouTube video
Navigation bar

Philippe is up next.

Analytics provides you a ton of data about visitor behavior on your site. This information is very valuable but it can be hard to interpret. Your customers also provide you valuable information when you talk to them.

What if you could see in real time what your visitors are doing on your Web site? Start a dialogue with those visitors. Listen to them. Understand what they are doing and why. It’s like a usability test in the real world.

Things to look for when you monitor your Web site: Visitor referrer information like search engine used, marketing campaign, geolocation, and navigation behavior like page views time on page, shopping cart content, and shopping cart abandonment.

Merge the two sets of data (real time monitoring and feedback) by engaging specific visitors into a chat or leverage the customer support chat transcripts (what if you don’t offer a customer chat feature?). Try to see the patterns or trends that are coming out.

Ultimate Feedback Look

• Start by monitoring visitor on your site
• Identify visitors with unexpected behavior
• Engage visitors by listening to them
• Convert customer feedback into action
• Refine and optimize your Web site

Phillipe presents two case studies. The first is about a university offering an online degree.

Problem: High abandonment rate in the online inquiry form
Cause: Customers annoyed by specific question about the age of applicant. Customers abandoned the form instead of answering the question.
Solution: Removed the question
Result: 20 percent increase in form completion.

Online retailed focused on outdoor gear

Problem: Low conversion rate on sunglasses
Cause: Customers were confused by the sizing chart for the sunglass
Solution: Redesigned the sizing section
Result: 25 percent increase in conversion rate and 20 percent decrease in product returns.

The morale of the story: If you know what’s wrong, you can fix it!

Testing helps companies increase conversion rates, improve the sales process, increase average order value and builds relationships with customers.

Last but not least is Rand Fishkin.

Rand opens by talking about the landing page contest SEOmoz ran a few months back. They let blog readers submit landing pages to them. They got lots of submissions and ended up testing 10 of the 40 entries they received. Rand shares the stats for each page. It’s exciting. Ultimately, the page where you had to scroll forever to get to the end of the page converted the best. That’s why you see so many of those pages.

Once the contest was over, the SEOmoz crew tweaked the winning page to make it convert even better.

Takeaways

• If you think it will work, it probably won’t
• Landing page design is not universal
• Testing is the only way to get better
• Doubling your conversion rate is far easier than doubling your traffic.

Question & Answer

How much did you pay for Offermatica?

He thinks its $10K a month and up.

Rand, it looks like you tested a handful of completely different pages. Can you tell us how Offermatica fit into that? What page elements did each have that helped it convert better?

Rand doesn’t actually answer the question and just says that they were doing A/B testing (testing two different pages), not multivariate testing where you test different images or different headers or different layouts.
I always thought that I should change pages incrementally so that Google wouldn’t think it was a different site. Is that right or am I paranoid?

Rand: Basically conversion testing is a legal form of cloaking. What you’re really doing is showing a bunch of different versions to people to find the best one, but you’re only showing one version to the search engines.

Tom: With Google, the spiders don’t look at the JavaScript content so the source code will look consistent. And even if the layout is different, the spirit of the content is similar.

Glenn: Typically you see smaller changes.

Multivariate testing about pleasing more of the people more of the time. When visitors come back to the page, is there any way to ensure that they see a different version of the page?

Tom: When the experiment is running, users will continue to see the same version of the page they saw the first time to protect the integrity of the experiment.

Lisa Barone is a writer, content marketer & VP of strategy at Overit Media. She's also a very active Twitterer, much to the dismay of the rest of the world.

See Lisa's author page for links to connect on social media.