Issues In Analytics

Final day!

Alex Bennert (Beyond Link) is moderating this morning’s analytics session with speakers Eric Enge (Stone Temple Consulting), John Marshall (Market Motive), Avinash Kaushik (Occam’s Razor Blog) and Jonah Stein (Alchemist Media).

Alex reveals she has changed the name of the panel. It’s no longer Analyzing the Analytics Partners. Now it is Issues in Analytics. Sweet, that’s way easier to spell!

First up is John Marshall. He’s going to walk us through the anatomy of a click. He says he’s going to give us enough technical information to show us why the data we get from our tools never matches the real number of clicks. He’s also probably going to give us enough technical information to completely confuse me, but that’s okay. Whose idea is it to always put the technical session on the last day? Someone needs to rethink this.

Okay, first a few assumptions John is making about the people in this room: He assumes everyone is using PPC ads with tracking parameters, that they have a functional Web site, that they’re using modern Web analytics tools, and that at some point they’ll need to understand why the numbers don’t match.

John is not going to cover breakage in ROI, cookies and cookie deletion, or anything behind the single, solitary, lone first click – the click that gets users into the site.

Walking through the single click: You click on an ad. That click is then sent to Google so that they can record it; it’s not immediately sent to the destination URL. The server then responds to the browser and redirects to the destination URL of the site. This gives you a 1 percent error right there because the browsers would drop the redirect.

Next, the browser executes the redirect. There’s a DNS lookup because you probably haven’t been to the site before.

Some sites redirect to hide tracking URLs in order to make it look pretty to users. When you do that the browsers responds by giving users just the home page. This may work if you’re using Apache, but he doesn’t recommend it. If you’re using IE6 or JavaScript you just broke your ability to track because the URL that appears in the address bar is the thing they’re tracking against. If you strip off the query string, you get nothing. Don’t do that.

Browser request landing page. At this point the log file based solution probably has a 5 percent error because this act will sometimes be cached.

Now the browser renders the landing page. It grabs all the elements, including the JavaScript. You’ll probably fail on JavaScript about 10 percent of the time because the JavaScript has so many roundtrips it has to do across the Internet that, if any one of those fails, you’ll lose the data.

John says the reason the tracking never matches is complex. Basically the more moving parts you have, the higher the number of errors you’ll get. Don’t worry about your numbers not matching up. Logs and JavaScript are equally inaccurate. Logs are better for catching click fraud. Largest contribute to error is implementation of JavaScript. Watch for redirection. It’s not always a good thing.

Eric Enge is up next.

Eric starts by posing a hypothetical:

Company A gets $1,000/day in PPC revenue. They want to use analytics to find the best converting keywords and prune poor converting words. The analytics package reports $800 in PPC revenue. Can they trust the data?

Company A is buying Company B, who says they have 50k uniques a day. Company A earns $.25 uniques and projects they’ll earn 12.5K a day from the acquisition. The acquisition closes and A puts their analytics on B sites. A’s packages shows only 40K a day.

What do you do about this? You want to cross check and calibrate. You don’t want to use your analytics package to tell you what your total revenue is. Put a parameter on the URL for PPC visitors. And for the situation above, you want to put your analytics on their site so you know what you’re getting.

Eric walks the audience through a recent analytics report he compiled with the help of SEOmoz. Those Mozzers are everywhere.

Sources of data variance:

• Bad or ambiguous data.
• Some data is thrown out.
• Packages make judgment calls
• Session tracking timeout.
• Industry standard is 30 minutes/ some use 15 minutes
• New search engine visit starts a new session?
• Too many judgment calls.

Sources of error include implementation problems, misunderstanding the terminology (visit URL vs. Entry Page URL), and JavaScript placement. The issue there is the delay before execution – users may move on before the JavaScript has time to execute. 1.4 second delay equals 2-4%. Best practice is still to put your JavaScript at the bottom of page.

Recipe for Success:

1. Accept that analytics comes with errors. The Web is too complex.
2. Know that trend analysis works
3. Eliminate the implementation error
4. Learn the terminology of your vendor
5. Focus on the strengths
6. Pick actionable KPIs
7. Measure errors
8. Cross check and calibrate
9. Use judgment

Jonah Stein is next.

The objective of analytics is to provide a rational basis for decision making so you can maximize the ability to obtain campaign objectives.

The prices range from “zero” to “a lot” and integration, testing and deployment ranges from “hours” to “lots of hours”. Analytics packages contain assumptions that affect results. You have to find the right one. You could set up all the packages on your site and compare the results. Of course, this is time consuming and it requires a lot of implementation.

Just knowing what you’re going to compare doesn’t solve the problem. How are you going to compare them? What is the baseline you are comparing against? Visits? Unique Visitors? Page Views? For most people, the only thing they want to compare is conversions.

If you’re going to compare tools make sure they are starting at the same time, otherwise you’ll be looking at different samples. IndexTools and ClickTracks still show sales with no revenue. Google has stopped doing this. ClickTracks only captures revenue for the first transaction in a session.

Auditing Conversions: Make sure you have a unique identifier for each order. Create two tables, one for each tool. Join all the invoices that match. Add all invoices unique to table one. Add all invoices with number two. Voila, you have your number.

PPC conversion auditing. Join tables at the keyword level. Invoice the invoices at the keyword level. (Note: Google analytics will not give you keyword level invoice IDs.)

• Overall results are fairly close.
• Analytics systems need to be tuned and refined to caprture ROI.
• Follow the warning level.
• Analytics vendors do too much.
• Analytics should not be relied on for ROI calculation.

Best way to measure ROI

• AdWords Conversion Track captures the most conversions with a 30 day cookie.
• Capture campaign & Keyword to your own cookie.
• Bring the data into your CRM database.
• Save marketing data at the earliest touch.
• Incentives to determine source at every touch.

All of these tools are excellent ways to bring data into your customer management system.

Avinash Kaushik is up next.

The days of "logging". The reason people moved away from Web logs was because of the IT guy. Marketers didn’t want to look at stats so we moved to a tag world.

There are other ways to collect data – packet sniffers, for example, are a decent way to collect data but they haven’t caught on because they require a lot of IT involvement.

A lot of people like hybrid forms of collecting data. This is really hard to pull off. It’s hard to put everything together and try to get something that’s readable. On paper, hybrids sound like a good idea but they’re hard to implement.

Current players in Web analytics – Google, WebTrends, Omniture, Coremetrics, Indextools, Webside story, Unica, Microsoft, etc.

Even though we have lots of Web analytics vendors, it’s getting harder to monetize. It’s hard to make money as an analytics provider because of silos. It’s not integrated the way it should be. Most companies approach Web analytics like it is God gift and they don’t have to do anything with the data. They don’t know how to analyze the information they’re getting. It’s causing a challenge in the field.

We’re at the very early stages of Web analytics. The evolution is not yet complete. It’s like a little baby.

Avinash lists the best parts about the many Web analytics programs out there:

• Omniture: One of the big companies in the space. They’re moving beyond clickstream analysis. Automated action "taking".
• WebTrends: Marketing optimization. Give us your search spend.
• WebSideStory/Visual Sciences: HBX plugin: Custom Excel reports. Visual Science is God’s gift to analysis.
• CoreMetrics: Got retail? Lifetime Individual Visitor Experience.
• IndexTools: Custom reporting, anything by anything. Enterprise for the cost of little.
• ClickTracks: Ease of use. Unleashing the power of segmentation.
• Microsoft: Visual, Free. Demographic Segmentation.
• Google Analytics: Data democracy. Best of breed search analytics.

The Web Analytics Association is now presenting 26 new Web analytics definitions.

Lisa Barone is a writer, content marketer & VP of strategy at Overit Media. She's also a very active Twitterer, much to the dismay of the rest of the world.

See Lisa's author page for links to connect on social media.