Decrypting Quality Scores

Time for some pay per click quality score action with Andrew Goodman (Page Zero Media) and Jon Kelly (Sure Hits). Chris Sherman from Search Engine Land is acting as moderator and Mary Berk (Microsoft), Nick Fox (Google), and David Pann (Yahoo) will be helping out with the Q&A. Now that the roll call’s done, let’s do it!

Up first is Andrew Goodman. Neil Patel gets everyone clapping for Andrew. Aw, way to go, Neil.
Before we get into details, Andrew says to think like a search products manager on the side of "the user" trying to determine what a high quality ad is. How would you design it?

Low quality is:

• Bad experience on site: Sites and pages that frustrate or mislead users.
• "Misdirection or poor relevance" – Advertising on irrelevant keywords, or keywords that are only pretending to be relevant>

See how simple it is? Nearly everything they’re doing can be subsumed under those two categories. The old formula that only used CTR to rank ads wasn’t sophisticated enough to achieve these objectives.

Three generations of Paid Search Ad Ranking

1. GoTo.com: pure bid for placement
2. AdWords: Max bid X CTR
3. Adwords 2.5: Quality Based Bidding.

How it Works: There are two quality scores. One affects your minimum bid and the other affects rank. There are also two distinct types of quality.

• A formula composed of signals of relevance and quality related to your keywords.
• A sometimes hypercritical assessment of your Web site and landing pages looking for "bad guys".

So is arbitrage the issue? Not per se, but a lot of what Google is doing had "stop the bad guys" motivations. The main indicators of angry users are not that hard to figure out. Arbitrage does fit into that category.

Introducing the AdWords Sandbox?

How do you get out of the new holding pen? The dig out process:

1. Media company wants to buy traffic – launching new verticals.
2. Past high quality not helping – many Quality Scores starting low – can’t figure out why
3. Also, show ad activation on timely news items
4. Multiple theories as to why initial low Quality Score.
5. Frustrated: New ad experiments
6. Frustrated: Tips from Google Rep
7. Social elements of exchange with rep. Get a promise of the review or vague comments

Once your ad is removed from the sandbox you have to ask yourself which happened:

• Did we build up appropriate signals and improve the account through testing?
• Did we exhibit frustration at the right , in the right way, to a Google rep, and get them to do a manual fix?

It’s Algorithmic, not Human

In what proportions? Case by case. He comments that Nick Fox can provide high quality overview of some of the data Google collects. There’s no denying that adjustments are made manually from time to time

Account-Wide Effects
When you look at the letter of the law, realize that if you establish a strong account history you’ll see a halo effect. You can throw in a bunch of crazy keywords and you’ll still be okay. As time passes, CTR is still your anchor.
Consider how much initial quality score can tell you about how Google views you. Formula for high initial quality score:

Anatomy of a high QS kickoff

• Narrow keywords
• Hand-built ads
• Tailored landing pages that work
• Geo-specific helps
• Segmenting and micro-targeting

Jon Kelly, aka one of my new favorite people, is up next.

Jon calls ad quality essential for the engines, which makes managing it key to your PPC campaign. If you’ve had puzzling results, you are not alone.

Quality Score is the big hammer. It affects both ad rank and minimum bid. He’s going to focus on ad rank. If you think about metrics in terms of who they affect and who they’re important too, what’s ultimately important is the CPM. That’s the ultimate metric. CPC is in many ways a compromise. It offers accountability, but it doesn’t have the same value as CPM.

Quality Score is a way to bring the value of the clickthrough rate back through the engines. The formula looks like this:

CTR & Bid Price x 1000 = eCPM for the search engine

As an example:

• $10 Bid
• 5% CTR (primary measure of Ad Quality)
• 50 Cents / Impression
• $500 eCPM for the engine

When you look at that model, the CTR is every bit as important as the CPM is for the engines. The search engine’s goal is to sell the space for as much as possible. They want to move the highest eCPM ads up top. Landing page quality maximizes long-term eCPM, that’s why the search engines are interested in it. They don’t just care about the first click; they care about every time someone clicks on an ad.

Key Quality Score Factors

1. Relative CTR by position
2. Account History
3. Landing Pages (Min. Bid)

You can’t hurry Google’s love. Your goal is to beat expectations. Higher on the list just means more data, faster.

For every given pair of keywords and for every keyword on every engine, there’s an expectation of how many times that ad will get clicked on. You have to beat the expectation. That’s how Quality Score works. When you look at your clickthrough rates it has to be on the assumption that everything else is equal.

Your key metric: Value per impression
Clickthrough rate (x) Conversion Rate = Total outcomes/impression

Your two key metrics are often at odds. Qualifying words raise Conversion but may lower CTR. Enticing words raise CTR but may lower conversion.

He uses the word "cheap" as an example. It can drive high clickthroughs, but will it drive conversions? The key question is: Is your product cheap? If it’s not, then it probably won’t. And what will the brand managers think?

Throughput Checklist

• Track Total Throughput (CTR & Conversion together) for all ad copy variants
• Make sure you isolate the variables
• Track to $ sales only if creative changes impact product choice or quantity
• Creative should match the scope & quality of your offer
• Match landing page with ad copy
• Experiment with different grouping structures
• Homogenize ad groups
• Only exact matches count (regardless of what you have set up)
• Use day-parting to focus on high demand and conversion periods

The engines can improve by adding more transparency into the scoring method, more precision on the actual score or its components, and give the benefit of the doubt when the data is thin – focus on long-term value.

Question & Answer

What should you do if you have an abysmally low Quality Score?

David: If you have a really poor score, a lot of times the problem is the keyword selection. If you have terms that are not relevant to each other in the same ad group, it’s not going to help. One of the key things is taking your keywords and testing them. That’s the best thing. Make sure the terms you’re selecting in your campaign are relevant to the text and each other.

Nick: Do the gut check to see whether or not it (the ad) makes sense from a user perspective. They recently launched a Quality Score diagnostic page – that’s an important thing to look at too.

Mary: She seconds Nick’s suggestion of a "gut check". Scrutinize your site for what a user seems.

Jon: He’s a checklist guy.

How many clicks does it take to establish the initial Quality Score so the campaign is considered mature?

Mary: It’s going to vary by engine. If you’re Google, you need a lot more information. If you’re Microsoft, you need a lot fewer.

David: It depends whether or not you get clicks. Tail terms need a lot fewer than Head terms. We want to give every advertiser an opportunity to get exposure.

Nick: It depends on the keyword and the situation. If there’s a keyword that’s always been bad, the burden is going to be higher because they have this overwhelming set of knowledge that this is a keyword that doesn’t work and it’s really hard to have a positive response for it. Google will throw out everything they know about that keyword. At the same time, that can also work in your favor. If you’re an advertiser with a strong history, you’re going to get the benefit of the doubt there.

[I’m not really liking that in order to do well in Google, you have to have a strong history of excellence. Why is there not an even playing field here? That’s kind of evil sounding.]

What do you think about dynamically inserting keywords into landing pages?

Jon: He says he’s seen some good success with it but he doesn’t recommend straight dynamic keyword insertion. He recommends mapping and inserting them by themes.

Nick: Keyword insertion is a great way to improve user relevance. The user is looking for something specific and keyword insertion is a great way to give them that. There are two things to be careful of. It’s very easy to do keyword insertion in a way that creates nonsensical ads. Users don’t search for the most logical queries. If you’re looking for blue shoes, "buy blue shoes on my site" is good, "buy shoes blue" is bad. Make sure you’re not overpromising in your ad and that you’re actually sending users to a page that is relevant.

How would quality score optimization vary for search versus content?

Nick: History in the question of search vs content, the prevailing strategy was to build a great search campaign and great results would follow after that, but that doesn’t work today. You want to think of search and content as separate. You want to have different campaigns optimized independently. The best strategy is to think of the two networks as distinct. The same tactics will work for both.

Should you create separate landing pages for each search engine?

Jon: No.

Andrew: Goes back to the days when people would try to SEO their sites for different engines. It’s one of those slippery slopes.

David: Maximize the user experience, which in turn will maximize ROI.

Are there any tools available to help you improve your Quality Score and give you reliable signals?

Jon: The focus for us has always been on analytics. Make sure you do your AB testing and that you can easily do the backend tracking that’s going to match a lead to a specific ad.

Nick: Google has some free tools. They have a project called Shining the Light on the Black Box. On your campaign management page, you can add a column that gives you Quality Score information. They also introduced a search query report in the Report Center. Shows the queries that users searched for when they clicked on your ads. It’s good for understanding user intent.

Mary: They’re working on tools but they don’t have any right now.

If an advertisers is unhappy with their QS, what’s the process for getting a human review?

David: Contact your account rep. They do daily reviews of escalations. A lot of times they find out there are fundamental issues with the campaign structure that is driving the problems. They don’t go in and adjust the algorithms based on every complaint

How can I reverse engineers my competitors Quality Score?

Jon: If you’re talking about reverse engineering your competitors score, it’s probably about ad rank.

Lisa Barone is a writer, content marketer & VP of strategy at Overit Media. She's also a very active Twitterer, much to the dismay of the rest of the world.

See Lisa's author page for links to connect on social media.