Quality Rater Handbooks: Another Tool for SEO Evaluation

You have probably noticed that webpages vary in quality. There are high quality pages: pages that are well written, trustworthy, organized, entertaining, enjoyable, beautiful, compelling, etc. You have probably also found pages that seem poorly written, unreliable, poorly organized, unhelpful, shallow, or even deceptive or malicious.

That’s the World Wide Web for you — as Google describes it. Google updated its handbook for human raters and the new version, dated June 2012, has a whole new section to instruct raters on quality rating landing pages. The new section is “Page Quality Rating Guidelines”. The updated search quality rater’s handbook was leaked on private forums and first reported last week.

I mention it here for a few reasons. First, to alert those who don’t know that Google employs human quality raters. Back in May, Matt Cutts explained human raters’ role in improving the search ranking algorithm. According to Matt, human raters don’t directly influence rankings but rather perform during initial testing phases of proposed changes to the organic ranking algorithm.

Editor’s Update: You can always find Google’s current Quality Rater Guidelines here. The latest update was released May 16, 2019.

Bing has human quality evaluators, too, who follow the “HRS Judging Guidelines.” The most recent version of the document, dated May 2012, was leaked last month. Ratings are query specific and address how well a page meets a searcher’s expectations and needs based on four criteria: intent, scope, authority, and quality:

1. Intent: Does the LP content address a possible intent for the query?
2. Scope: Does the range and depth of the LP content match what the user wants?
3. Authority: Is the trustworthiness of the content on the LP appropriate to the expectations of the user?
4. Quality: Do the appearance and organization of the LP providing a satisfying experience?

Evaluate Your Site By the Search Engines’ Books

Beyond the basic news flash, these leaked manuals are like windows into how search engines evaluate pages. Google asks raters to assign a rating to a page’s main content, layout, and overall website reputation. The latter is assessed by searches for reviews or complaints associated with a site company, brand, or domain. Ratings are to be query independent, as Google says, because page quality comes down to how well the page achieves its purpose (versus page utility which is aligned with relevance to a specific query).

High-quality content is defined in Google’s handbook as “very satisfying, useful, or helpful for its purpose.” Various purposes of a page, according to the handbook, include:

• Share objective, personal, or social information
• Share an opinion
• Entertain
• Share pictures, videos, or other media
• Sell a product or service
• Post questions and answers
• Provide file-sharing or downloads.

There’s an opportunity here to evaluate our own sites next to Google and Bing’s explicit quality markers. The search engines are pretty open about explaining what they expect to see from high-quality content, as we can see in the Google Webmaster quality guidelines and in Bing’s outline of how to build quality content.

But the human rater handbooks go a step beyond a list of dos and don’ts and give us a scorecard which, if turned on our own sites, can identify search engine optimization priorities. Here’s what this process of self-analysis might look like.

1. Make a list of the top landing pages you want to rate. These are your highest money makers and conversion winners, from most vital to least.
2. Note the purpose of the page. You can use the list above if you don’t already have a concise purpose assigned to the page.
3. Evaluate each page on your list using Google’s standards for query-independent page quality and Bing’s standards for meeting query-specific user needs.
4. Identify areas for improvement and get to it.

Resources for further reading:

• Google Search Quality Rating Guidelines Checklist
• What You NEED to Know about Google’s Quality Raters Guidelines

Virginia Nussey is the director of content marketing at MobileMonkey. Prior to joining this startup in 2018, Virginia was the operations and content manager at Bruce Clay Inc., having joined the company in 2008 as a writer and blogger.

See Virginia's author page for links to connect on social media.