Google Search Quality Raters
The new “Page Quality Rating Guidelines” section adds a whopping 32 pages to the handbook that Google provides to those human raters (via contractors like Lionbridge and Leapforce). An existing section on “URL Rating Tasks with User Locations” has been expanded from 12 to 16 pages, so that what was a 125-page document in early 2011 is now 161 pages long.
Version 3.27 of Google’s guidelines for search quality raters dated June 22, 2012 was recently leaked online. It’s actually one part of a larger leak that seems to have happened via private forums, and includes not only the rater guidelines, but also additional information and screenshots from some of the ratings tasks that the group performs. More on that later. Below, a look at the new section of the rater’s handbook, which was written about yesterday by Razvan Gavrilas on CognitiveSEO.com.
Google’s Page Quality Rating Guidelines
Here’s how Google explains the concept of Page Quality to its raters:
You have probably noticed that webpages vary in quality. There are high quality pages: pages that are well written, trustworthy, organized, entertaining, enjoyable, beautiful, compelling, etc. You have probably also found pages that seem poorly written, unreliable, poorly organized, unhelpful, shallow, or even deceptive or malicious. We would like to capture these observations in Page Quality rating.
Raters are asked to give each landing page an overall grade that might include Highest, High, Medium, Low or Lowest, but individual aspects of the page/document like “main content” and “layout” are also graded, as shown on this screenshot from the guidelines:-
Raters are told to ignore the location of the page when rating its quality and to avoid thinking about how helpful the page/document might be for a search query. “Page Quality rating is query-independent, meaning that the rating you assign does not depend on a query,” Google says. That’s because, the guidelines later say, queries help determine the overall utility of a page/document, but the purpose of a page helps determine its quality.
Lowest Quality Main Content (Pandas)
There’s a humorous example in the new material that you have to think Google used purposely. In teaching how to identify “Low or Lowest Quality Main Content” on a page, one of Google’s bullet items is
Using a lot of words to communicate only basic ideas or facts.
If that isn’t one of the hallmarks of a page that you’d find on a content farm, what is, right Content farms were a primary target of the Panda algorithm change an update that aimed to remove “thin” or low quality pages from Google’s index. Now, this new 32-page section of the Google search quality raters guidelines shows how Google grades page quality. It might be the closest thing available to an official, post-Panda quality content guide.
What Else Has Leaked
As I said above, the 161-page raters’ guidelines isn’t the only thing that has leaked recently. Additional documents and screenshots related to the human rater program reportedly began spreading last month via private forums. Gordon Campbell, a UK-based online marketer, recently contacted us with a link to his blog post showing screenshots of some of the side-by-side tasks that raters perform in this case, a “basic” and an “advanced” side-by-side task. The basic task screenshot is first, followed by the advanced task.
Side-by-side tasks ask raters to compare either individualor groups of search results based on a given query. They’re also used to compare Google’s current search results with “new” results that may be used after an algorithm change. I recently contacted the search quality rater that I interviewed earlier this year with a link to Gordon Campbell’s post and screenshots, and the rater confirmed that those are legitimate screenshots of a side-by-side task.
Depending on how widely the other material has leaked, we may be seeing additional blog posts around the web related to Google’s human search quality raters, the new guidelines document and the rating tasks they do.
The new “Page Quality Rating Guidelines” section adds a whopping 32 pages to the handbook that Google provides to those human raters (via contractors like Lionbridge and Leapforce). An existing section on “URL Rating Tasks with User Locations” has been expanded from 12 to 16 pages, so that what was a 125-page document in early 2011 is now 161 pages long.
Version 3.27 of Google’s guidelines for search quality raters dated June 22, 2012 was recently leaked online. It’s actually one part of a larger leak that seems to have happened via private forums, and includes not only the rater guidelines, but also additional information and screenshots from some of the ratings tasks that the group performs. More on that later. Below, a look at the new section of the rater’s handbook, which was written about yesterday by Razvan Gavrilas on CognitiveSEO.com.
Google’s Page Quality Rating Guidelines
Here’s how Google explains the concept of Page Quality to its raters:
You have probably noticed that webpages vary in quality. There are high quality pages: pages that are well written, trustworthy, organized, entertaining, enjoyable, beautiful, compelling, etc. You have probably also found pages that seem poorly written, unreliable, poorly organized, unhelpful, shallow, or even deceptive or malicious. We would like to capture these observations in Page Quality rating.
Raters are asked to give each landing page an overall grade that might include Highest, High, Medium, Low or Lowest, but individual aspects of the page/document like “main content” and “layout” are also graded, as shown on this screenshot from the guidelines:-
Lowest Quality Main Content (Pandas)
There’s a humorous example in the new material that you have to think Google used purposely. In teaching how to identify “Low or Lowest Quality Main Content” on a page, one of Google’s bullet items is
Using a lot of words to communicate only basic ideas or facts.
If that isn’t one of the hallmarks of a page that you’d find on a content farm, what is, right Content farms were a primary target of the Panda algorithm change an update that aimed to remove “thin” or low quality pages from Google’s index. Now, this new 32-page section of the Google search quality raters guidelines shows how Google grades page quality. It might be the closest thing available to an official, post-Panda quality content guide.
As I said above, the 161-page raters’ guidelines isn’t the only thing that has leaked recently. Additional documents and screenshots related to the human rater program reportedly began spreading last month via private forums. Gordon Campbell, a UK-based online marketer, recently contacted us with a link to his blog post showing screenshots of some of the side-by-side tasks that raters perform in this case, a “basic” and an “advanced” side-by-side task. The basic task screenshot is first, followed by the advanced task.
Side-by-side tasks ask raters to compare either individualor groups of search results based on a given query. They’re also used to compare Google’s current search results with “new” results that may be used after an algorithm change. I recently contacted the search quality rater that I interviewed earlier this year with a link to Gordon Campbell’s post and screenshots, and the rater confirmed that those are legitimate screenshots of a side-by-side task.
Depending on how widely the other material has leaked, we may be seeing additional blog posts around the web related to Google’s human search quality raters, the new guidelines document and the rating tasks they do.
Posts
