Google relies on raters, working in countries and languages around the world, to help us measure the quality of our
search results, ranking, and search experience. These raters perform a variety of different kinds of “rating tasks”
designed to give us information about the quality of different kinds of results in response to different kinds of
queries. The data they generate is rolled up statistically to give us within the Google search team a view of the quality
of our search results and search experience over time, as well as an ability to measure the effect of proposed changes
to Google’s search algorithms. Raters’ judgments do not directly impact Google’s search result rankings. While a
rater may give a particular URL a score, that score does not directly increase or decrease a given website’s ranking.
Instead these scores are used in aggregate to evaluate search quality and make decisions about changes.
This document is a “Cliff’s Notes” version of our search quality rating guidelines. By this, we mean that it is not the
entire version that raters use on a daily basis; however, it is a summary of the important topics. The raters’ version
includes instruction on using the rating interface, additional rating examples, etc. These guidelines are used as rating
specifications for search raters, and this document in particular focuses on a core type of rating task called “URL
rating.” In a URL rating task, a rater is shown a search query from their locale (country + language) and a URL that
could be returned by a search engine for that query. The raters “rate” the quality of that result for that query, on a
scale described within the document. Sounds simple, right? As you’ll see, there are many cases to think through, and
this document is used to guide raters on some of those cases and how to look at them.
Our search quality rating guidelines are in constant flux as we learn and search evolves over time. We’ve created this
version especially for those individuals who want to understand better how Google thinks about relevance and quality
of search results.