10,000 contractors told to flag upsetting-offensive content after months of criticism over hate speech, misinformation and fake news in search results
Google is using a 10,000-strong army of independent contractors to flag offensive or upsetting content, in order to ensure that queries like did the Holocaust happen dont push users to misinformation, propaganda and hate speech.
The review of search terms is being done by the companys quality raters, a little-known corps of worldwide contractors that Google uses to assess the quality of its systems. The raters are given searches based on real queries to conduct, and are asked to score the results on whether they meet the needs of users.
These contractors, introduced to the companys review process in 2013, work from a huge manual describing every potential problem they could find with a given search query: whether or not it meets the users expectations, whether the result offered is low or high quality, and whether its spam, porn or illegal.
In a new update to the rating system, rolled out on Tuesday, Google introduced another flag raters could use: the upsetting-offensive mark. Although the company did not cite a specific reason for the update, the move comes three months after the Guardian and Observer began a series of stories showing how the search engine promotes extremist content.
One story in particular highlighted how a search for did the Holocaust happen returned, as its top result, a link to the white supremacist forum Stormfront, explaining how to promote Holocaust denial to others.
That exact search result is now included by Google as one of the examples the company now uses to train its contractors on how and when to mark pages as upsetting-offensive.
Detailing why a result for Holocaust history returning a link to Stormfront should be flagged as problematic, the document explains: This result is a discussion of how to convince others that the Holocaust never happened. Because of the direct relationship between Holocaust denial and antisemitism, many people would consider it offensive.
By contrast, the same search query returning a result for the History Channel should not get the upsetting-offensive flag, even if users do find the topic of the Holocaust upsetting. While the Holocaust itself is a potentially upsetting topic for some, this result is a factually accurate source of historical information, the manual explains. Furthermore, the page does not exist to promote hate or violence against a group of people, contain racial slurs, or depict graphic violence.
Other examples given in the manual for the flag are a query for racism against blacks returning a page for the white supremacist blog Daily Stormer, and a query for Islam returning a result linking to far-right US activist Jan Morgans website.