# US 12,762,193 B2 · Preventing the distribution of forbidden network content using automatic variant detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,193 B2 - Document ID: US12762193B2 - Status: Granted - Kind: B2 - Application number: 18/071,306 - Filed: 2022-11-29 - Published: 2023-03-23 - Granted: 2026-10-13 - Earliest claimed: 2016-12-09 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Yintao Liu; Vaibhav Vaish; Rachel Xu; Zhaofu Chen - Web page: https://patentgrep.com/patent/US12762193B2 ## CPC - G06F 16/9535: Search customisation based on user profiles and personalisation - G06F 16/328: {Management therefor} - G06F 16/3349: Reuse of stored results of previous queries - G06F 16/355: Creation or modification of classes or clusters - G06F 16/374: {Thesaurus} - G06F 16/9035: Filtering based on additional data, e.g. user or group profiles - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The subject matter of this specification generally relates to preventing the distribution of forbidden network content. In one aspect, a system includes a front-end server that receives content for distribution over a data communication network. The back-end server identifies, in the query log, a set of received queries for which a given forbidden term was used to identify a search result in response to the received query even though the given forbidden term was not included in queries included in the set of received queries. The back-end server classifies, as variants of the given forbidden term, a term from one or more queries in the set of received queries that caused a search engine to use the given forbidden term to identify one or more search results in response to the one or more queries and prevents distribution of content that includes a variant. ## Claims ### Claim 1 A system comprising: one or more processors; and one or more storage devices storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving content for distribution over a data communication network; updating a list of forbidden variants of forbidden terms for use in blocking distribution of content that presents any of the forbidden variants, the updating comprising: accessing a data source that includes a set of received queries that were received from client devices and for which results were returned to the client devices; identifying, in the data source, a subset of the received queries that were each adjusted to include a forbidden term in place of, or in addition to, one or more words of the received query prior to using the received query to identify a result in response to the received query; identifying, in the subset of received queries, candidate variants of the forbidden term, wherein each candidate variant is a word or phrase that (i) is included in at least one received query of the subset of received queries and (ii) caused a search engine to adjust the at least one received query to include the forbidden term; determining a score for each candidate variant based on a number of the received queries in the subset of received queries for which the candidate variant caused the search engine to adjust the received query to include the forbidden term in the received query; selecting, from the candidate variants, a set of forbidden variants of the forbidden term based at least on the score for each candidate variant; and adding the set of forbidden variants to the list of forbidden variants; and preventing distribution of content that depicts any forbidden variant in the list of forbidden variants. ### Claim 2 (depends on 1) The system of claim 1, wherein identifying the subset of received queries comprises identifying a given received query that was generated adjusted by the search engine by expanding the given received query to include the forbidden term. ### Claim 3 (depends on 1) The system of claim 1, wherein the operations further comprise identifying, using a semantic network of terms, a term semantically linked to the forbidden term as a candidate variant of the forbidden term. ### Claim 4 (depends on 1) The system of claim 1, wherein selecting, from the candidate variants of the forbidden term, the set of forbidden variants of the forbidden term comprises: generating a ranking of the candidate variants by ordering the candidate variants based on the score for each candidate variant; and selecting, as the forbidden variants of the forbidden term, one or more of the candidate variants from the ranking of the candidate variants based on the score for each candidate variant. ### Claim 5 (depends on 4) The system of claim 4, wherein: the set of candidate variants includes a first candidate variant for which a spelling of the first candidate variant was corrected to the forbidden term and a second candidate variant that was added to a received query that included the forbidden term; the score for the first candidate variant is further based on an edit distance between the first candidate variant and the forbidden term; and the score for the second candidate variant is further based on inverse document frequency score for the second candidate variant. ### Claim 6 (depends on 1) The system of claim 1, wherein identifying, in the data source, the subset of received queries comprises using a map procedure to identify, from the data source, candidate variants of each forbidden term. ### Claim 7 (depends on 6) The system of claim 6, wherein selecting, from the candidate variants of the forbidden term, the set of forbidden variants of the forbidden term comprises using a reduce procedure for the forbidden term to select, from the candidate variants for the forbidden term, one or more forbidden variants of the forbidden term, wherein each reduce procedure is performed on a separate back-end server. ### Claim 8 A method for preventing distribution of forbidden content, comprising: receiving content for distribution over a data communication network; updating a list of forbidden variants of forbidden terms for use in blocking distribution of content that presents any of the forbidden variants, the updating comprising: accessing a data source that includes a set of received queries that were received from client devices and for which results were returned to the client devices; identifying, in the data source, a subset of the received queries that were each adjusted to include a forbidden term in place of, or in addition to, one or more words of the received query prior to using the received query to identify a result in response to the received query; identifying, in the subset of received queries, candidate variants of the forbidden term, wherein each candidate variant is a word or phrase that (i) is included in at least one received query of the subset of received query and (ii) caused a search engine to adjust the at least one received query to include the forbidden term; determining a score for each candidate variant based on a number of the received queries in the subset of received queries for which the candidate variant caused the search engine to adjust the received queries to include the forbidden term in the received query; selecting, from the candidate variants, a set of forbidden variants of the forbidden term based at least on the score for each candidate variant; and adding the set of forbidden variants to the list of forbidden variants; and preventing distribution of content that depicts any forbidden variant in the list of forbidden variants. ### Claim 9 (depends on 8) The method of claim 8, wherein identifying the subset of received queries comprises identifying a given received query that was generated adjusted by the search engine by expanding the given received query to include the forbidden term. ### Claim 10 (depends on 8) The method of claim 8, further comprising identifying, using a semantic network of terms, a term semantically linked to the forbidden term as a candidate variant of the forbidden term. ### Claim 11 (depends on 8) The method of claim 8, wherein selecting, from the candidate variants of the forbidden term, the set of forbidden variants of the forbidden term comprises: generating a ranking of the candidate variants by ordering the candidate variants based on the score for each candidate variant; and selecting, as the forbidden variants of the forbidden term, one or more of the candidate variants from the ranking of the candidate variants based on the score for each candidate variant. ### Claim 12 (depends on 11) The method of claim 11, wherein: the set of candidate variants includes a first candidate variant for which a spelling of the first candidate variant was corrected to the forbidden term and a second candidate variant that was added to a received query that included the forbidden term; the score for the first candidate variant is further based on an edit distance between the first candidate variant and the forbidden term; and the score for the second candidate variant is further based on inverse document frequency score for the second candidate variant. ### Claim 13 (depends on 8) The method of claim 8, wherein identifying, in the data source, the subset of received queries comprises using a map procedure to identify, from the data source, candidate variants of each forbidden term. ### Claim 14 (depends on 13) The method of claim 13, wherein selecting, from the candidate variants of the forbidden term, the set of forbidden variants of the forbidden term comprises using a reduce procedure for the forbidden term to select, from the candidate variants for the forbidden term, one or more forbidden variants of the forbidden term, wherein each reduce procedure is performed on a separate back-end server. ### Claim 15 A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by one or more data processing apparatus cause the data processing apparatus to perform operations comprising: receiving content for distribution over a data communication network; updating a list of forbidden variants of forbidden terms for use in blocking distribution of content that presents any of the forbidden variants, the updating comprising: accessing a data source that includes a set of received queries that were received from client devices and for which results were returned to the client devices; identifying, in the data source, a subset of the received queries that were each adjusted to include a forbidden term in place of, or in addition to, one or more words of the received query prior to using the received query to identify a result in response to the received query; identifying, in the subset of received queries, candidate variants of the forbidden term, wherein each candidate variant is a word or phrase that (i) is included in at least one received query of the subset of received queries and (ii) caused a search engine to adjust the at least one received query to include the forbidden term; determining a score for each candidate variant based on a number of the received queries in the subset of received queries for which the candidate variant caused the search engine to adjust the received query to include the forbidden term in the received query; selecting, from the candidate variants, a set of forbidden variants of the forbidden term based at least on the score for each candidate variant; and adding the set of forbidden variants to the list of forbidden variants; and preventing distribution of content that depicts any forbidden variant in the list of forbidden variants. ### Claim 16 (depends on 15) The non-transitory computer storage medium of claim 15, wherein identifying the subset of received queries comprises identifying a given received query that was generated adjusted by the search engine by expanding the given received query to include the forbidden term. ### Claim 17 (depends on 15) The non-transitory computer storage medium of claim 15, wherein the operations further comprises identifying, using a semantic network of terms, a term semantically linked to the forbidden term as a candidate variant of the forbidden term. ### Claim 18 (depends on 15) The non-transitory computer storage medium of claim 15, wherein selecting, from the candidate variants of the forbidden term, the set of forbidden variants of the forbidden term comprises: generating a ranking of the candidate variants by ordering the candidate variants based on the score for each candidate variant; and selecting, as the forbidden variants of the forbidden term, one or more of the candidate variants from the ranking of the candidate variants based on the score for each candidate variant. ### Claim 19 (depends on 18) The non-transitory computer storage medium of claim 18, wherein: the set of candidate variants includes a first candidate variant for which a spelling of the first candidate variant was corrected to the forbidden term and a second candidate variant that was added to a received query that included the forbidden term; the score for the first candidate variant is further based on an edit distance between the first candidate variant and the forbidden term; and the score for the second candidate variant is further based on inverse document frequency score for the second candidate variant. ### Claim 20 (depends on 15) The non-transitory computer storage medium of claim 15, wherein identifying, in the data source, the subset of received queries comprises using a map procedure to identify, from the data source, candidate variants of each forbidden term.