US 11,694,027 B2Granted

Hybrid language detection model

AssigneeServiceNow, Inc.

InventorsAnkit Goel, Prashant Pandey, Jebakumar Mathuram Santhosam Swvigaradoss

Life of this patent

Estimating the term…

Loading patent…

Abstract

An example embodiment may involve a software application executable on computing devices of a remote network management platform containing a computational instance associated with a managed network. A text string may be received, and characters of the string may be categorized among a plurality of symbol script families. A respective likelihood of the string corresponding to each family may be determined, and a respective probability of the string being in each language of each given family may also be determined. The respective probabilities for the languages of each given family may be weighted by the likelihoods of the given family, and then weighted sums of the probabilities for each language may be computed. The maximum of the weighted sums may correspond to the language of the text string. The respective probabilities may be determined according to hybrid N-gram and word language models for each family.

Claims20

Dependent claims

A system comprising:

persistent storage containing representations of a plurality of symbol script families;
and
one or more processors configured to:
obtain a text string comprising a sequence of characters, each of the characters having a respective value of a character encoding scheme;
categorize the characters as belonging to one of the plurality of symbol script families;
determine a respective character count for each given symbol script family of the plurality according to how many characters of the sequence are categorized as belonging to the given symbol script family;
for each respective symbol script family of the plurality, calculate a likelihood that the text string corresponds to a language belonging to the respective symbol script family according to a ratio of the character count for the given symbol script family to a total character count of characters in the sequence of characters of the given text string;
for each particular symbol script family having a non-zero likelihood, compute a respective probability that each of one or more languages belonging to the particular symbol script family is the language of the text string;
for any language belonging to more than one particular symbol script family, replace its respective probability with a sum of its respective probabilities across the more than one particular symbol script family to which it belongs;
and
determine the language of the text string according to a maximum of the respective probabilities.

8 more elements

A method comprising:

obtaining a text string comprising a sequence of characters, each of the characters having a respective value of a character encoding scheme;
categorizing the characters as belonging to one of a plurality of symbol script families;
determining a respective character count for each given symbol script family of the plurality according to how many characters of the sequence are categorized as belonging to the given symbol script family;
for each respective symbol script family of the plurality, calculating a likelihood that the text string corresponds to a language belonging to the respective symbol script family according to a ratio of the character count for the given symbol script family to a total character count of characters in the sequence of characters of the given text string;
for each particular symbol script family having a non-zero likelihood, computing a respective probability that each of one or more languages belonging to the particular symbol script family is the language of the text string;
for any language belonging to more than one particular symbol script family, replacing its respective probability with a sum of its respective probabilities across the more than one particular symbol script family to which it belongs;
and
determining the language of the text string according to a maximum of the respective probabilities.

5 more elements

An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by one or more computing devices of a system comprising persistent storage containing representations of a plurality of symbol script families, cause the system to perform operations comprising:

obtaining a text string comprising a sequence of characters, each of the characters having a respective value of a character encoding scheme;
categorizing the characters as belonging to one of the plurality of symbol script families;
determining a respective character count for each given symbol script family of the plurality according to how many characters of the sequence are categorized as belonging to the given symbol script family;
for each respective symbol script family of the plurality, calculating a likelihood that the text string corresponds to a language belonging to the respective symbol script family according to a ratio of the character count for the given symbol script family to a total character count of characters in the sequence of characters of the given text string;
for each particular symbol script family having a non-zero likelihood, computing a respective probability that each of one or more languages belonging to the particular symbol script family is the language of the text string;
for any language belonging to more than one particular symbol script family, replacing its respective probability with a sum of its respective probabilities across the more than one particular symbol script family to which it belongs;
and
determining the language of the text string according to a maximum of the respective probabilities.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/838,512Filed 13 Jun 2022 · continuation
Priority: 7 May 2020
Published: US 2022/0318498 A16 Oct 2022
Granted: 4 Jul 2023
Term ends: Not estimatedThe source records don’t support an estimate.

Classification4 codes

G06F 40/263
Natural language analysis: language identification
G06F 17/18
Complex mathematical operations: for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression…
G06F 40/58
Processing or translation of natural language: use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for…
G06F
Computing and data processing

Family

Application16/869,155Parent of this continuation · filed 7 May 2020
US 2022/0318498 A1Pre-grant publication · published 6 Oct 2022
US 11,392,768 B2Parent of this continuation
US 2021/0350075 A1Published application
US 11,694,027 B2This patent

From the US filing history. Foreign family members aren’t listed.