# US 12,223,279 B2 · Method for generating cross-lingual textual semantic model, and electronic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,279 B2 - Document ID: US12223279B2 - Status: Granted - Kind: B2 - Application number: 18/054,608 - Filed: 2022-11-11 - Published: 2023-03-16 - Granted: 2025-02-11 - Earliest claimed: 2021-12-29 - Assignees: Beijing Baidu Netcom Science Technology Co., Ltd. - Applicants: Beijing Baidu Netcom Science Technology Co., Ltd. - Inventors: Yaqian Han; Shuohuan Wang; Yu Sun - Web page: https://patentgrep.com/patent/US12223279B2 ## CPC - G06F 40/30: Semantic analysis - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06N 3/045: Combinations of networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method for generating a cross-lingual textual semantic model includes: acquiring a set of training data that includes pieces of monolingual non-parallel text and pieces of bilingual parallel text; determining a semantic vector of each piece of text in the set of training data by inputting each piece of text into an initial textual semantic model; determining a distance between semantic vectors of each two pieces of text in the set of training data based on the semantic vector of each piece of text in the set of training data; determining a gradient modification based on a parallel relationship between each two pieces of text in the set of training data and the distance between the semantic vectors of each two pieces of text in the set of training data; and acquiring a modified textual semantic model by modifying the initial textual semantic model based on the gradient modification. ## Claims ### Claim 1 A method for generating a cross-lingual textual semantic model, comprising: acquiring a set of training data, wherein the set of training data comprises pieces of monolingual non-parallel text and pieces of bilingual parallel text; determining a semantic vector of each piece of text in the set of training data by inputting each piece of text in the set of training data into an initial textual semantic model; determining a distance between semantic vectors of each two pieces of text in the set of training data based on the semantic vector of each piece of text in the set of training data; determining a gradient modification based on a parallel relationship between each two pieces of text in the set of training data and the distance between the semantic vectors of each two pieces of text in the set of training data; and acquiring a modified textual semantic model by modifying the initial textual semantic model based on the gradient modification. ### Claim 2 (depends on 1) The method of claim 1, wherein, determining the gradient modification based on the parallel relationship between each two pieces of text in the set of training data and the distance between the semantic vectors of each two pieces of text in the set of training data, comprises: determining a first semantic vector distance between a first piece of text and a second piece of text and second semantic vector distances between the first piece of text and remaining pieces of text in response to the first piece of text and the second piece of text being parallel; determining a loss value based on the first semantic vector distance and the second semantic vector distances; and determining the gradient modification based on the loss value. ### Claim 3 (depends on 2) The method of claim 2, wherein, determining the loss value based on the first semantic vector distance and the second semantic vector distances, comprises: determining the loss value based on a difference value between the first semantic vector distance and each of the second semantic vector distances; or determining the loss value based on a difference value between the first semantic vector distance and a first threshold and a difference value between each of the second semantic vector distances and a second threshold; or determining the loss value based on a ratio of the first semantic vector distance to a sum of semantic vector distances, wherein, the sum of semantic vector distances is a sum of the first semantic vector distance and the second semantic vector distances. ### Claim 4 (depends on 1) The method of claim 1, wherein, determining the gradient modification based on the parallel relationship between each two pieces of text in the set of training data and the distance between the semantic vectors of each two pieces of text in the set of training data, comprises: acquiring a plurality of training subsets by dividing the set of training data based on a number of pieces of text comprised in the set of training data; and determining a gradient modification of each of the plurality of training subsets based on a parallel relationship between each two pieces of text in each of the plurality of training subsets and a distance between semantic vectors of each two pieces of text in each of the plurality of training subsets. ### Claim 5 (depends on 1) The method of claim 1, further comprising: acquiring text to be recognized; and determining a semantic vector corresponding to the text to be recognized by inputting the text to be recognized into the modified textual semantic model. ### Claim 6 An electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to: acquire a set of training data, wherein the set of training data comprises pieces of monolingual non-parallel text and pieces of bilingual parallel text; determine a semantic vector of each piece of text in the set of training data by inputting each piece of text in the set of training data into an initial textual semantic model; determine a distance between semantic vectors of each two pieces of text in the set of training data based on the semantic vector of each piece of text in the set of training data; determine a gradient modification based on a parallel relationship between each two pieces of text in the set of training data and the distance between the semantic vectors of each two pieces of text in the set of training data; and acquire a modified textual semantic model by modifying the initial textual semantic model based on the gradient modification. ### Claim 7 (depends on 6) The device of claim 6, wherein the processor is configured to: determine a first semantic vector distance between a first piece of text and a second piece of text and second semantic vector distances between the first piece of text and remaining pieces of text in response to the first piece of text and the second piece of text being parallel; determine a loss value based on the first semantic vector distance and the second semantic vector distances; and determine the gradient modification based on the loss value. ### Claim 8 (depends on 7) The device of claim 7, wherein the processor is configured to: determine the loss value based on a difference value between the first semantic vector distance and each of the second semantic vector distances; or determine the loss value based on a difference value between the first semantic vector distance and a first threshold and a difference value between each of the second semantic vector distances and a second threshold; or determine the loss value based on a ratio of the first semantic vector distance to a sum of semantic vector distances, wherein, the sum of semantic vector distances is a sum of the first semantic vector distance and the second semantic vector distances. ### Claim 9 (depends on 6) The device of claim 6, wherein the processor is configured to: acquire a plurality of training subsets by dividing the set of training data based on a number of pieces of text comprised in the set of training data; and determine a gradient modification of each of the plurality of training subsets based on a parallel relationship between each two pieces of text in each of the plurality of training subsets and a distance between semantic vectors of each two pieces of text in each of the plurality of training subsets. ### Claim 10 (depends on 6) The device of claim 6, wherein the processor is configured to: acquire text to be recognized; and determine a semantic vector corresponding to the text to be recognized by inputting the text to be recognized into the modified textual semantic model. ### Claim 11 A non-transitory computer-readable storage medium stored with computer instructions, wherein, the computer instructions are configured to cause a computer to perform a method for generating a cross-lingual textual semantic model, the method comprising: acquiring a set of training data, wherein the set of training data comprises pieces of monolingual non-parallel text and pieces of bilingual parallel text; determining a semantic vector of each piece of text in the set of training data by inputting each piece of text in the set of training data into an initial textual semantic model; determining a distance between semantic vectors of each two pieces of text in the set of training data based on the semantic vector of each piece of text in the set of training data; determining a gradient modification based on a parallel relationship between each two pieces of text in the set of training data and the distance between the semantic vectors of each two pieces of text in the set of training data; and acquiring a modified textual semantic model by modifying the initial textual semantic model based on the gradient modification. ### Claim 12 (depends on 11) The storage medium of claim 11, wherein, determining the gradient modification based on the parallel relationship between each two pieces of text in the set of training data and the distance between the semantic vectors of each two pieces of text in the set of training data, comprises: determining a first semantic vector distance between a first piece of text and a second piece of text and second semantic vector distances between the first piece of text and remaining pieces of text in response to the first piece of text and the second piece of text being parallel; determining a loss value based on the first semantic vector distance and the second semantic vector distances; and determining the gradient modification based on the loss value. ### Claim 13 (depends on 12) The storage medium of claim 12, wherein, determining the loss value based on the first semantic vector distance and the second semantic vector distances, comprises: determining the loss value based on a difference value between the first semantic vector distance and each of the second semantic vector distances; or determining the loss value based on a difference value between the first semantic vector distance and a first threshold and a difference value between each of the second semantic vector distances and a second threshold; or determining the loss value based on a ratio of the first semantic vector distance to a sum of semantic vector distances, wherein, the sum of semantic vector distances is a sum of the first semantic vector distance and the second semantic vector distances. ### Claim 14 (depends on 11) The storage medium of claim 11, wherein, determining the gradient modification based on the parallel relationship between each two pieces of text in the set of training data and the distance between the semantic vectors of each two pieces of text in the set of training data, comprises: acquiring a plurality of training subsets by dividing the set of training data based on a number of pieces of text comprised in the set of training data; and determining a gradient modification of each of the plurality of training subsets based on a parallel relationship between each two pieces of text in each of the plurality of training subsets and a distance between semantic vectors of each two pieces of text in each of the plurality of training subsets. ### Claim 15 (depends on 11) The storage medium of claim 11, wherein, the method further comprises: acquiring text to be recognized; and determining a semantic vector corresponding to the text to be recognized by inputting the text to be recognized into the modified textual semantic model.