# US 2023/0267286 A1 · Translation model training method, translation method, apparatus, device, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0267286 A1 - Document ID: US20230267286A1 - Status: Published application - Kind: A1 - Application number: 17/879,965 - Filed: 2022-08-03 - Published: 2023-08-24 - Earliest claimed: 2022-02-22 - Assignees: Beijing Baidu Netcom Science Technology Co., Ltd. - Applicants: Beijing Baidu Netcom Science Technology Co., Ltd. - Inventors: Liwen Zhang; Meng Sun; Zhongjun He; Zhi Li - Web page: https://patentgrep.com/patent/US20230267286A1 ## CPC - G06F 40/35: Discourse or dialogue representation - G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation - G06F 40/211: Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars - G06F 40/44: Statistical methods, e.g. probability models - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Provided are a translation model training method, a translation method, a device, and a storage medium, and relates to a field of computer technology, and in particular, to artificial intelligence fields such as natural language processing, machine translation and the like. The translation model training method includes: processing a sample document, to obtain an RST discourse structure tree in a dependency form of the sample document, a side in the RST discourse structure tree in the dependency form indicating an RST relationship in a discourse of the sample document; determining an attention mechanism of a translation model to be trained, based on the RST relationship in the RST discourse structure tree in the dependency form; and inputting the RST discourse structure tree in the dependency form and the sample document into the translation model to be trained for training, to obtain a trained translation model. ## Claims ### Claim 1 A translation model training method, comprising: processing a sample document, to obtain an RST discourse structure tree in a dependency form of the sample document, a side in the RST discourse structure tree in the dependency form indicating an RST relationship in a discourse of the sample document; determining an attention mechanism of a translation model to be trained, based on the RST relationship in the RST discourse structure tree in the dependency form; and inputting the RST discourse structure tree in the dependency form and the sample document into the translation model to be trained for training, to obtain a trained translation model. ### Claim 2 (depends on 1) The method of claim 1, wherein the translation model adopts a transformer model, and determining the attention mechanism of the translation model to be trained, based on the RST relationship in the RST discourse structure tree in the dependency form, comprises: obtaining an attention value, based on an RST relationship matrix corresponding to the side in the RST discourse structure tree in the dependency form, a query matrix, a key matrix, and a value matrix. ### Claim 3 (depends on 2) The method of claim 2, wherein determining the attention mechanism of the translation model to be trained, based on the RST relationship in the RST discourse structure tree in the dependency form, further comprises: performing a linear transformation on a discourse representation of the sample document, to obtain the query matrix, the key matrix, and the value matrix. ### Claim 4 (depends on 2) The method of claim 2, wherein determining the attention mechanism of the translation model to be trained, based on the RST relationship in the RST discourse structure tree in the dependency form, further comprises: determining an attention score of a word wi and a word wj in the sample document based on a query vector Qi corresponding to the word wi, an RST relationship matrix Rij between a sentence containing the word wi and a sentence containing the word wj, and a transposition KjT of a key vector corresponding to the word wj. ### Claim 5 (depends on 3) The method of claim 3, wherein determining the attention mechanism of the translation model to be trained, based on the RST relationship in the RST discourse structure tree in the dependency form, further comprises: determining an attention score of a word wi and a word wj in the sample document based on a query vector Qi corresponding to the word wi, an RST relationship matrix Rij between a sentence containing the word wi and a sentence containing the word wj, and a transposition KjT of a key vector corresponding to the word wj. ### Claim 6 (depends on 4) The method of claim 4, wherein the RST relationship matrix Rij between the sentence containing the word wi and the sentence containing the word wj comprises an RST relationship matrix corresponding to a side of the sentence containing the word wi and sentence containing the word wj in the RST discourse structure tree in the dependency form. ### Claim 7 (depends on 5) The method of claim 5, wherein the RST relationship matrix Rij between the sentence containing the word wi and the sentence containing the word wj comprises an RST relationship matrix corresponding to a side of the sentence containing the word wi and sentence containing the word wj in the RST discourse structure tree in the dependency form. ### Claim 8 (depends on 4) The method of claim 4, wherein in a case of the sentence containing the word wi and the sentence containing the word wj do not have a corresponding side in the RST discourse structure tree, the RST relationship matrix Rij between the sentence containing the word wi and the sentence containing the word wj is negative infinity. ### Claim 9 (depends on 5) The method of claim 5, wherein in a case of the sentence containing the word wi and the sentence containing the word wj do not have a corresponding side in the RST discourse structure tree, the RST relationship matrix Rij between the sentence containing the word wi and the sentence containing the word wj is negative infinity. ### Claim 10 (depends on 6) The method of claim 6, wherein in a case of the sentence containing the word wi and the sentence containing the word wj do not have a corresponding side in the RST discourse structure tree, the RST relationship matrix Tij between the sentence containing the word wi and the sentence containing the word wj is negative infinity. ### Claim 11 (depends on 7) The method of claim 7, wherein in a case of the sentence containing the word wi and the sentence containing the word wj do not have a corresponding side in the RST discourse structure tree, the RST relationship matrix Rij between the sentence containing the word wi and the sentence containing the word wj is negative infinity. ### Claim 12 (depends on 1) The method of claim 1, wherein processing the sample document, to obtain the RST discourse structure tree, comprises: parsing the sample document, to obtain an RST discourse structure tree in a constituency form of the sample document; and transforming the RST discourse structure tree in the constituency form into the RST discourse structure tree in the dependency form. ### Claim 13 (depends on 1) A translation method, comprising: processing a document to be processed, to obtain an RST discourse structure tree in a dependency form of the document to be processed, a side in the RST discourse structure tree in the dependency form indicating an RST relationship in a discourse of the document to be processed; and inputting the RST discourse structure tree in the dependency form and the document to be processed into a trained translation model for performing a translation, to obtain a target document; wherein the trained translation model is obtained by performing training using the translation model training method of claim 1. ### Claim 14 An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores an instruction executable by the at least one processor, and the instruction is executed by the at least one processor to cause the at least one processor to execute: processing a sample document, to obtain an RST discourse structure tree in a dependency form of the sample document, a side in the RST discourse structure tree in the dependency form indicating an RST relationship in a discourse of the sample document; determining an attention mechanism of a translation model to be trained, based on the RST relationship in the RST discourse structure tree in the dependency form; and inputting the RST discourse structure tree in the dependency form and the sample document into the translation model to be trained for training, to obtain a trained translation model. ### Claim 15 (depends on 14) The electronic device of claim 14, wherein the translation model adopts a transformer model, and the instruction is executed by the at east one processor to cause the at least one processor to execute: obtaining an attention value, based on an RST relationship matrix corresponding to the side in the RST discourse structure tree in the dependency form, a query matrix, a key matrix, and a value matrix. ### Claim 16 (depends on 15) The electronic device of claim 15, wherein the instruction is executed by the at least one processor to cause the at least one processor to execute: performing a linear transformation on a discourse representation of the sample document, to obtain the query matrix, the key matrix, and the value matrix. ### Claim 17 (depends on 15) The electronic device of claim 15, wherein the instruction is executed by the at least one processor to cause the at least one processor to execute: determining an attention score of a word wi and a word wj in the sample document based on a query vector Qi corresponding to the word wi, an RST relationship matrix Rij between a sentence containing the word wi and a sentence containing the word wj, and a transposition KjT of a key vector corresponding to the word wj. ### Claim 18 (depends on 13) An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores an instruction executable by the at least one processor, and the instruction is executed by the at least one processor to cause the at least one processor to execute the method of claim 13. ### Claim 19 A non-transitory computer-readable storage medium storing a computer instruction, wherein the computer instruction is used to cause a computer to execute: processing a sample document, to obtain an RST discourse structure tree in a dependency form of the sample document, a side in the RST discourse structure tree in the dependency form indicating an RST relationship in a discourse of the sample document; determining an attention mechanism of a translation model to be trained, based on the RST relationship in the RST discourse structure tree in the dependency form; and inputting the RST discourse structure tree in the dependency form and the sample document into the translation model to be trained fir training, to obtain a trained translation model. ### Claim 20 (depends on 13) A non-transitory computer-readable storage medium storing a computer instruction, wherein the computer instruction is used to cause a computer to execute the method of claim 13.