# US 10,503,827 B2 · Supervised training for word embedding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,503,827 B2 - Document ID: US10503827B2 - Status: Granted - Kind: B2 - Application number: 15/275,029 - Filed: 2016-09-23 - Published: 2018-03-29 - Granted: 2019-12-10 - Earliest claimed: 2016-09-23 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Gakuto Kurata; Masayuki Suzuki; Ryuki Tachibana - Web page: https://patentgrep.com/patent/US10503827B2 ## CPC - G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method and system are provided for training word embedding of domain-specific words. The method includes training, by a processor, a first word embedding, using a general domain corpus, on one or more terms inputted by a user. The method further includes retraining, by the processor, the first word embedding, using a specific domain corpus, for a Neuro-Linguistic Programming task, to create a tuned word embedding. The method also includes training, by the processor, a Neural Network for the Neuro-Linguistic Programming task, using the specific domain corpus. The method additionally includes incorporating, by the processor, the trained Neural Network and tuned word embedding into a Neural Network-based Neuro-Linguistic Programming task. The retraining of the first word embedding and the training of the Neural Network are performed together, and the tuned word embedding is accelerated due to a change in a hyper parameter for domain-specific words. ## Claims ### Claim 1 A method for training word embedding of domain-specific words, comprising: training, by a processor, a first word embedding, using unsupervised data including a general domain corpus, on one or more terms inputted by a user; retraining, by the processor, the first word embedding, using a specific domain corpus, for a Neuro-Linguistic Programming task, to create a tuned word embedding, wherein the retraining of the first word embedding includes incorporating a list of terminologies from the domain specific corpus, and wherein the retraining of the first word embedding includes determining the list of terminologies by: calculating a term frequency-inverse document frequency for each term in the general domain corpus; calculating a term frequency-inverse document frequency for each term in the specific domain corpus; and selecting terms in which a ratio of the term frequency-inverse document frequency for each word in the general corpus to the term frequency-inverse document frequency for each word in the specific domain corpus is larger than a predefined threshold, wherein the retraining of the first word embedding corrects for inappropriate results due to a high degree of cosine similarity between multiple words of phrases in the first word embedding; training, by the processor, a Neural Network for the Neuro-Linguistic Programming task, using the specific domain corpus; and incorporating, by the processor, the trained Neural Network and tuned word embedding into a Neural Network-based Neuro-Linguistic Programming task, wherein the retraining of the first word embedding and the training of the Neural Network are performed together, and wherein the tuned word embedding is accelerated due to a change in a hyper parameter for domain-specific words. ### Claim 2 (depends on 1) The method of claim 1, wherein the hyper parameter is selected from the group consisting of a learning rate, a dropout rate, and a weight for a regularization term, and wherein, if the hyper parameter increases, the learning rate increases, and, if the hyper parameter decreases, the regularization term decreases. ### Claim 3 (depends on 1) The method of claim 1, wherein the trained Neural Network and the tuned word embedding are used in conjunction with a search engine to produce results in response to the one or more terms inputted by the user. ### Claim 4 (depends on 1) The method of claim 1, wherein the Neural Network-based Neuro-Linguistic Programming task is a Convolutional Neural Network-based question answering task. ### Claim 5 (depends on 1) The method of claim 1, wherein: the one or more terms inputted by the user represent a query; the trained Neural Network and the tuned word embedding are incorporated into the Neural Network-based Neuro-Linguistic Programming task to obtain a response to the query; and the incorporating of the trained Neural Network and the tuned word embedding into the Neural Network-based Neuro-Linguistic Programming task includes displaying, using a graphical user interface, the response to the user. ### Claim 6 A system for training word embedding of domain-specific words, comprising: a graphical user interface configured to enable a user to input one or more terms; a processor, configured to: train a first word embedding, using unsupervised data including a general domain corpus, on the one or more terms inputted by a user; retrain the first word embedding, using a specific domain corpus, for a Neuro-Linguistic Programming task, to create a tuned word embedding, wherein the retraining of the first word embedding includes incorporating a list of terminologies from the domain specific corpus, and wherein the retraining of the first word embedding includes determining the list of terminologies by: calculating a term frequency-inverse document frequency for each term in the general domain corpus; calculating a term frequency-inverse document frequency for each term in the specific domain corpus; and selecting terms in which a ratio of the term frequency-inverse document frequency for each word in the general corpus to the term frequency-inverse document frequency for each word in the specific domain corpus is larger than a predefined threshold, wherein the retraining of the first word embedding corrects for inappropriate results due to a high degree of cosine similarity between multiple words of phrases in the first word embedding; train a Neural Network for the Neuro-Linguistic Programming task, using the specific domain corpus; and incorporate the trained Neural Network and tuned word embedding into a Neural Network-based Neuro-Linguistic Programming task; and a memory configured to store the one or more terms and the tuned word embedding, wherein the processor retrains the first word embedding and trains the Neural Network together, and wherein the tuned word embedding is accelerated due to a change in a hyper parameter for domain-specific words. ### Claim 7 (depends on 6) The system of claim 6, wherein the hyper parameter is selected from the group consisting of a learning rate, a dropout rate, and a weight for a regularization term, and wherein, if the hyper parameter increases, the learning rate increases, and, if the hyper parameter decreases, the regularization term decreases. ### Claim 8 (depends on 6) The system of claim 6, wherein the trained Neural Network and the tuned word embedding are used in conjunction with a search engine to produce results in response to the one or more terms inputted by the user. ### Claim 9 (depends on 6) The system of claim 6, wherein the Neural Network-based Neuro-Linguistic Programming task is a Convolutional Neural Network-based question answering task. ### Claim 10 (depends on 6) The system of claim 6, wherein: the one or more terms inputted by the user represent a query; the trained Neural Network and the tuned word embedding are incorporated into the Neural Network-based Neuro-Linguistic Programming task to obtain a response to the query; and the graphical user interface is further configured to display the response to the user. ### Claim 11 A non-transitory computer-readable storage medium including a computer-readable program for training word embedding of domain-specific words, wherein the computer-readable program when executed on a computer causes the computer to perform a method comprising the steps of: training, by a processor, a first word embedding, using unsupervised data including a general domain corpus, on one or more terms inputted by a user; retraining, by the processor, the first word embedding, using a specific domain corpus, for a Neuro-Linguistic Programming task, to create a tuned word embedding, wherein the retraining of the first word embedding includes incorporating a list of terminologies from the domain specific corpus, and wherein the retraining of the first word embedding includes determining the list of terminologies by: calculating a term frequency-inverse document frequency for each term in the general domain corpus; calculating a term frequency-inverse document frequency for each term in the specific domain corpus; and selecting terms in which a ratio of the term frequency-inverse document frequency for each word in the general corpus to the term frequency-inverse document frequency for each word in the specific domain corpus is larger than a predefined threshold, wherein the retraining of the first word embedding corrects for inappropriate results due to a high degree of cosine similarity between multiple words of phrases in the first word embedding; training, by the processor, a Neural Network for the Neuro-Linguistic Programming task, using the specific domain corpus; and incorporating, by the processor, the trained Neural Network and tuned word embedding into a Neural Network-based Neuro-Linguistic Programming task, wherein the retraining of the first word embedding and the training of the Neural Network are performed together, and wherein the tuned word embedding is accelerated due to a change in a hyper parameter for domain-specific words. ### Claim 12 (depends on 11) The computer-readable storage medium of claim 11, wherein the hyper parameter is selected from the group consisting of a learning rate, a dropout rate, and a weight for a regularization term, and wherein, if the hyper parameter increases, the learning rate increases, and, if the hyper parameter decreases, the regularization term decreases. ### Claim 13 (depends on 11) The computer-readable storage medium of claim 11, wherein the Neural Network-based Neuro-Linguistic Programming task is a Convolutional Neural Network-based question answering task. ### Claim 14 (depends on 11) The computer-readable storage medium of claim 11, wherein the trained Neural Network and the tuned word embedding are used in conjunction with a search engine to produce results in response to the one or more terms inputted by the user.