# US 2019/0377747 A1 · Utilizing Word Embeddings for Term Matching in Question Answering Systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377747 A1
- Document ID: US20190377747A1
- Status: Published application
- Kind: A1
- Application number: 16/546,770
- Filed: 2019-08-21
- Published: 2019-12-12
- Earliest claimed: 2015-06-02
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: James J. Fan; Chang Wang; Bing Xiang; Bowen Zhou
- Web page: https://patentgrep.com/patent/US20190377747A1

## CPC

- G06F 16/3344: {using natural language analysis}
- G06F 16/24578: {using ranking}
- G06F 16/313: {Selection or weighting of terms for indexing}
- G06F 16/3329: Natural language query formulation
- G06F 40/30: Semantic analysis
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Software that generates an answer to an input question using a source document by performing the following operations: (i) receiving a question; (ii) generating a plurality of vectors including a first vector representation of a term in the question and a second vector representation of a term in a source document; (iii) providing each dimension of each of the first vector representation and the second vector representation into a respective input node of an artificial neural network; (iv) determining whether the source document is relevant to answering the question based, at least in part, on an output generated by the artificial neural network; and (v) in response to determining that the source document is relevant, generating an answer to the question utilizing the source document.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving, by a question answering (QA) system, an input question asked by a user, wherein the QA system is a computer system configured to answer input questions utilizing a plurality of source documents;
generating, by the QA system, a plurality of vectors including a first vector representation of a term in the input question and a second vector representation of a term in a source document of the plurality of source documents, wherein the first vector representation and the second vector representation are generated by combining respective pluralities of dimensions for the first vector representation and the second vector representation to form multi-dimension vectors;
providing, by the QA system, each dimension of each of the first vector representation and the second vector representation as input into a respective input node of an artificial neural network, wherein the artificial neural network is trained to identify pairs of similar vector dimensions;
determining, by the QA system, whether the source document is relevant to answering the input question based, at least in part, on: (i) an output generated by the artificial neural network based on the provided input, and (ii) additional outputs generated by the artificial neural network based on additional inputs, the additional inputs including respective dimensions of respective vector representations for respective input question terms and source document terms corresponding to each possible remaining combination of one input question term and one source document term; and
in response to determining that the source document is relevant to answering the input question, generating, by the QA system, an answer to the input question utilizing the source document.

### Claim 2 (depends on 1)

The method of claim 1, further comprising training, by the QA system, the artificial neural network, wherein training the artificial neural network comprises: (i) providing, at respective input nodes of an input layer of the artificial neural network, vector dimensions of a set of ground truth vector representations corresponding to ground truth question/answer pairs, (ii) providing, at a first output node of an output layer of the artificial neural network, the pairs of vector dimensions of the set of ground truth vector representations that are similar, (iii) providing, at a second output node of the output layer of the artificial neural network, the pairs of vector dimensions of the set of ground truth vector representations that are not similar, and (iv) training the artificial neural network using backpropagation.

### Claim 3 (depends on 1)

The method of claim 1, wherein the determining of whether the source document is relevant to answering the input question is further based, in part, on one or more unsupervised learning methods.

### Claim 4 (depends on 3)

The method of claim 3, wherein the one or more unsupervised learning methods include comparing the first vector representation and the second vector representation using a similarity function.

### Claim 5 (depends on 4)

The method of claim 4, wherein the similarity function is at least one of a cosine similarity function and a Euclidean distance function.

### Claim 6 (depends on 1)

The method of claim 1, wherein the respective pluralities of dimensions for the first vector representation and the second vector representation are determined using dimensionality reduction on a word co-occurrence matrix.

### Claim 7 (depends on 1)

The method of claim 1, further comprising providing, by the QA system, the answer to the user.

### Claim 8

A computer program product comprising a computer readable storage medium having stored thereon:
program instructions to receive, by a question answering (QA) system, an input question asked by a user, wherein the QA system is a computer system configured to answer input questions utilizing a plurality of source documents;
program instructions to generate, by the QA system, a plurality of vectors including a first vector representation of a term in the input question and a second vector representation of a term in a source document of the plurality of source documents, wherein the first vector representation and the second vector representation are generated by combining respective pluralities of dimensions for the first vector representation and the second vector representation to form multi-dimension vectors;
program instructions to provide, by the QA system, each dimension of each of the first vector representation and the second vector representation as input into a respective input node of an artificial neural network, wherein the artificial neural network is trained to identify pairs of similar vector dimensions;
program instructions to determine, by the QA system, whether the source document is relevant to answering the input question based, at least in part, on: (i) an output generated by the artificial neural network based on the provided input, and (ii) additional outputs generated by the artificial neural network based on additional inputs, the additional inputs including respective dimensions of respective vector representations for respective input question terms and source document terms corresponding to each possible remaining combination of one input question term and one source document term; and
program instructions to, in response to determining that the source document is relevant to answering the input question, generate, by the QA system, an answer to the input question utilizing the source document.

### Claim 9 (depends on 8)

The computer program product of claim 8, the computer readable storage medium having further stored thereon program instructions to train, by the QA system, the artificial neural network, wherein training the artificial neural network comprises: (i) providing, at respective input nodes of an input layer of the artificial neural network, vector dimensions of a set of ground truth vector representations corresponding to ground truth question/answer pairs, (ii) providing, at a first output node of an output layer of the artificial neural network, the pairs of vector dimensions of the set of ground truth vector representations that are similar, (iii) providing, at a second output node of the output layer of the artificial neural network, the pairs of vector dimensions of the set of ground truth vector representations that are not similar, and (iv) training the artificial neural network using backpropagation.

### Claim 10 (depends on 8)

The computer program product of claim 8, wherein the determining of whether the source document is relevant to answering the input question is further based, in part, on one or more unsupervised learning methods.

### Claim 11 (depends on 10)

The computer program product of claim 10, wherein the one or more unsupervised learning methods include comparing the first vector representation and the second vector representation using a similarity function.

### Claim 12 (depends on 11)

The computer program product of claim 11, wherein the similarity function is at least one of a cosine similarity function and a Euclidean distance function.

### Claim 13 (depends on 8)

The computer program product of claim 8, wherein the respective pluralities of dimensions for the first vector representation and the second vector representation are determined using dimensionality reduction on a word co-occurrence matrix.

### Claim 14 (depends on 8)

The computer program product of claim 8, the computer readable storage medium having further stored thereon program instructions to provide, by the QA system, the answer to the user.

### Claim 15

A question answering (QA) system configured to answer input questions utilizing a plurality of source documents, the QA system comprising:
one or more processors; and
a computer readable storage medium;
wherein:
the one or more processors are structured, located, connected and/or programmed to run program instructions stored on the computer readable storage medium; and
the stored program instructions include:
program instructions to receive, by the QA system, an input question asked by a user;
program instructions to generate, by the QA system, a plurality of vectors including a first vector representation of a term in the input question and a second vector representation of a term in a source document of the plurality of source documents, wherein the first vector representation and the second vector representation are generated by combining respective pluralities of dimensions for the first vector representation and the second vector representation to form multi-dimension vectors;
program instructions to provide, by the QA system, each dimension of each of the first vector representation and the second vector representation as input into a respective input node of an artificial neural network, wherein the artificial neural network is trained to identify pairs of similar vector dimensions;
program instructions to determine, by the QA system, whether the source document is relevant to answering the input question based, at least in part, on: (i) an output generated by the artificial neural network based on the provided input, and (ii) additional outputs generated by the artificial neural network based on additional inputs, the additional inputs including respective dimensions of respective vector representations for respective input question terms and source document terms corresponding to each possible remaining combination of one input question term and one source document term; and
program instructions to, in response to determining that the source document is relevant to answering the input question, generate, by the QA system, an answer to the input question utilizing the source document.

### Claim 16 (depends on 15)

The QA system of claim 15, the stored program instructions further including program instructions to train, by the QA system, the artificial neural network, wherein training the artificial neural network comprises: (i) providing, at respective input nodes of an input layer of the artificial neural network, vector dimensions of a set of ground truth vector representations corresponding to ground truth question/answer pairs, (ii) providing, at a first output node of an output layer of the artificial neural network, the pairs of vector dimensions of the set of ground truth vector representations that are similar, (iii) providing, at a second output node of the output layer of the artificial neural network, the pairs of vector dimensions of the set of ground truth vector representations that are not similar, and (iv) training the artificial neural network using backpropagation.

### Claim 17 (depends on 15)

The QA system of claim 15, wherein the determining of whether the source document is relevant to answering the input question is further based, in part, on one or more unsupervised learning methods.

### Claim 18 (depends on 17)

The QA system of claim 17, wherein the one or more unsupervised learning methods include comparing the first vector representation and the second vector representation using a similarity function.

### Claim 19 (depends on 15)

The QA system of claim 15, wherein the respective pluralities of dimensions for the first vector representation and the second vector representation are determined using dimensionality reduction on a word co-occurrence matrix.

### Claim 20 (depends on 15)

The QA system of claim 15, the stored program instructions further including program instructions to provide, by the QA system, the answer to the user.
