# US 2021/0240933 A1 · Relation extraction using full dependency forests

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0240933 A1
- Document ID: US20210240933A1
- Status: Published application
- Kind: A1
- Application number: 16/777,261
- Filed: 2020-01-30
- Published: 2021-08-05
- Earliest claimed: 2020-01-30
- Assignees: Tencent America LLC
- Applicants: Tencent America LLC
- Inventors: Linfeng Song
- Web page: https://patentgrep.com/patent/US20210240933A1

## CPC

- G06F 40/205: Parsing
- G06F 40/295: Named entity recognition
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06F 40/211: Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
- G06F 40/279: Recognition of textual entities
- G06F 40/30: Semantic analysis
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/09: Supervised learning
- G06F 16/9027: {Trees}
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06N 5/025: Extracting rules from data
- G10L 15/16: using artificial neural networks
- G10L 15/183: using context dependencies, e.g. language models
- G10L 15/19: Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method, computer program, and computer system is provided for extracting relations between one or more entities in a sentence. A forest corresponding to probabilities of relations between each pair of the entities is generated, and the generated forest generated forest is encoded with relation information for each of the pairs of entities. One or more features are extracted based on the generated forest and the encoded relation information, and a relation is predicted between the entities of each pair of entities based on the extracted features.

## Claims

### Claim 1

A method of extracting relations between one or more entities in a sentence by a computer, the method comprising:
generating a forest corresponding to probabilities of relations between each pair of the entities;
encoding the generated forest with relation information for each of the pairs of entities;
extracting one or more features based on the generated forest and the encoded relation information; and
predicting a relation between the entities of each pair of entities based on the extracted features.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the forest comprises:
determining a contextual representation associated with each of the entities;
calculating a relation score for each of the entities based on the determined contextual representations;
determining a probability associated with a relation between the entities of each pair of entities based on the calculated relation scores.

### Claim 3 (depends on 2)

The method of claim 2, wherein the contextual representations are generated by processing the entities using bidirectional long short-term memory (Bi-LSTM).

### Claim 4 (depends on 1)

The method of claim 1, wherein encoding the forest comprises:
concatenating one or more states associated with each entity;
calculating a weighted sum corresponding to a probability of a relation between one or more of the entities; and
correlating the concatenated word states and the calculated weighted sum.

### Claim 5 (depends on 4)

The method of claim 4, further comprising organizing the correlated word states and weighted sum based on a word order associated the sentence.

### Claim 6 (depends on 1)

The method of claim 1, wherein the extracting one or more features comprises:
correlating representation vectors associated with target mentions corresponding to the entities;
generating one or more convolution kernels;
applying the generated convolution kernels to the correlated representation vectors; and
determining one or more features based on applying max pooling to the correlated representation vectors.

### Claim 7 (depends on 1)

The method of claim 1, wherein the features are extracted using a data-dependent convolutional neural network.

### Claim 8 (depends on 7)

The method of claim 7, wherein the data-dependent convolutional neural network utilizes convolution kernels that are not randomly initialized.

### Claim 9 (depends on 1)

The method of claim 1, wherein the predicting the relation between the entities of each pair of entities based on the extracted features comprises:
combining the extracted features with the contextual representations; and
determining a logistic regression based on the combined extracted features and contextual representations.

### Claim 10 (depends on 1)

The method of claim 1, wherein the forest comprises a three-dimensional tensor that encodes one or more possible dependency relations between the entities and one or more parser confidence scores corresponding to the entities.

### Claim 11

A computer system for extracting relations between one or more entities in a sentence, the computer system comprising:
one or more computer-readable non-transitory storage media configured to store computer program code; and
one or more computer processors configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:
generating code configured to cause the one or more computer processors to generate a forest corresponding to probabilities of relations between each pair of the entities,
encoding code configured to cause the one or more computer processors to encode the generated forest with relation information for each of the pairs of entities;
extracting code configured to cause the one or more computer processors to extract one or more features based on the generated forest and the encoded relation information; and
predicting code configured to cause the one or more computer processors to predict a relation between the entities of each pair of entities based on the extracted features.

### Claim 12 (depends on 11)

The system of claim 11, wherein the generating code comprises:
contextual representation determining code configured to cause the one or more computer processors to determine a contextual representation associated with each of the entities;
calculating code configured to cause the one or more computer processors to calculate a relation score for each of the entities based on the determined contextual representations;
probability determining code configured to cause the one or more computer processors to determine a probability associated with a relation between the entities of each pair of entities based on the calculated relation scores.

### Claim 13 (depends on 12)

The system of claim 12, wherein the contextual representations are generated by processing the entities using bidirectional long short-term memory (Bi-LSTM).

### Claim 14 (depends on 11)

The system of claim 11, wherein encoding code comprises:
concatenating code configured to cause the one or more computer processors to concatenate one or more states associated with each entity;
calculating code configured to cause the one or more computer processors to calculate a weighted sum corresponding to a probability of a relation between one or more of the entities; and
correlating code configured to cause the one or more computer processors to correlate the concatenated word states and the calculated weighted sum.

### Claim 15 (depends on 14)

The system of claim 14, further comprising:
organizing code configured to cause the one or more computer processors to organize the correlated word states and weighted sum based on a word order associated the sentence.

### Claim 16 (depends on 11)

The system of claim 11, wherein the extracting code comprises:
correlating code configured to cause the one or more computer processors to correlate representation vectors associated with target mentions corresponding to the entities;
generating code configured to cause the one or more computer processors to generate one or more convolution kernels;
applying code configured to cause the one or more computer processors to apply the generated convolution kernels to the correlated representation vectors; and
determining code configured to cause the one or more computer processors to determine one or more features based on applying max pooling to the correlated representation vectors.

### Claim 17 (depends on 11)

The system of claim 11, wherein the features are extracted using a data-dependent convolutional neural network.

### Claim 18 (depends on 17)

The system of claim 17, wherein the data-dependent convolutional neural network utilizes convolution kernels that are not randomly initialized.

### Claim 19 (depends on 11)

The system of claim 11, wherein the predicting code comprises:
combining code configured to cause the one or more computer processors to combine the extracted features with the contextual representations; and
determining code configured to cause the one or more computer processors to determine a logistic regression based on the combined extracted features and contextual representations.

### Claim 20

A non-transitory computer readable medium having stored thereon a computer program for extracting relations between one or more entities in a sentence, the computer program configured to cause one or more computer processors to:
generate a forest corresponding to probabilities of relations between each pair of the entities,
encode the generated forest with relation information for each of the pairs of entities;
extract one or more features based on the generated forest and the encoded relation information; and
predict a relation between the entities of each pair of entities based on the extracted features.