# US 2023/0267322 A1 · Method and system for aspect-level sentiment classification by merging graphs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267322 A1
- Document ID: US20230267322A1
- Status: Published application
- Kind: A1
- Application number: 17/676,775
- Filed: 2022-02-21
- Published: 2023-08-24
- Earliest claimed: 2022-02-21
- Assignees: Chinabank Payment (Beijing) Technology Co., Ltd.
- Applicants: Chinabank Payment (Beijing) Technology Co., Ltd.
- Inventors: Xiaochen Hou; Peng Qi; Guangtao Wang; Zhitao Ying; Jing Huang; Xiaodong He; Bowen Zhou
- Web page: https://patentgrep.com/patent/US20230267322A1

## CPC

- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06N 3/08: Learning methods
- G06F 18/213: Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F 18/29: {Graphical models, e.g. Bayesian networks}
- G06F 40/205: Parsing
- G06F 40/279: Recognition of textual entities
- G06F 40/30: Semantic analysis
- G06N 3/045: Combinations of networks
- G06N 3/048: Activation functions
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06F 40/211: Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

System and method for aspect-level sentiment classification. The system includes a computing device, the computing device has a processer and a storage device storing computer executable code. The computer executable code is configured to: receive an aspect term-sentence pair; embed the aspect term-sentence pair; parse the sentence using multiple parsers to obtain dependency trees, and perform edge union to obtain a merged graph; combine the embedding and the merged graph to obtain a relation graph; perform a relation graph neural network on the relation graph; extract hidden representation of the aspect term from updated relation neural network; and classify the aspect term based on the extracted representation to obtain a predicted classification label of the aspect term. During training, the computer executable code is further configured to calculate a loss function based on the predicted label and the ground truth label, and adjust parameters of models.

## Claims

### Claim 1

A system comprising a computing device, the computing device comprising a processer and a storage device storing computer executable code, wherein the computer executable code, when executed at the processor, is configured to:
receive an aspect term-sentence pair, the aspect term-sentence pair comprising an aspect term and a sentence, and the sentence comprising the aspect term;
embed the aspect term-sentence pair to obtain embedding of each word in the sentence;
parse the sentence using a plurality of parsers to obtain a plurality of dependency trees;
perform edge union on the plurality of dependency trees to obtain a merged graph, each node and each edge in the plurality of dependency trees included in the merged graph;
represent each node in the merged graph by corresponding one of the embeddings of the words to obtain a relation graph;
perform a relation neural network on the relation graph to obtain updated relation neural network;
extract hidden representation of the aspect term from the updated relation neural network to obtain extracted representation of the aspect term; and
classify the aspect term based on the extracted representation to obtain a predicted classification label of the aspect term.

### Claim 2 (depends on 1)

The system of claim 1, wherein the computer executable code is further configured to, when the aspect term-sentence pair is a training aspect term-sentence pair and the aspect term has a classification label in regard to the sentence:
receive the classification label of the aspect term;
calculate a loss function based on the predicted classification label of the aspect term and the classification label of the aspect term; and
adjust parameters of models in the computer executable code based on the loss function.

### Claim 3 (depends on 1)

The system of claim 1, wherein the computer executable code is further configured to add a position dimension to the embedding of each of the words.

### Claim 4 (depends on 1)

The system of claim 1, wherein the number of the plurality of parsers is three, and the three parsers are CoreNLP, Stanza, and Berkeley neural parser.

### Claim 5 (depends on 1)

The system of claim 1, wherein the computer executable code is further configured to add a child-to-parent relation to each parent-to-child relation to the merged graph, and add a self-loop to each node in the merged graph, so as to obtain the relation graph.

### Claim 6 (depends on 1)

The system of claim 1, wherein the relation neural network is a relational graph attention network (RGAT).

### Claim 7 (depends on 6)

The system of claim 6, wherein hidden representation of node i in the relation graph at l-th layer of the RGAT is calculated by:
hi(l)=∥k=1Kσ(Σj∈NiαijkWkhi(l-1,k)),
wherein K is a number of attention heads, Ni is a neighborhood of node i in the relation graph, ∥ is a concatenation operation, Wk represents learnable weights in the RGAT, and σ represents a ReLU activation function, and αijk is an attention score between node i and node j with head k.

### Claim 8 (depends on 1)

The system of claim 1, wherein the computer executable code is further configured to, when the aspect term comprises at least two words, average extracted hidden representations of the at least two words to obtain the hidden representation of the aspect term.

### Claim 9 (depends on 1)

The system of claim 1, wherein the computer executable code is configured to classify the aspect term using a two layer multilayer perceptron (MLP).

### Claim 10 (depends on 1)

The system of claim 1, wherein the classifications comprise a positive sentiment, a neutral sentiment, and negative sentiment.

### Claim 11

A method comprising:
receiving, by a computing device, an aspect term-sentence pair, the aspect term-sentence pair comprising an aspect term and a sentence, and the sentence comprising the aspect term;
embedding, by the computing device, the aspect term-sentence pair to obtain embedding of each word in the sentence;
parsing, by the computing device, the sentence using a plurality of parsers to obtain a plurality of dependency trees;
performing edge union, by the computing device, on the plurality of dependency trees to obtain a merged graph, each node and each edge in the plurality of dependency trees included in the merged graph;
representing, by the computing device, each node in the merged graph by corresponding one of the embeddings of the words to obtain a relation graph;
performing a relation neural network, by the computing device, on the relation graph to obtain updated relation neural network;
extracting, by the computing device, hidden representation of the aspect term from the updated relation neural network to obtain extracted representation of the aspect term; and
classifying, by the computing device, the aspect term based on the extracted representation to obtain a predicted classification label of the aspect term.

### Claim 12 (depends on 11)

The method of claim 11, further comprising, when the aspect term-sentence pair is a training aspect term-sentence pair and the aspect term has a classification label in regard to the sentence:
receiving the classification label of the aspect term;
calculating a loss function based on the predicted classification label of the aspect term and the classification label of the aspect term; and
adjusting parameters of models in the computer executable code based on the loss function.

### Claim 13 (depends on 11)

The method of claim 11, further comprising: adding a position dimension to the embedding of each of the words.

### Claim 14 (depends on 11)

The method of claim 11, wherein the number of the plurality of parsers is three, and the three parsers are CoreNLP, Stanza, and Berkeley neural parser.

### Claim 15 (depends on 11)

The method of claim 11, further comprising: adding a child-to-parent relation to each parent-to-child relation to the merged graph, and adding a self-loop to each node in the merged graph, so as to obtain the relation graph.

### Claim 16 (depends on 11)

The method of claim 11, wherein the relation neural network is a relational graph attention network (RGAT).

### Claim 17 (depends on 11)

The method of claim 11, wherein the classifications comprise a positive sentiment, a neutral sentiment, and negative sentiment.

### Claim 18

A non-transitory computer readable medium storing computer executable code, wherein the computer executable code, when executed at a processor of an active computing device, is configured to:
receive an aspect term-sentence pair, the aspect term-sentence pair comprising an aspect term and a sentence, the sentence comprising the aspect term, and the aspect term having a classification label;
embed the aspect term-sentence pair to obtain embedding of each word in the sentence;
parse the sentence using a plurality of parsers to obtain a plurality of dependency trees;
perform edge union on the plurality of dependency trees to obtain a merged graph, each node and each edge in the plurality of dependency trees included in the merged graph;
represent each node in the merged graph by corresponding one of the embeddings of the words to obtain a relation graph;
perform a relation neural network on the relation graph to obtain updated relation neural network;
extract hidden representation of the aspect term from the updated relation neural network to obtain extracted representation of the aspect term;
classify the aspect term based on the extracted representation to obtain a predicted classification label of the aspect term;
calculate a loss function based on the predicted classification label of the aspect term and the classification label of the aspect term; and
adjust parameters of models in the computer executable code based on the loss function.

### Claim 19 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein the computer executable code is further configured to, when the aspect term-sentence pair is a training aspect term-sentence pair and the aspect term has a classification label in regard to the sentence:
receive the classification label of the aspect term;
calculate a loss function based on the predicted classification label of the aspect term and the classification label of the aspect term; and
adjust parameters of models in the computer executable code based on the loss function.

### Claim 20 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein the computer executable code is further configured to add a position dimension to the embedding of each of the words, add a child-to-parent relation to each parent-to-child relation to the merged graph, and add a self-loop to each node in the merged graph, so as to obtain the relation graph.
