# US 2024/0256832 A1 · Graph machine learning model based techniques for evaluating knowledge graph datasets

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0256832 A1
- Document ID: US20240256832A1
- Status: Published application
- Kind: A1
- Application number: 18/178,047
- Filed: 2023-03-03
- Published: 2024-08-01
- Earliest claimed: 2023-01-13
- Assignees: Optum Services (Ireland) Limited
- Applicants: Optum Services (Ireland) Limited
- Inventors: Premnath Kandhasamy NARAYANAN; David S. MONAGHAN; Brian CARTER; Amirhossein YAZDAVAR; Triet PHAM
- Web page: https://patentgrep.com/patent/US20240256832A1

## CPC

- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- G06N 20/20: Ensemble learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Various embodiments of the present disclosure describe data evaluation techniques that leverage a graph-based machine learning model to evaluate a knowledge graph. The techniques include using a target graph model to generate a predictive representation for a graph node of a graph training dataset. The techniques include using a feature prediction model to generate predicted feature values for the graph node based on the predictive representation. The techniques include generating a data evaluation score for the graph training dataset based on the predicted feature values. The techniques include using the target graph model to generate a predictive output for the graph node based on the predictive representation and then generating an evaluation output for the target graph model based on the evaluation score and the predictive output.

## Claims

### Claim 1

A computer-implemented method comprising:
generating, by one or more processors and using a target graph model, a predictive representation for a graph node of a graph training dataset;
generating, by the one or more processors and using a feature prediction model, one or more predicted feature values for the graph node based on the predictive representation;
generating, by the one or more processors, a data evaluation score for the graph training dataset based on the one or more predicted feature values;
generating, by the one or more processors and using the target graph model, a predictive output for the graph node based on the predictive representation, wherein the predictive output comprises a node classification for the graph node, wherein the one or more predicted feature values correspond to one or more evaluation features of the graph training dataset, and wherein the target graph model is previously trained to generate an evaluation feature-agnostic predictive representation that at least partially prevents the target graph model from generating the predictive output based on the one or more evaluation features; and
generating, by the one or more processors, an evaluation output for the target graph model based on the data evaluation score and the predictive output.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the target graph model comprises a first graph neural network (GNN) and the feature prediction model comprises a second GNN.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the target graph model and the feature prediction model are jointly trained using a joint objective function.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein:
the joint objective function comprises a first objective function and a second objective function,
the first objective function comprises a first optimization function for the target graph model, and
the second objective function comprises a second optimization function for the feature prediction model.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein the first objective function comprises a comparison between a predictive output of the target graph model and a ground truth label for the graph node.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the predictive representation comprises a feature embedding that encodes one or more features of the graph node and one or more adjacent features of one or more neighboring nodes of the graph node in the graph training dataset.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein the one or more neighboring nodes of the graph node comprise one or more nodes of the graph training dataset that are connected to the graph node by one or more edges.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the data evaluation score is based on a graph feature confidence score indicative of a predicted accuracy of the one or more predicted feature values.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the graph feature confidence score is based on a predicted feature confidence score indicative of a confidence level for the one or more predicted feature values.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein the predicted feature confidence score is generated by the feature prediction model.

### Claim 11

A computing apparatus comprising memory and one or more processors communicatively coupled to the memory, the one or more processors configured to:
generate, using a target graph model, a predictive representation for a graph node of a graph training dataset;
generate, using a feature prediction model, one or more predicted feature values for the graph node based on the predictive representation;
generate data evaluation score for the graph training dataset based on the one or more predicted feature values;
generate, using the target graph model, a predictive output for the graph node based on the predictive representation, wherein the predictive output comprises a node classification for the graph node, wherein the one or more predicted feature values correspond to one or more evaluation features of the graph training dataset, and wherein the target graph model is previously trained to generate an evaluation feature-agnostic predictive representation that at least partially prevents the target graph model from generating the predictive output based on the one or more evaluation features; and
generate an evaluation output for the target graph model based on the data evaluation score and the predictive output.

### Claim 12 (depends on 11)

The computing apparatus of claim 11, wherein the target graph model comprises a first graph neural network (GNN) and the feature prediction model comprises a second GNN.

### Claim 13 (depends on 11)

The computing apparatus of claim 11, wherein the target graph model and the feature prediction model are jointly trained using a joint objective function.

### Claim 14 (depends on 13)

The computing apparatus of claim 13, wherein:
the joint objective function comprises a first objective function and a second objective function,
the first objective function comprises a first optimization function for the target graph model, and
the second objective function comprises a second optimization function for the feature prediction model.

### Claim 15 (depends on 14)

The computing apparatus of claim 14, wherein the first objective function comprises a comparison between the predictive output generated by the target graph model and a ground truth label for the graph node.

### Claim 16

One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to:
generate, using a target graph model, a predictive representation for a graph node of a graph training dataset;
generate, using a feature prediction model, one or more predicted feature values for the graph node based on the predictive representation;
generate data evaluation score for the graph training dataset based on the one or more predicted feature values;
generate, using the target graph model, a predictive output for the graph node based on the predictive representation, wherein the predictive output comprises a node classification for the graph node, wherein the one or more predicted feature values correspond to one or more evaluation features of the graph training dataset, and wherein the target graph model is previously trained to generate an evaluation feature-agnostic predictive representation that at least partially prevents the target graph model from generating the predictive output based on the one or more evaluation features; and
generate an evaluation output for the target graph model based on the data evaluation score and the predictive output.

### Claim 17 (depends on 16)

The one or more non-transitory computer-readable storage media of claim 16, wherein the predictive representation comprises a feature embedding that encodes one or more features of the graph node and one or more adjacent features of one or more neighboring nodes of the graph node in the graph training dataset.

### Claim 18 (depends on 16)

The one or more non-transitory computer-readable storage media of claim 16, wherein the target graph model comprises a first graph neural network (GNN) and the feature prediction model comprises a second GNN.

### Claim 19 (depends on 16)

The one or more non-transitory computer-readable storage media of claim 16, wherein the target graph model and the feature prediction model are jointly trained using a joint objective function.

### Claim 20 (depends on 19)

The one or more non-transitory computer-readable storage media of claim 19, wherein:
the joint objective function comprises a first objective function and a second objective function,
the first objective function comprises a first optimization function for the target graph model, and
the second objective function comprises a second optimization function for the feature prediction model.
