# US 2023/0267302 A1 · Large-Scale Architecture Search in Graph Neural Networks via Synthetic Data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267302 A1
- Document ID: US20230267302A1
- Status: Published application
- Kind: A1
- Application number: 17/940,568
- Filed: 2022-09-08
- Published: 2023-08-24
- Earliest claimed: 2022-02-18
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Bryan Thomas Perozzi; Anton Tsitsulin; John Joseph Palowitch; Brandon Mayer
- Web page: https://patentgrep.com/patent/US20230267302A1

## CPC

- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/105: {Shells for specifying net layout}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for graph model search and/or for architecture insight can include training and testing a plurality of graph models. For example, the systems and methods can generate a plurality of synthetic graph datasets, which can then be utilized to train a plurality of graph models with varying graph model architectures. The trained graph models can then be evaluated based on outputs generated by the models based on test inputs. The evaluation data can then be utilized for providing particular graph model insight and/or may be utilized to enable task-specific graph model search.

## Claims

### Claim 1

A computing system, the system comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
generating, by one or more generators, a plurality of synthetic graph datasets, wherein the plurality of synthetic graph datasets comprise structured-graph data;
training a plurality of graph models with at least a subset of the plurality of synthetic graph datasets to generate a plurality of trained graph models;
processing one or more inputs from the plurality of synthetic graph datasets with the plurality of trained graph models to generate a plurality of graph outputs;
determining a particular graph model of the plurality of graph models based on a comparison between the plurality of graph outputs; and
storing data associated with the particular graph model.

### Claim 2 (depends on 1)

The computing system of claim 1, wherein the operations further comprising:
generating an evaluation representation associated with the plurality of graph models based on the plurality of graph outputs; and
providing the evaluation representation for display.

### Claim 3 (depends on 1)

The computing system of claim 1, wherein each of the plurality of synthetic graph datasets comprise a realization of a parameterized probability distribution.

### Claim 4 (depends on 1)

The computing system of claim 1, wherein each of the plurality of synthetic graph datasets comprise one or more training graphs, one or more training features, and one or more training labels.

### Claim 5 (depends on 1)

The computing system of claim 1, wherein the one or more generators comprise one or more attributed-graph generators.

### Claim 6 (depends on 1)

The computing system of claim 1, wherein the one or more generators comprise one or more label generators.

### Claim 7 (depends on 1)

The computing system of claim 1, wherein each of the plurality of graph models comprise a graph neural network.

### Claim 8 (depends on 1)

The computing system of claim 1, wherein the subset of the plurality of synthetic graph datasets and the one or more inputs from the plurality of synthetic graph datasets differ.

### Claim 9 (depends on 1)

The computing system of claim 1, wherein the operations further comprise:
obtaining, from a user computing device, a user-input graph model;
training the user-input graph model with a first synthetic graph dataset of the plurality of synthetic graph datasets to generate a first trained graph model;
training the user-input graph model with a second synthetic graph dataset of the plurality of synthetic graph datasets to generate a second trained graph model;
processing a test portion of the plurality of synthetic graph datasets with the first trained graph model to generate a plurality of first user-model outputs;
processing the test portion of the plurality of synthetic graph dataset with the second trained graph model to generate a plurality of second user-model outputs; and
comparing the plurality of first user-model outputs and plurality of second user-model outputs.

### Claim 10 (depends on 9)

The computing system of claim 9, wherein the operations further comprise:
generating evaluation data based at least in part on the plurality of first user-model outputs and plurality of second user-model outputs; and
providing the evaluation data to the user computing device.

### Claim 11 (depends on 1)

The computing system of claim 1, wherein the operations further comprise:
generating comparison data based on the plurality of first user-model outputs, the plurality of second user-model outputs, and the plurality of graph outputs; and
providing the comparison data to the user computing device.

### Claim 12 (depends on 1)

The computing system of claim 1, wherein the operations further comprise:
obtaining input data associated with a specific task; and
wherein training the plurality of graph models with at least the subset of the plurality of synthetic graph datasets to generate the plurality of trained graph models comprises:
training the plurality of graph models to perform the specific task.

### Claim 13 (depends on 12)

The computing system of claim 12, wherein the plurality of synthetic graph datasets are generated based on the input data; and wherein the plurality of synthetic graph datasets comprise a plurality of labels associated with the specific task.

### Claim 14 (depends on 1)

The computing system of claim 1, wherein training the plurality of graph models with at least the subset of the plurality of synthetic graph datasets to generate the plurality of trained graph models comprises:
training a first graph model of the plurality of graph models with a first synthetic graph dataset of the plurality of synthetic graph datasets to generate a first trained graph model;
training a first graph model of the plurality of graph models with a second synthetic graph dataset of the plurality of synthetic graph datasets to generate a second trained graph model;
training a second graph model of the plurality of graph models with a first synthetic graph dataset of the plurality of synthetic graph datasets to generate a third trained graph model;
training a second graph model of the plurality of graph models with a second synthetic graph dataset of the plurality of synthetic graph datasets to generate a fourth trained graph model; and
wherein the plurality of trained graph models comprises the first trained graph model, the second trained graph model, the third trained graph model, and the fourth trained graph model.

### Claim 15

A computer-implemented method, the method comprising:
generating, by a computing system comprising one or more processors, a plurality of synthetic graph datasets, wherein the plurality of synthetic graph datasets comprise structured-graph data;
training, by the computing system, a plurality of graph models with at least a subset of the plurality of synthetic graph datasets to generate a plurality of trained graph models;
processing, by the computing system, one or more inputs from the plurality of synthetic graph datasets with the plurality of trained graph models to generate a plurality of graph outputs;
generating, by the computing system, an evaluation representation associated with the plurality of graph models based on the plurality of graph outputs; and
providing, by the computing system, the evaluation representation for display.

### Claim 16 (depends on 15)

The method of claim 15, wherein the evaluation representation comprises evaluation data descriptive of node classification for the plurality of graph models.

### Claim 17 (depends on 15)

The method of claim 15, wherein the evaluation representation comprises evaluation data descriptive of link prediction for the plurality of graph models.

### Claim 18

One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices, cause the one or more computing devices to perform operations, the operations comprising:
obtaining input data associated with a user;
generating, by one or more generators, a plurality of synthetic graph datasets based at least in part on the input data, wherein the plurality of synthetic graph datasets comprise structured-graph data;
training a plurality of graph models with at least a subset of the plurality of synthetic graph datasets to generate a plurality of trained graph models;
processing one or more inputs from the plurality of synthetic graph datasets with the plurality of trained graph models to generate a plurality of graph outputs;
generating an output representation associated with the plurality of graph models based on the plurality of graph outputs; and
providing the output representation for display.

### Claim 19 (depends on 18)

The one or more non-transitory computer-readable media of claim 18, wherein the output representation comprises a graphical depiction of a feature center distance based on the plurality of graph outputs associated with the plurality of graph models.

### Claim 20 (depends on 18)

The one or more non-transitory computer-readable media of claim 18, wherein the output representation comprises vector graph statistic data and hyperparameter evaluation data.
