Training graph neural networks using a de-noising objective

One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training a neural network that includes one or more graph neural network layers, the operations comprising:

• generating data defining a graph that comprises:
• (i) a set of nodes, (ii) a node embedding for each node, and (iii) a set of edges that each connect a respective pair of nodes, comprising:
• obtaining a respective initial feature representation for each node;
• generating a respective final feature representation for each node, wherein, for each of one or more of the nodes, the respective final feature representation is a modified feature representation that is generated from the respective feature representation for the node using respective noise;
• and
• generating the data defining the graph using the respective final feature representations of the nodes;
• processing the data defining the graph using one or more of the graph neural network layers of the neural network to generate a respective updated node embedding of each node;
• processing, for each of one or more of the nodes having modified feature representations, the updated node embedding of the node to generate a respective de-noising prediction for the node that characterizes a de-noised feature representation for the node that does not include the noise used to generate the modified feature representation of the node;
• and
• determining an update to current values of neural network parameters of the neural network to optimize an objective function that measures errors in the de-noising predictions for the nodes.

A system comprising:

• one or more computers;
• and
• one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations for training a neural network that includes one or more graph neural network layers, the operations comprising:
• generating data defining a graph that comprises:
• (i) a set of nodes, (ii) a node embedding for each node, and (iii) a set of edges that each connect a respective pair of nodes, comprising:
• obtaining a respective initial feature representation for each node;
• generating a respective final feature representation for each node, wherein, for each of one or more of the nodes, the respective final feature representation is a modified feature representation that is generated from the respective feature representation for the node using respective noise;
• and
• generating the data defining the graph using the respective final feature representations of the nodes;
• processing the data defining the graph using one or more of the graph neural network layers of the neural network to generate a respective updated node embedding of each node;
• processing, for each of one or more of the nodes having modified feature representations. the updated node embedding of the node to generate a respective de-noising prediction for the node that characterizes a de-noised feature representation for the node that does not include the noise used to generate the modified feature representation of the node;
• and
• determining an update to current values of neural network parameters of the neural network to optimize an objective function that measures errors in the de-noising predictions for the nodes.