Augmenting neural networks

A system comprising:

• one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
• maintaining, by a computational graph system that manages execution of computational graphs representing neural network operations for users of the computational graph system, data specifying a plurality of pre-trained neural networks, wherein each of the pre-trained neural networks is a neural network that has been trained on training data to determine trained values of respective parameters of the neural network;
• obtaining data specifying a user computational graph representing neural network operations, the user computational graph comprising a plurality of nodes connected by edges;
• modifying the user computational graph by modifying a subgraph of the user computational graph using a particular pre-trained neural network from the plurality of pre-trained neural networks, comprising:
• identifying the subgraph of the user computational graph, wherein the subgraph of the user computational graph receives a subgraph input, and generates a subgraph output after processing the subgraph input, wherein the subgraph output is provided as an input to a succeeding node in the user computational graph;
• selecting the particular pre-trained neural network from the plurality of pre-trained neural networks, wherein the particular pre-trained neural network receives an input conforming to the subgraph input, and generates an output conforming to the subgraph output;
• and
• replacing the subgraph with the particular pre-trained neural network by inserting a remote call node into the user computational graph to modify the user computational graph, wherein the remote call node is configured to, during execution of the modified user computational graph:
• receive data representing the subgraph input,
• provide, using a remote call, the subgraph input as an input to the particular pre-trained neural network,
• obtain, in response to the remote call, a remote output generated by the particular pre-trained neural network for the subgraph input, and
• provide, as the subgraph output, the remote output generated by the particular pre-trained neural network to the succeeding node;
• and
• storing the modified user computational graph in a memory device.

One or more non-transitory computer storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

• maintaining, by a computational graph system that manages execution of computational graphs representing neural network operations for users of the computational graph system, data specifying a plurality of pre-trained neural networks, wherein each of the pre-trained neural networks is a neural network that has been trained on training data to determine trained values of respective parameters of the neural network;
• obtaining data specifying a user computational graph representing neural network operations, the user computational graph comprising a plurality of nodes connected by edges;
• modifying the user computational graph by modifying a subgraph of the user computational graph using a particular pre-trained neural network from the plurality of pre-trained neural networks, comprising:
• identifying the subgraph of the user computational graph, wherein the subgraph of the user computational graph receives a subgraph input, and generates a subgraph output after processing the subgraph input, wherein the subgraph output is provided as an input to a succeeding node in the user computational graph;
• selecting the particular pre-trained neural network from the plurality of pre-trained neural networks, wherein the particular pre-trained neural network receives an input conforming to the subgraph input, and generates an output conforming to the subgraph output;
• and
• replacing the subgraph with the particular pre-trained neural network by inserting a remote call node into the user computational graph to modify the user computational graph, wherein the remote call node is configured to, during execution of the modified user computational graph:
• receive data representing the subgraph input,
• provide, using a remote call, the subgraph input as an input to the particular pre-trained neural network,
• obtain, in response to the remote call, a remote output generated by the particular pre-trained neural network for the subgraph input, and
• provide, as the subgraph output, the remote output generated by the particular pre-trained neural network to the succeeding node;
• and
• storing the modified user computational graph in a memory device.