Machine learning algorithm search with symbolic programming

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for searching for an output machine learning algorithm to perform a particular machine learning task given an input machine learning algorithm, the operations comprising:

• receiving data specifying an input machine learning algorithm that performs the particular machine learning task;
• receiving data specifying a search algorithm that searches for candidate machine learning algorithms and an evaluation function that evaluates the performance of candidate machine learning algorithms;
• generating data representing a symbolic tree from the input machine learning algorithm, wherein the symbolic tree is a tree of a plurality of nodes, wherein each node represents a component in the input machine learning algorithm and is associated with a concrete value;
• generating data representing a hyper symbolic tree from the symbolic tree by changing each node in a subset of the plurality of the nodes in the symbolic tree to a respective new node having a respective placeholder value with a corresponding set of possible concrete values;
• searching an algorithm search space that defines a set of possible concrete symbolic trees from the hyper symbolic tree, wherein each node in a possible concrete symbolic tree corresponds to a respective node in the hyper symbolic tree and takes a particular concrete value from a corresponding set of possible concrete values associated with the respective node in the hyper symbolic tree, wherein each possible concrete symbolic tree corresponds to a candidate machine learning algorithm, and wherein searching the algorithm search space comprises performing the following operations at least once:
• generating data representing a candidate concrete symbolic tree by using the search algorithm to search for the candidate concrete symbolic tree in the algorithm search space;
• generating a candidate machine learning algorithm from the candidate concrete symbolic tree,
• training the candidate machine learning algorithm on the particular machine learning task,
• determining, using the evaluation function, a performance metric specifying a performance of the trained candidate machine learning algorithm on the particular machine learning task;
• and
• selecting one or more trained candidate machine learning algorithms among the trained candidate machine learning algorithms based on the determined performance metrics.

One or more computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for searching for an output machine learning algorithm to perform a particular machine learning task given an input machine learning algorithm, the operations comprising:

• receiving data specifying an input machine learning algorithm that performs the particular machine learning task;
• receiving data specifying a search algorithm that searches for candidate machine learning algorithms and an evaluation function that evaluates the performance of candidate machine learning algorithms;
• generating data representing a symbolic tree from the input machine learning algorithm, wherein the symbolic tree is a tree of a plurality of nodes, wherein each node represents a component in the input machine learning algorithm and is associated with a concrete value;
• generating data representing a hyper symbolic tree from the symbolic tree by changing each node in a subset of the plurality of the nodes in the symbolic tree to a respective new node having a respective placeholder value with a corresponding set of possible concrete values;
• searching an algorithm search space that defines a set of possible concrete symbolic trees from the hyper symbolic tree, wherein each node in a possible concrete symbolic tree corresponds to a respective node in the hyper symbolic tree and takes a particular concrete value from a corresponding set of possible concrete values associated with the respective node in the hyper symbolic tree, wherein each possible concrete symbolic tree corresponds to a candidate machine learning algorithm, and wherein searching the algorithm search space comprises performing the following operations at least once:
• generating data representing a candidate concrete symbolic tree by using the search algorithm to search for the candidate concrete symbolic tree in the algorithm search space;
• generating a candidate machine learning algorithm from the candidate concrete symbolic tree,
• training the candidate machine learning algorithm on the particular machine learning task,
• determining, using the evaluation function, a performance metric specifying a performance of the trained candidate machine learning algorithm on the particular machine learning task;
• and
• selecting one or more trained candidate machine learning algorithms among the trained candidate machine learning algorithms based on the determined performance metrics.

A method comprising:

• providing data specifying an input machine learning algorithm to an algorithm search system;
• causing the algorithm search system to search for an output machine learning algorithm to perform a particular machine learning task using operations comprising:
• receiving the data specifying an input machine learning algorithm that performs the particular machine learning task;
• receiving data specifying a search algorithm that searches for candidate machine learning algorithms and an evaluation function that evaluates the performance of candidate machine learning algorithms;
• generating data representing a symbolic tree from the input machine learning algorithm, wherein the symbolic tree is a tree of a plurality of nodes, wherein each node represents a component in the input machine learning algorithm and is associated with a concrete value;
• generating data representing a hyper symbolic tree from the symbolic tree by changing each node in a subset of the plurality of the nodes in the symbolic tree to a respective new node having a respective placeholder value with a corresponding set of possible concrete values;
• searching an algorithm search space that defines a set of possible concrete symbolic trees from the hyper symbolic tree, wherein each node in a possible concrete symbolic tree corresponds to a respective node in the hyper symbolic tree and takes a particular concrete value from a corresponding set of possible concrete values associated with the respective node in the hyper symbolic tree, wherein each possible concrete symbolic tree corresponds to a candidate machine learning algorithm, and wherein searching the algorithm search space comprises performing the following operations at least once:
• generating data representing a candidate concrete symbolic tree by using the search algorithm to search for the candidate concrete symbolic tree in the algorithm search space;
• generating a candidate machine learning algorithm from the candidate concrete symbolic tree,
• training the candidate machine learning algorithm on the particular machine learning task, and
• determining, using the evaluation function, a performance metric specifying a performance of the trained candidate machine learning algorithm on the particular machine learning task;
• and
• selecting one or more trained candidate machine learning algorithms among the trained candidate machine learning algorithms based on the determined performance metrics;
• and
• using the output machine learning algorithm to perform the particular machine learning task.