Wide and deep machine learning models

A method of training a combined machine learning model, the combined machine learning model being configured to process a machine learning input to generate an output for the machine learning input, the combine machine learning model including a deep machine learning model, a wide machine learning model, and a combining layer that is configured to process outputs of the deep machine learning model and the wide machine learning model to generate the predicted output for the machine learning input, and the method comprising:

• obtaining training data comprising, for each of a plurality of training inputs, (i) features of the training input that comprise features of a content recommendation setting, and (ii) a known output for the training input;
• and
• for each of the training inputs:
• processing a first subset of the features of the training input using the deep machine learning model to generate a deep model intermediate predicted output for the training input in accordance with current values of parameters of the deep machine learning model;
• processing a second subset of the features of the training input using the wide machine learning model to generate a wide model intermediate predicted output for the training input in accordance with current values of parameters of the wide machine learning model, wherein processing the second subset of the features of the training input comprises:
• applying cross-product transformation to at least some of the features of the training input in a second subset to generate transformed features, and
• processing, using a generalized linear model of the wide machine learning model, the transformed features and the features in the second subset to generate a wide model intermediate predicted output;
• processing the deep model intermediate predicted output and the wide model intermediate predicted output for the training input using the combining layer to generate a predicted output for the training input, wherein the predicted output comprises a respective score for each content item in one or more content items and wherein each respective score represents a respective likelihood that a particular objective will be satisfied if the corresponding content item is presented in the content recommendation setting;
• backpropagating a gradient determined from an error between the predicted output for the training input and the known output for the training input through the combining layer to the wide machine learning model and the deep machine learning model to jointly adjust the current values of the parameters of the deep machine learning model and the wide machine learning model.

A non-transitory computer-readable medium having instructions stored thereon which, when executed by at least one computer, cause the at least one computer to perform operations to train a combined machine learning model, the combined machine learning model being configured to process a machine learning input to generate an output for the machine learning input, the combine machine learning model including a deep machine learning model, a wide machine learning model, and a combining layer that is configured to process outputs of the deep machine learning model and the wide machine learning model to generate the output for the machine learning input, and the operations comprising:

• obtaining training data comprising, for each of a plurality of training inputs, (i) features of the training input that comprise features of a content recommendation setting, and (ii) a known output for the training input;
• and
• for each of the training inputs:
• processing a first subset of the features of the training input using the deep machine learning model to generate a deep model intermediate predicted output for the training input in accordance with current values of parameters of the deep machine learning model;
• processing a second subset of the features of the training input using the wide machine learning model to generate a wide model intermediate predicted output for the training input in accordance with current values of parameters of the wide machine learning model, wherein processing the second subset of the features of the training input comprises:
• applying cross-product transformation to at least some of the features of the training input in a second subset to generate transformed features, and
• processing, using a generalized linear model of the wide machine learning model, the transformed features and the features in the second subset to generate a wide model intermediate predicted output;
• processing the deep model intermediate predicted output and the wide model intermediate predicted output for the training input using the combining layer to generate a predicted output for the training input, wherein the predicted output comprises a respective score for each content item in one or more content items and wherein each respective score represents a respective likelihood that a particular objective will be satisfied if the corresponding content item is presented in the content recommendation setting;
• backpropagating a gradient determined from an error between the predicted output for the training input and the known output for the training input through the combining layer to the wide machine learning model and the deep machine learning model to jointly adjust the current values of the parameters of the deep machine learning model and the wide machine learning model.