Management and Evaluation of Machine-Learned Models Based on Locally Logged Data

A computer-implemented method to manage machine-learned models, the method comprising:

• obtaining, by a user computing device, a plurality of machine-learned models;
• evaluating, by the user computing device, at least one performance metric for each of the plurality of machine-learned models, wherein the at least one performance metric for each machine-learned model is evaluated relative to data that is stored locally at the user computing device;
• providing, by the user computing device, the performance metrics respectively evaluated for the plurality of machine-learned models to a server computing device;
• receiving, by the user computing device, a selection of a first machine-learned model of the plurality of machine-learned models from the server computing device based at least in part on the performance metrics respectively evaluated for the plurality of machine-learned models;
• and
• using, by the user computing device, the selected first machine-learned model to obtain one or more predictions.

A computing device, the computing device comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing device to:
• obtain a machine-learned model, wherein the machine-learned model is one of a plurality of machine-learned models that have been previously trained on different types of training data;
• train the machine-learned model further based on a first portion of data that is stored locally at the computing device to obtain a user-specific variant of the machine-learned model;
• evaluate at least one performance metric for the user-specific variant of the machine-learned model, wherein the at least one performance metric for the user-specific variant of the machine-learned model is evaluated relative to a second portion of the data that is stored locally at the computing device;
• determine whether to activate the user-specific variant of the machine-learned model based at least in part on the at least one performance metric evaluated for the user-specific variant of the machine-learned model;
• and
• when it is determined that the user-specific variant of the machine-learned model should be activated, use the user-specific variant of the machine-learned model to obtain one or more predictions.

One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors of a server computing system, cause the server computing system to:

• obtain a plurality of performance metrics respectively associated with a plurality of machine-learned models, wherein the performance metric for each machine-learned model indicates a performance of such machine-learned model when evaluated against a set of data that is stored locally at a user computing device;
• select at least one of the plurality of machine-learned models based at least in part on the plurality of performance metrics;
• and
• cause use of the selected at least one machine-learned model at the user computing device.