Back to results
US 12,614,113 B2Granted
Using consistency metadata for filtering of machine learning data across jobs
AssigneeAmazon Technologies, Inc.
InventorsLeo Parker Dirac, Jin Li, Tianming Zheng, Donghui Zhuo
Life of this patent
Estimating the term…
Abstract
Consistency metadata, including a parameter for a pseudo-random number source, are determined for training-and-evaluation iterations of a machine learning model. Using the metadata, a first training set comprising records of at least a first chunk is identified from a plurality of chunks of a data set. The first training set is used to train a machine learning model during a first training-and-evaluation iteration. A first test set comprising records of at least a second chunk is identified using the metadata, and is used to evaluate the model during the first training-and-evaluation iteration.
Claims20
A computer-implemented method, comprising:
- scheduling, at a first set of servers by a machine learning service of a cloud computing environment in response to one or more programmatic requests from a client, a training job for a particular machine learning model, wherein a parameter of the one or more programmatic requests comprises consistency metadata to be used to ensure that a test set used for the particular machine learning model does not overlap with a training set of the machine learning model, wherein the first set of servers has access to a first pseudo-random number source, and wherein one or more pseudo-random numbers obtained from the first pseudo-random number source are used at the first set of servers to select, as part of the training job, a training set for the particular machine learning model from a data set;
- scheduling, at a second set of servers by the machine learning service in response to the one or more programmatic requests, a test job for the particular machine learning model, wherein the second set of servers has access to a second pseudo-random number source;
- and
- selecting, at the second set of servers as part of the test job, a test set for the particular machine learning model from the data set, wherein the test set is selected using one or more pseudo-random numbers obtained from the second pseudo-random number source after synchronizing, using at least the consistency metadata, a state of the second pseudo-random number source with a state of the first pseudo-random number source.
1 more element
A system, comprising:
- one or more computing devices;
- wherein the one or more computing devices include instructions that upon execution on or across the one or more computing devices:
- schedule, at a first set of servers by a machine learning service of a cloud computing environment in response to one or more programmatic requests from a client, a training job for a particular machine learning model, wherein a parameter of the one or more programmatic requests comprises consistency metadata to be used to ensure that a test set used for the particular machine learning model does not overlap with a training set of the machine learning model, wherein the first set of servers has access to a first pseudo-random number source, and wherein one or more pseudo-random numbers obtained from the first pseudo-random number source are used at the first set of servers to select, as part of the training job, a training set for the particular machine learning model from a data set;
- schedule, at a second set of servers by the machine learning service in response to the one or more programmatic requests, a test job for the particular machine learning model, wherein the second set of servers has access to a second pseudo-random number source;
- and
- select, at the second set of servers as part of the test job, a test set for the particular machine learning model from the data set, wherein the test set is selected using one or more pseudo-random numbers obtained from the second pseudo-random number source after synchronizing, using at least the consistency metadata, a state of the second pseudo-random number source with a state of the first pseudo-random number source.
3 more elements
One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors:
- schedule, at a first set of servers by a machine learning service of a cloud computing environment in response to one or more programmatic requests from a client, a training job for a particular machine learning model, wherein a parameter of the one or more programmatic requests comprises consistency metadata to be used to ensure that a test set used for the particular machine learning model does not overlap with a training set of the machine learning model, wherein the first set of servers has access to a first pseudo-random number source, and wherein one or more pseudo-random numbers obtained from the first pseudo-random number source are used at the first set of servers to select, as part of the training job, a training set for the particular machine learning model from a data set;
- schedule, at a second set of servers by the machine learning service in response to the one or more programmatic requests, a test job for the particular machine learning model, wherein the second set of servers has access to a second pseudo-random number source;
- and
- select, at the second set of servers as part of the test job, a test set for the particular machine learning model from the data set, wherein the test set is selected using one or more pseudo-random numbers obtained from the second pseudo-random number source after synchronizing, using at least the consistency metadata, a state of the second pseudo-random number source with a state of the first pseudo-random number source.
1 more element
Dependencies are inferred from the claim text.