Dynamically optimizing a data set distribution
AssigneeGroupon, Inc.
InventorsDavid Alan Johnston, Jonathan Esterhazy, Gaston L'Huillier, Hernan Enrique Arroyo Garcia
Life of this patent
Estimating the term…
Abstract
In general, embodiments of the present invention provide systems, methods and computer readable media configured to receive configuration data describing a desired data set distribution, and, in response to receiving new data instances, use the configuration data and the new data instances to dynamically optimize the distribution of data already stored in a data reservoir that has been discretized into bins representing the desired data distribution.
Claims36
A computer-implemented method for optimizing data sampling to generate a data set distribution of a set of input data instances, the method comprising:
- receiving a data set optimization job, the data set optimization job comprising the set of input data instances having an input range and distribution configuration data describing a first discretization of the data set distribution into a plurality of data bins, wherein the data set optimization job is associated with an input data evaluator, and wherein the first discretization defines, for each data bin of the plurality of data bins, a range of evaluation values, and wherein the first discretization further defines, for each data bin of the plurality of data bins, an associated label, a prediction range, and a size capacity for the data bin, and further wherein the associated label for each data bin identifies any of the set of input data instances that are associated with the data bin;
- and
- determining whether to update the data set distribution by performing operations comprising, for each input data instance of the set of input data instances:
- generating an instance evaluation of the input data instance;
- for each data bin of the plurality of data bins, generating an evaluation determination, wherein the evaluation determination for each data bin indicates whether to offer the input data instance to the data bin;
- for each offered data bin of the plurality of data bins whose evaluation determination indicates offering of the input data instance to the data bin, generating an update determination for the offered data bin based on the range of evaluation values associated with the offered data bin and the instance evaluation of the input data instance, wherein the update determination for each offered data bin indicates whether to associate the input data instance with the offered data bin;
- and
- for each associated data bin being an offered data bin of the plurality of data bins whose update determination indicates associating the input data instance to the associated data bin, updating the associated label of the associated data bin.
A computer program product, stored on a non-transitory computer readable medium, comprising instructions that when executed on one or more computers cause the one or more computers to perform operations for optimizing data sampling to generate a data set distribution of a set of input data instances, the operations comprising:
- receiving a data set optimization job, the data set optimization job comprising the set of input data instances having an input range and distribution configuration data describing a first discretization of the data set distribution, into a plurality of data bins, wherein the data set optimization job is associated with an input data evaluator, and wherein the first discretization defines, for each data bin of the plurality of data bins, a range of evaluation values, and wherein the first discretization further defines, for each data bin of the plurality of data bins, one or more evaluation criteria and an associated label, a prediction range, and a size capacity for the data bin, and further wherein the associated label for each data bin identifies any of the set of input data instances that are associated with the data bin;
- and determining whether to update the data set distribution by performing operations comprising for each input data instance of the set of input data instances:
- generating an instance evaluation of the input data instance;
- for each data bin of the plurality of data bins, generating an evaluation determination, wherein the evaluation determination for each data bin indicates whether to offer the input data instance to the data bin;
- for each offered data bin of the plurality of data bins whose evaluation determination indicates offering of the input data instance to the data bin, generating an update determination for the offered data bin based on the range of evaluation values associated with the offered data bin and the instance evaluation of the input data instance, wherein the update determination for each offered data bin indicates whether to associate the input data instance with the offered data bin;
- and
- for each associated data bin being an offered data bin of the plurality of data bins whose update determination indicates associating the input data instance to the associated data bin, updating the associated label of the associated data bin in an instance.
A system comprising:
- one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations for optimizing data sampling to generate a data set distribution of a set of input data instances, the operations comprising:
- receiving a data set optimization job, the data set optimization job comprising the set of input data instances having an input range and distribution configuration data describing a first discretization of into a plurality of data bins, wherein the data set optimization job is associated with an input data evaluator, and wherein the first discretization defines, for each data bin of the plurality of data bins, a range of evaluation values, and wherein the first discretization further defines, for each data bin of the plurality of data bins, an associated label, a prediction range, and a size capacity for the data bin, and further wherein the associated label for each data bin identifies any of the set of input data instances that are associated with the data bin;
- and
- determining whether to update the data set distribution by performing operations comprising, for each of the input data instances
- generating an instance evaluation of the input data instance;
- for each data bin of the plurality of data bins, generating an evaluation determination, wherein the evaluation determination for each data bin indicates whether to offer the input data instance to the data bin;
- for each offered data bin of the plurality of data bins whose evaluation determination indicates offering of the input data instance to the data bin, generating an update determination for the offered data bin based on the range of evaluation values associated with the offered data bin and the instance evaluation of the input data instance, wherein the update determination for each offered data bin indicates whether to associate the input data instance with the data bin;
- and
- for each associated data bin being an offered data bin of the plurality of data bins whose update determination indicates associating the input data instance to the associated data bin, updating the associated label of the associated data bin.
Dependencies are inferred from the claim text.