Object count prediction using distributed processing

A system for predicting a count of generated objects by processing a plurality of stored objects distributed over two or more storage locations, the system comprising one or more processors configured for:

• receiving a communication that corresponds to a request for a predicted count of generated objects that are associated with one or more particular category identifiers (IDs) of a plurality of category IDs, the communication identifying the one or more particular category IDs;
• identifying a hash function;
• at each storage location of two or more storage locations:
• retrieving a plurality of stored objects, each stored object of the plurality of stored objects including:
• a profile ID that includes an identification of a user device that interacted with a webpage;
• a site ID that identifies the webpage;
• and
• at least one category ID of the plurality of category IDs;
• identifying, for each stored object of the plurality of stored objects, a hash ID for the stored object generated using the hash function;
• assigning, for each stored object of the plurality of stored objects, the stored object to a bucket of a plurality of buckets, the assignment being based at least in part on the hash ID;
• for each bucket of the plurality of buckets:
• assigning, for each stored object of the plurality of stored objects that is assigned to the bucket, the stored object to one or more sub-buckets of a set of sub-buckets corresponding to the bucket, each sub-bucket of the set of sub-buckets corresponding to a category ID of the plurality of category IDs;
• and
• generating, for each sub-bucket of the set of sub-buckets corresponding to the bucket, a k-minimum value for the sub-bucket, based at least in part on the hash IDs of the one or more stored objects of the plurality of stored objects assigned to the sub-bucket;
• identifying an inter-location group of sub-buckets, each sub-bucket of the inter-location group of sub-buckets corresponding to a particular category ID of the one or more particular category IDs and being associated with a storage location of two or more storage locations, wherein the inter-location group of sub-buckets includes at least two sub-buckets associated with different storage locations of the two or more storage locations;
• generating, based at least in part on combining each k-minimum value generated for the inter-location group of sub-buckets, the predicted count of the generated objects;
• and
• transmitting a response to the communication that is indicative of the predicted count of the generated objects.

A non-transitory computer readable medium encoded with instructions executable by a processor for predicting a count of generated objects by processing a plurality of stored objects distributed over two or more storage locations, comprising instructions for:

• receiving a communication that corresponds to a request for a predicted count of generated objects that are associated with one or more particular category identifiers (IDs) of a plurality of category IDs, the communication identifying the one or more particular category IDs;
• identifying a hash function;
• at each storage location of two or more storage locations:
• retrieving a plurality of stored objects, each stored object of the plurality of stored objects including:
• a profile ID that includes an identification of a user device that interacted with a webpage;
• a site ID that identifies the webpage;
• and
• at least one category ID of the plurality of category IDs;
• identifying, for each stored object of the plurality of stored objects, a hash ID for the stored object generated using the hash function;
• assigning, for each stored object of the plurality of stored objects, the stored object to a bucket of a plurality of buckets, the assignment being based at least in part on the hash ID;
• for each bucket of the plurality of buckets:
• assigning, for each stored object of the plurality of stored objects that is assigned to the bucket, the stored object to one or more sub-buckets of a set of sub-buckets corresponding to the bucket, each sub-bucket of the set of sub-buckets corresponding to a category ID of the plurality of category IDs;
• and
• generating, for each sub-bucket of the set of sub-buckets corresponding to the bucket, a k-minimum value for the sub-bucket based, at least in part, on the hash IDs of the one or more stored objects of the plurality of stored objects assigned to the sub-bucket;
• identifying an inter-location group of sub-buckets, each sub-bucket of the inter-location group of sub-buckets corresponding to a particular category ID of the one or more particular category IDs and being associated with a storage location of two or more storage locations, wherein the inter-location group of sub-buckets includes at least two sub-buckets associated with different storage locations of the two or more storage locations;
• generating, based at least in part on combining each k-minimum value generated for the inter-location group of sub-buckets, the predicted count of the generated objects;
• and
• transmitting a response to the communication that is indicative of the predicted count of the generated objects.