Meta-estimation of bloom filters with vectors of counts

A system for improving data sketch processing in data processing systems, comprising:

• one or more processors;
• and
• one or more memory devices storing computer-readable instructions that, when implemented, cause the one or more processors to perform operations, the operations comprising:
• generating a first initial sketch comprising a plurality of first initial sketch registers, wherein a subset of the plurality of first initial sketch registers are populated according to one or more generation parameters and based at least in part on hashes of a plurality of data records;
• and
• generating a first meta-estimation sketch comprising a plurality of first meta-estimation sketch registers populated based at least in part on hashes of index values of each of the subset of the plurality of first initial sketch registers.

A method for improving data sketch processing in data processing systems, comprising:

• maintaining, by a data processing system comprising one or more processors and a memory, a plurality of data records comprising a plurality of identifiers and one or more attributes;
• generating, by the data processing system, a first data structure that represents the plurality of data records, the first data structure having a plurality of first registers, wherein at least one of the plurality of first registers is populated based on hashes of each of the plurality of identifiers;
• identifying, by the data processing system, a subset of the plurality of first registers that are equal to a predetermined value, each of the subset of the plurality of first registers identified by an index value;
• generating, by the data processing system, a second data structure that represents the subset of the plurality of first registers, the second data structure having a plurality of second registers, wherein at least one of the plurality of second registers is populated based on hashes of the index values of each of the subset of the plurality of first registers;
• and
• storing, by the data processing system, in the memory, the second data structure that represents the subset of the plurality of first registers.