Optimizing large scale data analysis

A system, comprising:

• one or more memory devices storing instructions;
• and
• one or more data processing apparatus that are configured to interact with the one or more memory devices, and upon execution of the instructions, perform operations including:
• obtaining a dataset comprising data for a plurality of objects, wherein each object in the dataset performs activities in a digital environment and the data for the plurality of objects represents the activities;
• for each data item in the dataset:
• generating, using an identifier for an object specified in the data item, a hashed parameter for the object, wherein the hashed parameter has a binary representation;
• identifying a register from among a set of registers based on a first portion of the binary representation of the hashed parameter, wherein each register in the set of registers comprises a data structure that stores data about objects in the dataset, wherein the data structure for each register in the set of registers has a first field, a second field, and a third field, and wherein the third field stores an aggregation amount representing a number of occurrences of a corresponding object in the dataset;
• determining, based on a comparison of a second portion of the binary representation of the hashed parameter with data in the first field and the second field of the data structure of the register, that the hashed parameter for the object contributes to the aggregation amount;
• and
• in response to determining that the hashed parameter for the object contributes to the aggregation amount, updating the aggregation amount stored in the third field of the register;
• and
• generating, based on aggregate amounts stored in the set of registers, a reporting output that indicates a set of data items, wherein each data item identifies an estimated number of objects in the dataset that performed activities in the digital environment at a particular aggregation amount.

A non-transitory computer readable medium storing instructions that, when executed by one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising:

• obtaining a dataset comprising data for a plurality of objects, wherein each object in the dataset performs activities in a digital environment and the data for the plurality of objects represents the activities;
• for each data item in the dataset:
• generating, using an identifier for an object specified in the data item, a hashed parameter for the object, wherein the hashed parameter has a binary representation;
• identifying a register from among a set of registers based on a first portion of the binary representation of the hashed parameter, wherein each register in the set of registers comprises a data structure that stores data about objects in the dataset, wherein the data structure for each register in the set of registers has a first field, a second field, and a third field, and wherein the third field stores an aggregation amount representing a number of occurrences of a corresponding object in the dataset;
• determining, based on a comparison of a second portion of the binary representation of the hashed parameter with data in the first field and the second field of the data structure of the register, that the hashed parameter for the object contributes to the aggregation amount;
• and
• in response to determining that the hashed parameter for the object contributes to the aggregation amount, updating the aggregation amount stored in the third field of the register;
• and
• generating, based on aggregate amounts stored in the set of registers, a reporting output that indicates a set of data items, wherein each data item identifies an estimated number of objects in the dataset that performed activities in the digital environment at a particular aggregation amount.