Generating real-time aggregates at scale for inclusion in one or more modified fields in a produced subset of data

A method implemented by a data processing system for producing a subset of data from a plurality of data sources, modifying one or more attributes of one or more respective fields of the subset and displaying an editor interface that enables segmentation of data records by displaying one or more representations of the one or more modified fields, the method including:

• selecting a plurality of data sources to be represented in an editor interface;
• generating a subset of data included in the plurality of data sources, by:
• for each of the data sources,
• selecting one or more data structures from that data source, with each data structure including one or more fields;
• for at least one selected data structure,
• modifying one or more attributes of one or more respective fields in that data structure;
• storing in memory the selected data structures included in the subset, with at least one of the stored data structures including the one or more modified attributes of the one or more respective fields;
• displaying, in the editor interface, representations of the stored data structures, with at least one of the representations being of the one or more modified attributes of the one or more respective fields, with each representation including one or more selectable portions, with a selectable portion representing a field of a data structure;
• receiving, through the editor interface, selection data specifying selection of one or more selectable portions;
• and
• segmenting a plurality of received data records by identifying which of the received data records have one or more fields that correspond to one or more fields represented in the one or more selectable portions selected.

A method performed by a data processing system for generating near real-time aggregates, the method including:

• intermittently receiving data records from one or more data sources;
• for a given data record received,
• identifying at least a first field and a second field in the given data record;
• detecting a first value in the first field and a second value in the second field;
• and
• generating a compound key in accordance with the first value of the first field and the second value of the second field;
• accessing, from memory, aggregation data related to at least the first field or the second field;
• generating a compound key value by generating a data record with a field storing the compound key and one or more fields each storing an item of the aggregation data, wherein the compound key value represents a near real-time aggregation of data related to at least the first field or the second field;
• and
• recording an occurrence of the given data record by storing in memory the compound key value.