Dynamic distributed generation of data representations from highly condensed data

A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause a processor to perform operations comprising:

• store, at a node device of a grid of node devices, a data block of a plurality of data blocks of a data blob, wherein:
• the plurality of data blocks of the data blob are distributed among the grid of node devices to be stored by the grid of node devices;
• the data blob comprises multiple data items;
• and
• each data block of the plurality of data blocks comprises a subset of the multiple data items;
• receive, at the node device, data blob metadata that comprises an indication of organization of the multiple data items within the data blob;
• receive, at the node device, a command that originates from a requesting device to generate at least one row of a representation portion of a plurality of representation portions from the data block, wherein:
• a representation data that provides a two-dimensional (2D) representation of data items of the data blob is to be assembled from the plurality of representation portions;
• the plurality of representation portions is to be generated from a subset of data blocks of the plurality of data blocks by a subset of node devices of the grid of node devices that store the subset of data blocks;
• and
• the subset of node devices comprises the node device;
• use the data blob metadata and a node block map of a plurality of node block maps to derive one or more transforms to be performed at the node device with the data block to generate the at least one row of the representation portion, wherein:
• the plurality of node block maps is provided to the subset of node devices to generate the plurality of representation portions from the subset of data blocks;
• and
• the node block map comprises an indication of which data items of the data block are required to generate the representation portion, what data values are to be generated from the required data items, and an organization of the generated data values within the representation portion;
• and
• perform, at the node device, the one or more transforms with the data block to generate the at least one row of the representation portion.

A computer-implemented method comprising:

• storing, at a node device of a grid of node devices, a data block of a plurality of data blocks of a data blob, wherein:
• the plurality of data blocks of the data blob are distributed among the grid of node devices to be stored by the grid of node devices;
• the data blob comprises multiple data items;
• and
• each data block of the plurality of data blocks comprises a subset of the multiple data items;
• receiving, at the node device, data blob metadata that comprises an indication of organization of the multiple data items within the data blob;
• receiving, at the node device, a command that originates from a requesting device to generate at least one row of a representation portion of a plurality of representation portions from the data block, wherein:
• a representation data that provides a two-dimensional (2D) representation of data items of the data blob is to be assembled from the plurality of representation portions;
• the plurality of representation portions is to be generated from a subset of data blocks of the plurality of data blocks by a subset of node devices of the grid of node devices that store the subset of data blocks;
• and
• the subset of node devices comprises the node device;
• using the data blob metadata and a node block map of a plurality of node block maps to derive one or more transforms to be performed at the node device with the data block to generate the at least one row of the representation portion, wherein:
• the plurality of node block maps is provided to the subset of node devices to generate the plurality of representation portions from the subset of data blocks;
• and
• the node block map comprises an indication of which data items of the data block are required to generate the representation portion, what data values are to be generated from the required data items, and an organization of the generated data values within the representation portion;
• and
• performing, at the node device, the one or more transforms with the data block to generate the at least one row of the representation portion.