Real-time partitioned processing streaming

A method for processing data sets by using a distributed network to generate and process partitioned streams, the method comprising:

• receiving, by a message allocator processor of the distributed network, a plurality of data sets from one or more producer devices;
• for each of the plurality of data sets:
• identifying a tag or characteristic of the data set;
• identifying an initial partition stream from amongst a plurality of initial partition streams that corresponds to the tag or the characteristic;
• and
• appending the data set to the identified initial partition stream;
• managing a set of task processors;
• generating, by a first task processor hi the set of task processors via performance of a first task, processed data sets corresponding to the data sets appended to the identified initial partition stream;
• generating, by the first task processor, a processed partition stream that includes the processed data sets in the identified initial partition stream;
• facilitating, by the first task processor, routing the processed partition stream for further processing of the processed data sets hi accordance with one or more other tasks;
• calculating, by a second task processor in the set of task processors, a plurality of scores, each score of the plurality of scores calculated for a data set of the data sets appended to the identified initial partition stream;
• determining, by a third task processor in the set of task processors, that each data set of the data sets appended to the identified initial partition stream was received from the one or more producer devices within a given time period;
• generating, by the third task processor, a real-time analytic variable based on the plurality of scores;
• and
• facilitating, by the third task processor, providing the real-time analytic variable to a client device.

One or more non-transitory computer-readable storage media collectively storing computer-executable instructions that, when executed by one or more computer systems, configure the one or more computer systems to collectively perform operations comprising:

• receiving a plurality of data sets from one or more producer devices;
• for each of the plurality of data sets:
• identifying a tag or characteristic of the data set;
• identifying an initial partition stream from amongst a plurality of initial partition streams that corresponds to the tag or the characteristic;
• and
• appending the data set to the identified initial partition stream;
• managing a set of task processors;
• generating, by a first task processor in the set of task processors via performance of a first task, processed data sets corresponding to the data sets appended to the identified initial partition stream;
• generating, by the first task processor, a processed partition stream that includes the processed data sets in the identified initial partition stream;
• facilitating, by the first task processor, routing the processed partition stream for further processing of the processed data sets in accordance with one or more other tasks;
• calculating, by a second task processor in the set of task processors, a plurality of scores, each score of the plurality of scores calculated for a data set of the data sets appended to the identified initial partition stream;
• determining, by a third task processor in the set of task processors, that each data set of the data sets appended to the identified initial partition stream was received from the one or more producer devices within a given time period;
• generating, by the third task processor, a real-time analytic variable based on the plurality of scores;
• and
• facilitating, by the third task processor, providing the real-time analytic variable to a client device.