System, method, and program for performing aggregation process for each piece of received data

A system for receiving data keys that are values used for classification from outside and performing an aggregation process, the system comprising:

• a processor;
• and a computer readable medium having program instructions embodied therewith, the program instructions executable by the processor to implement the aggregation process;
• a reception unit for receiving pieces of data, each piece of data having a key value, as target of the aggregation process, and for classifying the each piece of data according to the key value of the each piece of data;
• a first storage unit for storing a database;
• a processing unit for invoking at least one operator that is for processing the each piece of data when the each piece of data is received as an input data having a key value of the input data by the reception unit, wherein pieces of data having identical key values are processed in order of arrival, wherein the processing includes one or more aggregation processes, each aggregation process comprising:
• performing a predetermined process on the input data by the at least one operator, outputting at least one of a first output data and a second output data, the first output data having the key value of the input data, and the second output data having the key value of the input data and to be stored in the database held by the first storage unit;
• determining if the aggregation process has been interrupted due to an occurrence of a fault;
• and
• in response to the determining if the aggregation process has been interrupted, cancelling a result of the aggregation process, returning the database to its state before the performing the aggregation process;
• and determining if the aggregation process has ended normally;
• and
• a second storage unit for, in response to the determining that the aggregation process has ended normally, storing a next piece of data in the database held by the first storage unit, the next piece of data having the key of the input data and generated in a process performed on the input data by the at least one operator, and further for storing the first output data output by the processing unit, wherein, when receiving the input each piece of data, when the first output data is stored in the second storage unit, the reception unit receives the first output data as the input data, and when there is no first output data stored in the second storage unit, the reception unit receives the input each piece of data as the input data, and the content of the database where the second output data is stored is fixed.

A computer program product for performing an aggregation process for each piece of received data, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:

• receiving the data as pieces of data arriving one after another, wherein each piece of data from the pieces of data has a key value;
• classifying each piece of data according to the key value of the each piece of data;
• processing pieces of data having identical key values in order of arrival, wherein the processing includes one or more aggregation processes, the one or more aggregation processes comprising:
• receiving a piece of data from the each piece of data as an input data having a key value of the input data;
• invoking, by a processing unit, at least one operator that is for performing an aggregation process for the input data;
• performing a predetermined process on the input data by the at least one operator;
• outputting at least one of a first output data and a second output data by the at least one operator, the first output data having the key value of the input data, and the second output data having the key value of the input data and to be stored in a database in a storage unit, wherein, when receiving the input piece of data, when the first output data is stored in the storage unit, a the reception unit receives the first output data as the input data, and the content of the database where the second output data is stored is fixed;
• determining if the aggregation process has been interrupted due to an occurrence of a fault and, in response to the determining that the aggregation process has been interrupted:
• cancelling a result of the aggregation process;
• and returning the database to its state before the performing the aggregation process;
• determining if the aggregation process has ended normally;
• and in response to the determining that the aggregation process has ended normally:
• storing a next piece of data in the database, the next piece of data having the key of the input data, and having been generated in the process by the at least one operator;
• receiving each subsequent piece of data from the each piece of received data;
• and performing, by at least one additional operator, a subsequent aggregation process on the each subsequent piece of data without performing a collective aggregation process after accumulating the each piece of data.