Systems and methods for correlating large datasets of electronic data records
AssigneeNasdaq Technology AB
InventorCorentin Plouet
Life of this patent
Estimating the term…
Abstract
A computer system is provided that stores reference data records and event data records that each correspond to one of the reference data records. Tokens are generated from the reference data records and sorted into subsets. Each reference data record that has a given token is included in the corresponding subset. A correlation value is calculated between members (e.g., the reference data records) of the subsets. The correlation values are calculated by determining a common time, calculating a reference value, calculating a percentage difference from the reference value, and then determining a correlation value (e.g., a correlation coefficient) using the percentage difference.
Claims20
A computer system comprising:
- an electronic storage system configured to store:
- a graph data structure;
- and
- transaction request records regarding data transaction requests processed by one or more computer systems, each of the data transaction requests including a numerical value and time data, each of the data transaction requests linked to one of a plurality of entity records;
- a processing system comprising instructions that, when executed by at least one hardware processor, are configured to cause the at least one hardware processor to perform operations comprising:
- updating the graph data structure based on a plurality of reference tokens, with each one of the plurality of reference tokens being associated with multiple different ones of the plurality of entity records, wherein each of the multiple different ones of the plurality of entity records is recorded into the graph data structure as a node;
- calculating, for each corresponding one of the plurality of reference tokens, a correlation value for each pair of the different ones of the plurality of entity records that are associated to the respective one of the plurality of reference tokens, wherein determination of each correlation value in connection with a first entity record and a second entity record includes:
- calculating a common baseline value for data transaction requests that are associated with the first entity record and the second entity record;
- and
- calculating, for each of the data transaction requests associated with the first entity record and second entity record, a percentage difference between the common baseline value and the numerical value of each respective data transaction request;
- and
- storing, to an edge of the graph data structure that is between a node for the first entity record and a node for the second entity record, the correlation value that is based on the percentage difference.
A method performed a computer system, the method comprising:
- storing, to an electronic storage system that is coupled to the computer system, a graph data structure and transaction request records regarding data transaction requests processed by one or more transaction computer systems, each of the data transaction requests including a numerical value and time data, each of the data transaction requests linked to one of a plurality of entity records;
- updating the graph data structure based on a plurality of reference tokens, with each one of the plurality of reference tokens being associated with multiple different ones of the plurality of entity records;
- performing a correlation process for a first entity record and a second entity record, of the different ones of the plurality of entity records that are associated with a given reference token, the correlation process including:
- (a) calculating a common baseline value for data transaction requests that are associated with the first entity record and the second entity record;
- and
- (b) calculating, for each of the data transaction requests associated with the first entity record and second entity record, a percentage difference between the common baseline value and the numerical value of each respective data transaction request;
- (c) calculating, based on the calculated percentage difference, a correlation coefficient between the first entity record and the second entity record;
- and
- (d) storing, to an edge of the graph data structure that is between a node associated with the first entity record and a node associated with the second entity record, the correlation coefficient.
A non-transitory computer readable storage medium having stored thereon computer readable instructions for use with a computer system that includes at least one processor, the stored computer readable instructions comprising instructions that cause the computer system to perform operations comprising:
- storing, to an electronic storage system that is coupled to the computer system, a graph data structure and transaction request records regarding data transaction requests processed by one or more transaction computer systems, each of the data transaction requests including a numerical value and time data, each of the data transaction requests linked to one of a plurality of entity records;
- updating the graph data structure based on a plurality of reference tokens, with each one of the plurality of reference tokens being associated with multiple different ones of the plurality of entity records;
- performing a correlation process for a first entity record and a second entity record, of the different ones of the plurality of entity records that are associated with a given reference token, the correlation process including:
- (a) calculating a common baseline value for data transaction requests that are associated with the first entity record and the second entity record;
- and
- (b) calculating, for each of the data transaction requests associated with the first entity record and second entity record, a percentage difference between the common baseline value and the numerical value of each respective data transaction request;
- (c) calculating, based on the calculated percentage difference, a correlation coefficient between the first entity record and the second entity record;
- and
- (d) storing, to an edge of the graph data structure that is between a node associated with the first entity record and a node associated with the second entity record, the correlation coefficient.
Dependencies are inferred from the claim text.