# US 2012/0259865 A1 · Automated correlation discovery for semi-structured processes

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0259865 A1
- Document ID: US20120259865A1
- Status: Published application
- Kind: A1
- Application number: 13/083,832
- Filed: 2011-04-11
- Published: 2012-10-11
- Earliest claimed: 2011-04-11
- Assignees: International Business Machines Corporation
- Inventors: Geetika T. Lakshmanan; Szabolcs Rozsnyai; Aleksander Slominski
- Web page: https://patentgrep.com/patent/US20120259865A1

## CPC

- G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computer-implemented method, system, and article of manufacture for determining a set of correlated data among heterogeneous computer applications. The method includes providing a computer system having software modules, receiving statistics on data relating to a first event and a second event, generating a confidence score for an attribute set, where the attribute set includes an attribute from the first event data and an attribute from the second event data, and selecting the attribute set as a set of correlated data if the confidence score is within a threshold value.

## Claims

### Claim 1

A computer-implemented method of determining a set of correlated data among heterogeneous computer applications, said method comprising:
providing a computer system, wherein said system comprises software modules embodied on a computer-readable medium, wherein said software modules comprise an input receiver module, a data processing module, a data transmitting module, and a data storage module;
receiving statistics on data relating to a first event that is capable of being processed by a first computer application and storing said statistics in said data storage module, wherein said statistics on said first event data is received from a data source, said first event data includes a first attribute, said first attribute includes at least one value, and said receiving statistics on said first event data is performed by said input receiver module;
receiving statistics on data relating to a second event that is capable of being processed by a second computer application and storing said statistics in said data storage module, wherein said statistics on said second event data is received from said data source, said second event data includes a second attribute, said second attribute includes at least one value, and said receiving statistics on said second event data is performed by said input receiver module;
generating a confidence score for an attribute set based on said received statistics on said first event data and said second event data, wherein said attribute set includes said first attribute and said second attribute, and said generating said confidence score is performed by said data processing module; and
selecting said attribute set as a set of correlated data if said confidence score is within a threshold value, wherein said selecting is performed by said data processing module.

### Claim 2 (depends on 1)

The method according to claim 1, wherein:
said statistics on said first event data is selected from the group consisting of:
(i) a count value, wherein said count value is the total number of instances in which said first attribute appears within said first event data;
(ii) a card value, wherein said card value is the total number of unique values for said first attribute;
(iii) an average attribute length value, wherein said average attribute length value is the average length of all values of said first attribute;
(iv) a cardinality value, wherein said cardinality value is the frequency of occurrence of said at least one value of said first attribute and wherein said occurrence is within said first attribute; and
(v) an attribute type for said first attribute; and
said statistics on said second event data is selected from the group consisting of:
(i) a count value, wherein said count value is the total number of instances in which said second attribute appears within said second event data;
(ii) a card value, wherein said card value is the total number of unique values for said second attribute;
(iii) an average attribute length value, wherein said average attribute length value is the average length of all values of said second attribute;
(iv) a cardinality value, wherein said cardinality value is the frequency of occurrence of said at least one value of said second attribute and wherein said occurrence is within said second attribute; and
(v) an attribute type for said second attribute.

### Claim 3 (depends on 1)

The method according to claim 1, wherein said generating comprises:
determining a difference set value, an average attribute length difference value, and a Levenshtein distance value for said attribute set;
assigning a first weight to said difference set value for said attribute set and weighing said difference set value according to said first weight;
assigning a second weight to said average attribute length difference value for said attribute set and weighing said average attribute length difference value according to said second weight;
assigning a third weight to said Levenshtein distance value for said attribute set and weighing said Levenshtein distance value according to said third weight; and
aggregating said weighted difference set value for said attribute set, said weighted average attribute length difference value for said attribute set, and said weighted Levenshtein distance value for said attribute set to generate said confidence score for said attribute set.

### Claim 4 (depends on 1)

The method according to claim 1, wherein said step of generating a confidence score for an attribute set based on said received statistics on said first event data and said received statistics on said second event data comprises:
determining if said first attribute is a highly indexable attribute;
determining if said second attribute is a mappable attribute;
if said first attribute is a highly indexable attribute and said second attribute is a mappable attribute,
determining a difference set value, an average attribute length difference value, and a Levenshtein distance value for said attribute set;
(i) assigning a first weight to said difference set value for said attribute set and weighing said difference set value according to said first weight;
(ii) assigning a second weight to said average attribute length difference value for said attribute set and weighing said average attribute length difference value according to said second weight;
(iii) assigning a third weight to said Levenshtein distance value for said attribute set and weighing said Levenshtein distance value according to said third weight; and
(iv) aggregating said weighted difference set value for said attribute set, said weighted average attribute length difference value for said attribute set, and said weighted Levenshtein distance value for said attribute set to generate said confidence score for said attribute set.

### Claim 5 (depends on 4)

The method according to claim 4, wherein:
determining if said first attribute is a highly indexable attribute comprises analyzing said statistics on said first event to determine if (i) the ratio between said card value for said first attribute and said count value for said first attribute is greater than an alpha value and if (ii) said average attribute length value for said first attribute is greater than an epsilon value.

### Claim 6 (depends on 1)

The method according to claim 1, wherein said step of generating a confidence score for an attribute set based on said received statistics on said first event data and said received statistics on said second event data comprises:
determining if said attribute type for said first attribute is a certain attribute type;
determining if said attribute type for said second attribute is said certain attribute type;
determining if said first attribute is a highly indexable attribute;
determining if said second attribute is a mappable attribute;
if said attribute type for said first attribute is said certain attribute type and said inferred attribute type for said second attribute is said certain attribute type and if said first attribute is a highly indexable attribute and said second attribute is a mappable attribute,
(i) determining a difference set value, an average attribute length difference value, and a Levenshtein distance value for said attribute set;
(ii) assigning a first weight to said difference set value for said attribute set and weighing said difference set value according to said first weight;
(iii) assigning a second weight to said average attribute length difference value for said attribute set and weighing said average attribute length difference value according to said second weight;
(iv) assigning a third weight to said Levenshtein distance value for said attribute set and weighing said Levenshtein distance value according to said third weight; and
(v) aggregating said weighted difference set value for said attribute set, said weighted average attribute length difference value for said attribute set, and said weighted Levenshtein distance value for said attribute set to generate said confidence score for said attribute set.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising:
removing said selected attribute set from said set of correlated data if said set of correlated data contains another attribute set having a first attribute that is the same as said first attribute and having a second attribute that is the same as said second attribute, wherein said removing is performed by said data processing module.

### Claim 8 (depends on 1)

The method according to claim 1, wherein
said system further comprises a display module,
said method further comprises generating a graph that shows a path between said first attribute and said second attribute based on said statistics on said first event data and said statistics on said second event data,
said generating said graph is performed if said confidence score is within a threshold value, and
said generating said graph is performed by said display module.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising:
if said attribute set is selected as a set of correlated data, transmitting information about said set of correlated data to said first computer application and said second computer application such that said first computer application is capable of processing said first event using said at least one value of said second attribute and said second computer application is capable of processing said second event using said at least one value of said first attribute, wherein said transmitting is performed by said data transmitting module.

### Claim 10

A computer-implemented system for determining a set of correlated data among heterogeneous computer applications, said system comprising:
an input receiver module configured to receive statistics on data relating to a first event that is capable of being processed on a first computer application and statistics on data relating to a second event that is capable of being processed on a second computer application, wherein said first event data includes a first attribute, said first attribute includes at least one value, said second event data includes a second attribute, and said second attribute includes at least one value;
a data processing module configured to generate a confidence score for an attribute set based on said received statistics on said first event data and said second event data, wherein said attribute set includes said first attribute and said second attribute, and for selecting said attribute set as a set of correlated data if said confidence score is within a threshold value; and
a data storage module configured to store said received statistics on said first event data and said second event data and for storing said generated confidence score.

### Claim 11 (depends on 10)

The system according to claim 10, wherein:
said statistics on said first event data is selected from the group consisting of:
(i) a count value, wherein said count value is the total number of instances in which said first attribute appears within said first event data;
(ii) a card value, wherein said card value is the total number of unique values for said first attribute;
(iii) an average attribute length value, wherein said average attribute length value is the average length of all values of said first attribute;
(iv) a cardinality value, wherein said cardinality value is the frequency of occurrence of said at least one value of said first attribute and wherein said occurrence is within said first attribute; and
(v) an attribute type for said first attribute;
said statistics on said second event data is selected from the group consisting of:
(i) a count value, wherein said count value is the total number of instances in which said second attribute appears within said second event data;
(ii) a card value, wherein said card value is the total number of unique values for said second attribute;
(iii) an average attribute length value, wherein said average attribute length value is the average length of all values of said second attribute;
(iv) a cardinality value, wherein said cardinality value is the frequency of occurrence of said at least one value of said second attribute and wherein said occurrence is within said second attribute; and
(v) an attribute type for said second attribute.

### Claim 12 (depends on 10)

The system according to claim 10, wherein
said input receiver module is configured to receive said statistics on said first event data and said statistics on said second event data from a data source.

### Claim 13 (depends on 10)

The system according to claim 10, wherein:
said data processing module is further configured to calculate said statistics on said first event and said statistics on said second event;
said data storage module is further configured to store said calculated statistics on said first event and said second event; and
said input receiver module is further configured to receive said calculated statistics on said first event data and said second event data from said data storage module.

### Claim 14 (depends on 10)

The system according to claim 10, wherein said data processing module is configured to generate said confidence score by:
determining a difference set value, an average attribute length difference value, and a Levenshtein distance value for said attribute set;
assigning a first weight to said difference set value for said attribute set and weighing said difference set value according to said first weight;
assigning a second weight to said average attribute length difference value for said attribute set and weighing said average attribute length difference value according to said second weight;
assigning a third weight to said Levenshtein distance value for said attribute set and weighing said Levenshtein distance value according to said third weight; and
aggregating said weighted difference set value for said attribute set, said weighted average attribute length difference value for said attribute set, and said weighted Levenshtein distance value for said attribute set to generate said confidence score for said attribute set.

### Claim 15 (depends on 10)

The system according to claim 10, wherein said data processing module is configured to generate said confidence score by:
determining if said first attribute is a highly indexable attribute;
determining if said second attribute is a mappable attribute;
if said first attribute is a highly indexable attribute and said second attribute is a mappable attribute,
(i) determining a difference set value, an average attribute length difference value, and a Levenshtein distance value for said attribute set;
(ii) assigning a first weight to said difference set value for said attribute set and weighing said difference set value according to said first weight;
(iii) assigning a second weight to said average attribute length difference value for said attribute set and weighing said average attribute length difference value according to said second weight;
(iv) assigning a third weight to said Levenshtein distance value for said attribute set and weighing said Levenshtein distance value according to said third weight; and
(v) aggregating said weighted difference set value for said attribute set, said weighted average attribute length difference value for said attribute set, and said weighted Levenshtein distance value for said attribute set to generate said confidence score for said attribute set.

### Claim 16 (depends on 10)

The system according to claim 10, wherein said data processing module is configured to generate said confidence score by:
determining if said attribute type for said first attribute is a certain attribute type;
determining if said attribute type for said second attribute is said certain attribute type;
determining if said first attribute is a highly indexable attribute;
determining if said second attribute is a mappable attribute;
if said attribute type for said first attribute is said certain attribute type and said inferred attribute type for said second attribute is said certain attribute type and if said first attribute is a highly indexable attribute and said second attribute is a mappable attribute,
(i) determining a difference set value, an average attribute length difference value, and a Levenshtein distance value for said attribute set;
(ii) assigning a first weight to said difference set value for said attribute set and weighing said difference set value according to said first weight;
(iii) assigning a second weight to said average attribute length difference value for said attribute set and weighing said average attribute length difference value according to said second weight;
(iv) assigning a third weight to said Levenshtein distance value for said attribute set and weighing said Levenshtein distance value according to said third weight; and
(v) aggregating said weighted difference set value for said attribute set, said weighted average attribute length difference value for said attribute set, and said weighted Levenshtein distance value for said attribute set to generate said confidence score for said attribute set.

### Claim 17 (depends on 10)

The system according to claim 10, further comprising:
a display module configured to display a graph that shows a path between said first attribute and said second attribute based on said statistics on said first event data and said second event data.

### Claim 18 (depends on 10)

The system according to claim 10, wherein
said data processing module is further configured remove said selected attribute set from said set of correlated data if said set of correlated data contains another attribute set having a first attribute that is the same as said first attribute and having a second attribute that is the same as said second attribute.

### Claim 19 (depends on 10)

The system according to claim 10, further comprising:
a. a data transmitting module configured to transmit information about said set of correlated data to said first computer application and said second computer application such that said first computer application is capable of processing said first event using said at least one value of said second attribute and said second computer application is capable of processing said second event using said at least one value of said first attribute, wherein said transmitting is performed if said attribute set is selected as a set of correlated data.

### Claim 20

An article of manufacture comprising a computer readable storage medium having computer readable instructions embodied therein that, when executed by a computer, implements a method of determining a set of correlated data among heterogeneous computer applications, said method comprising:
receiving statistics on data relating to a first event that is capable of being processed by a first computer application and storing said statistics in a data storage module, wherein said statistics on said first event data is received from a data source, said first event data includes a first attribute, said first attribute includes at least one value, and said receiving statistics on said first event data is performed by an input receiver module;
receiving statistics on data relating to a second event that is capable of being processed by a second computer application and storing said statistics in said data storage module, wherein said statistics on said second event data is received from said data source, said second event data includes a second attribute, said second attribute includes at least one value, and said receiving statistics on said second event data is performed by said input receiver module;
generating a confidence score for an attribute set based on said received statistics on said first event data and said second event data, wherein said attribute set includes said first attribute and said second attribute, and said generating said confidence score is performed by a data processing module; and
selecting said attribute set as a set of correlated data if said confidence score is within a threshold value, wherein said selecting is performed by said data processing module.
