# US 2019/0317938 A1 · Method, program, and system for automatic discovery of relationship between fields in environment where different types of data sources coexist

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0317938 A1
- Document ID: US20190317938A1
- Status: Published application
- Kind: A1
- Application number: 16/089,532
- Filed: 2017-03-27
- Published: 2019-10-17
- Earliest claimed: 2016-03-31
- Assignees: Smart Insight Corporation
- Applicants: Smart Insight Corporation
- Inventors: Rahul AGARWALLA; Kiyoshi MACHIDA; Toshiaki NAKAGAWA
- Web page: https://patentgrep.com/patent/US20190317938A1

## CPC

- G06F 12/00: Accessing, addressing or allocating within memory systems or architectures (digital input from, or digital output to record carriers, e.g. to disk storage units, G06F3/06)
- G06F 16/2458: Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F 16/2456: {Join operations}
- G06F 16/256: {in federated or virtual databases}
- G06F 16/288: {Entity relationship models}
- G06F 16/9024: {Graphs; Linked lists (G06F16/9027 takes precedence)}
- G06F 17/11: for solving equations {, e.g. nonlinear equations, general mathematical optimization problems (optimization specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

[PROBLEM TO BE SOLVED] To efficiently associate fields for performing data analysis across a plurality of data sources having different designs, facilitating various analyses. [SOLUTION] Set indices for each field in a plurality of data stores, determine similarity between each field, and generate an enterprise data graph, a graph format data with highly similar fields as nodes. To determine similarity, a technique such as a morphological analysis provided by a search engine may be used. By using the enterprise data graph, it is possible to handle various inquiry requests that run across multiple data stores.

## Claims

### Claim 1

A computer-executable method for analyzing data in multiple data stores comprising:
removing duplicates from a set of strings in multiple fields in tables in the multiple data stores;
storing the set of strings into an inverted index;
evaluating similarity among the multiple fields based on similarity of the strings stored in the inverted index; and
generating a graph, each node of the graph representing a table containing a similar field, and an edge of the graph representing similarity between the tables.

### Claim 2 (depends on 1)

The method according to claim 1, wherein,
the evaluating similarity step further comprising:
dividing the set of strings in the multiple fields by applying morphological analysis;
calculating a cosine similarity between the strings; and
applying a logistics function to the cosine similarity.

### Claim 3 (depends on 2)

The method according to claim 2, wherein,
the evaluating similarity step further comprising:
storing a set of strings in the multiple fields into a-temporary tables;
applying a natural join to the temporary tables; and
calculating a similarity between the temporary tables.

### Claim 4 (depends on 1)

The method according to claim 1, further comprising:
evaluating similarity between the multiple fields in tables in the multiple data stores, based on attributes of the fields;
wherein the attributes include cardinality, non-distinctive values, a range of a histogram, a number of null values, and a number of non-null values.

### Claim 5 (depends on 1)

The method according to claim 1, further comprising:
receiving a query to a first field in a first table in a first data store;
identifying, using the graph, a second field that is similar to the first field; and
displaying information about the second field, a second table containing the second field or a second data store containing the second table.

### Claim 6 (depends on 1)

The method according to claim 1, further comprising:
displaying information on a first data store;
receiving a query to a first field in a first table in a first data store;
identifying, using the graph, a second field that is similar to the first field;
displaying information on a second data store containing the second field; and
updating display of the information of the first data store and the information of the second data store simultaneously.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising:
receiving a query to a first field in a first table in a first data store;
identifying, using the graph, a second field that is similar to the first field; and
displaying the result of the query to the first data store and a second data store containing a second table containing the second field on a same screen.

### Claim 8 (depends on 1)

The method according to claim 1, further comprising:
receiving a keyword from a user;
searching tables containing a field containing the keyword from the inverted index; and
displaying the table or the field.

### Claim 9

A non-transitory computer-readable storage medium comprising a recorded computer program for analyzing data in multiple data stores comprising computer-executable instructions for:
removing duplicates from a set of strings in multiple fields in tables in the multiple data stores;
storing the set of strings into an inverted index;
evaluating similarity among the multiple fields based on similarity of the strings stored in the inverted index; and,
generating a graph, each node of the graph representing a table containing a similar field, and an edge of the graph representing similarity between the tables.

### Claim 10 (depends on 9)

The non-transitory computer-readable storage medium comprising the recorded computer program according to claim 9, wherein:
the instructions for evaluating similarity further comprising computer-executable instructions for:
dividing the set of strings in the multiple fields by applying morphological analysis;
calculating a consine similarity between the strings; and
applying a logistics function to the cosine similarity.

### Claim 11 (depends on 9)

The non-transitory computer-readable storage medium comprising the recorded computer program according to claim 9, wherein:
the instructions for evaluating similarity further comprising computer-executable instructions for:
storing a set of strings in the multiple fields into temporary tables;
applying a natural join to the temporary tables; and
calculating a similarity between the temporary tables.

### Claim 12 (depends on 9)

The non-transitory computer-readable storage medium comprising the recorded computer program according to claim 9 further comprising computer-executable instructions for:
evaluating similarity between the multiple fields in tables in the multiple data stores, based on attributes of the fields;
wherein the attributes include cardinality, non-distinctive values, a range of a histogram, a number of null values, and a number of non-null values.

### Claim 13 (depends on 9)

The non-transitory computer-readable storage medium comprising the recorded computer program according to claim 9 further comprising computer-executable instructions for:
receiving a query to a first field in a first table in a first data store;
identifying, using the graph, a second field that is similar to the first field; and
displaying information about the second field, a second table containing the second field or a second data store containing the second table.

### Claim 14 (depends on 9)

The non-transitory computer-readable storage medium comprising the recorded computer program according to claim 9 further comprising computer-executable instructions for:
displaying information on a first data store;
receiving a query to a first field in a first table in a first data store;
identifying, using the graph, a second field that is similar to the first field;
displaying information on a second data store containing the second field; and
updating display of the information of the first data store and the information of the second data store simultaneously.

### Claim 15 (depends on 9)

The non-transitory computer-readable storage medium comprising the recorded computer program according to claim 9 further comprising computer-executable instructions for:
receiving a query to a first field in a first table in a first data store;
identifying, using the graph, a second field that is similar to the first field; and
displaying the result of the query to the first data store and a second data store containing a second table containing the second field on a same screen.

### Claim 16 (depends on 9)

The non-transitory computer-readable storage medium comprising the recorded computer program according to claim 9 further comprising computer-executable instructions for:
receiving a keyword from a user;
searching tables containing a field containing the keyword from the inverted index; and
displaying the table or the field.

### Claim 17

A computer system for analyzing data in multiple data stores comprising:
a duplicate-remover that removes duplicates from a set of strings in multiple fields in tables in the multiple data stores;
a storer that stores the set of strings into an inverted index;
an evaluator that evaluates similarity among the multiple fields based on similarity of the strings stored in the inverted index; and,
a generator that generates a graph, each node of the graph representing a table containing a similar field, and an edge of the graph representing similarity between the tables.

### Claim 18 (depends on 17)

The computer system according to claim 17 wherein:
the evaluator further comprising:
a divider that divides the set of strings in the multiple fields by applying morphological analysis;
a calculator that calculates a cosine similarity between the strings; and
an applier that applies a logistics function to the cosine similarity.

### Claim 19 (depends on 18)

The computer system according to claim 18 wherein:
the evaluator further comprising:
a storer that stores a set of strings in the multiple fields into temporary tables;
an applier that applies a natural join to the temporary tables; and
a calculator that calculates a similarity between the temporary tables.

### Claim 20 (depends on 17)

The computer system according to claim 17 further comprising:
an evaluator that evaluates similarity between the multiple fields in tables in the multiple data stores, based on attributes of the fields;
wherein the attributes include cardinality, non-distinctive values, a range of a histogram, a number of null values, and a number of non-null values.
