# US 2021/0279271 A1 · Data module creation from images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279271 A1
- Document ID: US20210279271A1
- Status: Published application
- Kind: A1
- Application number: 16/810,190
- Filed: 2020-03-05
- Published: 2021-09-09
- Earliest claimed: 2020-03-05
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Ali Khanafer; Mohammed Mostafa
- Web page: https://patentgrep.com/patent/US20210279271A1

## CPC

- G06F 16/51: Indexing; Data structures therefor; Storage structures
- G06F 16/5854: {using shape and object relationship}
- G06T 7/13: Edge detection
- G06T 7/162: involving graph-based methods
- G06T 7/73: using feature-based methods
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An approach is provided in which the approach performs object detection on an image to identify a set of edges and a set of tables from a set of objects corresponding to a set of data. The approach determines a set of associations between the set of tables and the set of edges in response to adjusting a set of bounding boxes corresponding to the set of edges. The approach generates a data module based on the set of associations, the set of tables, and the set of edges, and utilizes the data module to access the set of data.

## Claims

### Claim 1

A method implemented by an information handling system that includes a memory and a processor, the method comprising:
performing object detection on an image comprising a plurality of objects corresponding to a set of data, wherein the object detection identifies the plurality of objects as a set of edges and a set of tables;
determining a set of associations between the set of tables and the set of edges in response to adjusting a set of bounding boxes corresponding to the set of edges;
generating a data module based on the set of associations, the set of tables, and the set of edges; and
utilizing the data module to access the set of data.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
in response to determining that a selected one of the set of edges is a horizontal edge, adjusting a width of one of the set of bounding boxes that correspond to the selected edge;
identifying a first table bounding box and a second table bounding box that overlap the adjusted bounding box, wherein the first table bounding box corresponds to a first one of the set of tables and the second table bounding box corresponds to a second one of the set of tables; and
associating the first table and the second table to the selected edge.

### Claim 3 (depends on 2)

The method of claim 2 further comprising:
adjusting a height of the adjusted bounding box in response to determining that the adjusted bounding box fails to overlap the first table bounding box and the second table bounding box; and
continuing to expand the width of the adjusted bounding box and the height of the adjusted bounding box until the adjusted bounding box overlaps the first table bounding box and the second table bounding box.

### Claim 4 (depends on 1)

The method of claim 1 further comprising:
creating a connecting graph based on the determined set of associations;
translating the connecting graph into a JSON file; and
utilizing the JSON file to generate the data module.

### Claim 5 (depends on 4)

The method of claim 4 wherein the connecting graph is a logical schema diagram.

### Claim 6 (depends on 4)

The method of claim 4 wherein each of the set of edges comprise a set of cardinality symbols that indicate a relationship selected from the group consisting of a one-to-one cardinality, a one-to-many cardinality, and a many-to-many cardinality.

### Claim 7 (depends on 4)

The method of claim 4 wherein an analytics system converts the JSON file into the data module, and wherein the analytics system utilizes the data module to access the set of data.
(the final draft will include a set of system claims and a set of computer program claims that mirror the above method claims for a total of 20 claims).

### Claim 8

An information handling system comprising:
one or more processors;
a memory coupled to at least one of the processors;
a set of computer program instructions stored in the memory and executed by at least one of the processors in order to perform actions of:
performing object detection on an image comprising a plurality of objects corresponding to a set of data, wherein the object detection identifies the plurality of objects as a set of edges and a set of tables;
determining a set of associations between the set of tables and the set of edges in response to adjusting a set of bounding boxes corresponding to the set of edges;
generating a data module based on the set of associations, the set of tables, and the set of edges; and
utilizing the data module to access the set of data.

### Claim 9 (depends on 8)

The information handling system of claim 8 wherein the processors perform additional actions comprising:
in response to determining that a selected one of the set of edges is a horizontal edge, adjusting a width of one of the set of bounding boxes that correspond to the selected edge;
identifying a first table bounding box and a second table bounding box that overlap the adjusted bounding box, wherein the first table bounding box corresponds to a first one of the set of tables and the second table bounding box corresponds to a second one of the set of tables; and
associating the first table and the second table to the selected edge.

### Claim 10 (depends on 9)

The information handling system of claim 9 wherein the processors perform additional actions comprising:
adjusting a height of the adjusted bounding box in response to determining that the adjusted bounding box fails to overlap the first table bounding box and the second table bounding box; and
continuing to expand the width of the adjusted bounding box and the height of the adjusted bounding box until the adjusted bounding box overlaps the first table bounding box and the second table bounding box.

### Claim 11 (depends on 8)

The information handling system of claim 8 wherein the processors perform additional actions comprising:
creating a connecting graph based on the determined set of associations;
translating the connecting graph into a JSON file; and
utilizing the JSON file to generate the data module.

### Claim 12 (depends on 11)

The information handling system of claim 11 wherein the connecting graph is a logical schema diagram.

### Claim 13 (depends on 11)

The information handling system of claim 11 wherein each of the set of edges comprise a set of cardinality symbols that indicate a relationship selected from the group consisting of a one-to-one cardinality, a one-to-many cardinality, and a many-to-many cardinality.

### Claim 14 (depends on 11)

The information handling system of claim 11 wherein an analytics system converts the JSON file into the data module, and wherein the analytics system utilizes the data module to access the set of data.

### Claim 15

A computer program product stored in a computer readable storage medium, comprising computer program code that, when executed by an information handling system, causes the information handling system to perform actions comprising:
performing object detection on an image comprising a plurality of objects corresponding to a set of data, wherein the object detection identifies the plurality of objects as a set of edges and a set of tables;
determining a set of associations between the set of tables and the set of edges in response to adjusting a set of bounding boxes corresponding to the set of edges;
generating a data module based on the set of associations, the set of tables, and the set of edges; and
utilizing the data module to access the set of data.

### Claim 16 (depends on 15)

The computer program product of claim 15 wherein the information handling system performs further actions comprising:
in response to determining that a selected one of the set of edges is a horizontal edge, adjusting a width of one of the set of bounding boxes that correspond to the selected edge;
identifying a first table bounding box and a second table bounding box that overlap the adjusted bounding box, wherein the first table bounding box corresponds to a first one of the set of tables and the second table bounding box corresponds to a second one of the set of tables; and
associating the first table and the second table to the selected edge.

### Claim 17 (depends on 16)

The computer program product of claim 16 wherein the information handling system performs further actions comprising:
adjusting a height of the adjusted bounding box in response to determining that the adjusted bounding box fails to overlap the first table bounding box and the second table bounding box; and
continuing to expand the width of the adjusted bounding box and the height of the adjusted bounding box until the adjusted bounding box overlaps the first table bounding box and the second table bounding box.

### Claim 18 (depends on 15)

The computer program product of claim 15 wherein the information handling system performs further actions comprising:
creating a connecting graph based on the determined set of associations;
translating the connecting graph into a JSON file; and
utilizing the JSON file to generate the data module.

### Claim 19 (depends on 18)

The computer program product of claim 18 wherein the connecting graph is a logical schema diagram.

### Claim 20 (depends on 18)

The computer program product of claim 18 wherein each of the set of edges comprise a set of cardinality symbols that indicate a relationship selected from the group consisting of a one-to-one cardinality, a one-to-many cardinality, and a many-to-many cardinality.
