# US 2025/0157209 A1 · Extracting key information from document using trained machine-learning models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0157209 A1
- Document ID: US20250157209A1
- Status: Published application
- Kind: A1
- Application number: 19/002,208
- Filed: 2024-12-26
- Published: 2025-05-15
- Earliest claimed: 2021-09-28
- Assignees: Oracle International Corporation
- Applicants: Oracle International Corporation
- Inventors: Yakupitiyage Don Thanuja Samodhye Dharmasiri; Xu Zhong; Ahmed Ataallah Ataallah Abobakr; Hongtao Yang; Budhaditya Saha; Shaoke Xu; Shashi Prasad Suravarapu; Mark Edward Johnson; Thanh Long Duong
- Web page: https://patentgrep.com/patent/US20250157209A1

## CPC

- G06V 10/82: using neural networks
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 30/153: {using recognition of characters or words}
- G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Techniques for extracting key information from a document using machine-learning models in a chatbot system is disclosed herein. In one particular aspect, a method is provided that includes receiving a set of data, which includes key fields, within a document at a data processing system that includes a table detection module, a key information extraction module, and a table extraction module. Text information and corresponding location data are extracted via optical character recognition. The table detection module detects whether one or more tables are present in the document and, if applicable, a location of each of the tables. The key information extraction module extracts text from the key fields. The table extraction module extracts each of the tables based on input from the optical character recognition and the table detection module. Extraction results include the text from the key fields and each of the tables can be output.

## Claims

### Claim 1

A computer-implemented method comprising:
extracting, by performing an optical character recognition operation, text information and location data associated with the text information from a document;
detecting, by a first module, one or more tables in the document, wherein the first module is a trained machine-learning model trained for object detection;
for each of the one or more tables determined to be in the document, detecting, by the first module, a location of each of the one or more tables in the document;
extracting, by a second module, each of the one or more tables determined to be in the document based on: (i) the text information and the location data associated with extracted text information, and (ii) the location of each of the one or more tables; and
outputting extraction results for the document, the extraction results including the one or more tables.

### Claim 2 (depends on 1)

The method of claim 1, further comprising extracting, by a key information extraction module and based on the text information and the location data, text from one or more key fields of the document, wherein the key information extraction module is a first trained neural network, and wherein the second module is a table extraction module that is a third trained neural network.

### Claim 3 (depends on 2)

The method of claim 2, further comprising training the first trained neural network, the trained machine-learning model, and/or the third trained neural network by:
receiving a set of training data that comprises synthetic training data;
performing a cross-entropy loss based on the set of training data;
generating one or more location graphs based on the cross-entropy loss; and
applying a graph convolutional network to the one or more location graphs to train the first trained neural network, the trained machine-learning model, and/or the third trained neural network.

### Claim 4 (depends on 1)

The method of claim 1, wherein a key information extraction module is used to extract the text from the one or more key fields, wherein the key information extraction module is a trained neural network that is trained via cross-entropy loss, wherein data used in the cross-entropy loss includes synthetic data generated using real data as a template of the synthetic data, and wherein the cross-entropy loss includes a location graph having a graph convolutional network applied to the location graph.

### Claim 5 (depends on 4)

The method of claim 4, wherein the location graph is a first location graph, wherein the cross-entropy loss includes a second location graph, wherein the first location graph represents a relative vertical location of the synthetic data, and wherein the second location graph represents a relative horizontal location of the synthetic data.

### Claim 6 (depends on 1)

The method of claim 1, wherein the second module is a trained neural network that is trained by:
receiving a bounding box of a table of a training document and a plurality of bounding boxes corresponding to a plurality of cells of the table;
extracting tokens within the table; and
labeling table coordinates of the tokens, the table coordinates including a starting row, a starting column, a row span, and a column span.

### Claim 7 (depends on 1)

The method of claim 1, wherein extracting, by the second module, each of the one or more tables determined to be within the document includes:
extracting a plurality of tokens from the one or more tables, the plurality of tokens corresponding to a plurality of cells of the one or more tables;
determining, based on the plurality of tokens, table coordinates of the plurality of tokens, the table coordinates including a starting row, a starting column, a row span, and a column span; and
using the table coordinates and the text information and location data associated with the text information to extract each of the one or more tables.

### Claim 8

A system comprising:
one or more processors; and
a non-transitory computer-readable medium coupled to the one or more processors, the non-transitory computer-readable medium storing instructions executable by the one or more processors to cause the one or more processors to perform operations comprising:
extracting, by performing an optical character recognition operation, text information and location data associated with the text information from a document;
detecting, by a first module, one or more tables in the document, wherein the first module is a trained machine-learning model trainable for object detection;
for each of the one or more tables determined to be in the document, detecting, by the first module, a location of each of the one or more tables in the document;
extracting, by a second module, each of the one or more tables determined to be in the document based on: (i) the text information and the location data associated with extracted text information, and (ii) the location of each of the one or more tables; and
outputting extraction results for the document, the extraction results including the one or more tables.

### Claim 9 (depends on 8)

The system of claim 8, wherein the operations further comprise extracting, by a key information extraction module and based on the text information and the location data, text from one or more key fields of the document, wherein the key information extraction module is a first trained neural network, and wherein the second module is a table extraction module that is a third trained neural network.

### Claim 10 (depends on 9)

The system of claim 9, wherein the operations further comprise training the first trained neural network, the trained machine-learning model, and/or the third trained neural network by:
receiving a set of training data that comprises synthetic training data;
performing a cross-entropy loss based on the set of training data;
generating one or more location graphs based on the cross-entropy loss; and
applying a graph convolutional network to the one or more location graphs to train the first trained neural network, the trained machine-learning model, and/or the third trained neural network.

### Claim 11 (depends on 8)

The system of claim 8, wherein a key information extraction module is usable to extract the text from the one or more key fields, wherein the key information extraction module is a trained neural network that is trainable via cross-entropy loss, wherein data used in the cross-entropy loss includes synthetic data generated using real data as a template of the synthetic data, and wherein the cross-entropy loss includes a location graph having a graph convolutional network applied to the location graph.

### Claim 12 (depends on 11)

The system of claim 11, wherein the location graph is a first location graph, wherein the cross-entropy loss includes a second location graph, wherein the first location graph represents a relative vertical location of the synthetic data, and wherein the second location graph represents a relative horizontal location of the synthetic data.

### Claim 13 (depends on 8)

The system of claim 8, wherein the second module is a trained neural network that is trained by:
receiving a bounding box of a table of a training document and a plurality of bounding boxes corresponding to a plurality of cells of the table;
extracting tokens within the table; and
labeling table coordinates of the tokens, the table coordinates including a starting row, a starting column, a row span, and a column span.

### Claim 14 (depends on 8)

The system of claim 8, wherein the operation of extracting, by the second module, each of the one or more tables determined to be within the document includes:
extracting a plurality of tokens from the one or more tables, the plurality of tokens corresponding to a plurality of cells of the one or more tables;
determining, based on the plurality of tokens, table coordinates of the plurality of tokens, the table coordinates including a starting row, a starting column, a row span, and a column span; and
using the table coordinates and the text information and location data associated with the text information to extract each of the one or more tables.

### Claim 15

A non-transitory computer-readable medium storing instructions executable by one or more processors for causing the one or more processors to perform operations comprising:
extracting, by performing an optical character recognition operation, text information and location data associated with the text information from a document;
detecting, by a first module, one or more tables in the document, wherein the first module is a trained machine-learning model trainable for object detection;
for each of the one or more tables determined to be in the document, detecting, by the first module, a location of each of the one or more tables in the document;
extracting, by a second module, each of the one or more tables determined to be in the document based on: (i) the text information and the location data associated with extracted text information, and (ii) the location of each of the one or more tables; and
outputting extraction results for the document, the extraction results including the one or more tables.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the operations further comprise extracting, by a key information extraction module and based on the text information and the location data, text from one or more key fields of the document, wherein the key information extraction module is a first trained neural network, and wherein the second module is a table extraction module that is a third trained neural network.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the operations further comprise training the first trained neural network, the trained machine-learning model, and/or the third trained neural network by:
receiving a set of training data that comprises synthetic training data;
performing a cross-entropy loss based on the set of training data;
generating one or more location graphs based on the cross-entropy loss; and
applying a graph convolutional network to the one or more location graphs to train the first trained neural network, the trained machine-learning model, and/or the third trained neural network.

### Claim 18 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein a key information extraction module is usable to extract the text from the one or more key fields, wherein the key information extraction module is a trained neural network that is trainable via cross-entropy loss, wherein data used in the cross-entropy loss includes synthetic data generated using real data as a template of the synthetic data, and wherein the cross-entropy loss includes a location graph having a graph convolutional network applied to the location graph.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the location graph is a first location graph, wherein the cross-entropy loss includes a second location graph, wherein the first location graph represents a relative vertical location of the synthetic data, and wherein the second location graph represents a relative horizontal location of the synthetic data.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the operation of extracting, by the second module, each of the one or more tables determined to be within the document includes:
extracting a plurality of tokens from the one or more tables, the plurality of tokens corresponding to a plurality of cells of the one or more tables;
determining, based on the plurality of tokens, table coordinates of the plurality of tokens, the table coordinates including a starting row, a starting column, a row span, and a column span; and
using the table coordinates and the text information and location data associated with the text information to extract each of the one or more tables.
