# US 12,118,816 B2 · Continuous learning for document processing and analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,816 B2
- Document ID: US12118816B2
- Status: Granted
- Kind: B2
- Application number: 17/518,177
- Filed: 2021-11-03
- Published: 2023-05-04
- Granted: 2024-10-15
- Earliest claimed: 2021-11-03
- Assignees: ABBYY Development LLC
- Applicants: ABBYY Development Inc.
- Inventors: Stanislav Semenov; Andrei Zyuzin
- Web page: https://patentgrep.com/patent/US12118816B2

## CPC

- G06V 30/413: Classification of content, e.g. text, photographs or tables
- G06F 16/355: Creation or modification of classes or clusters
- G06F 16/383: using metadata automatically derived from the content
- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- G06V 30/1448: {based on markings or identifiers characterising the document or the area}
- G06V 30/1452: {based on positionally close symbols, e.g. amount sign or URL-specific characters}
- G06V 30/19107: {Clustering techniques}
- G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
- G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
- G06F 18/2433: Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06V 10/82: using neural networks
- G06V 30/418: Document matching, e.g. of document images
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A document processing method includes receiving one or more sets of documents, and assigning each document to one or more basic clusters based on the metadata of the document. It further includes for each cluster, training a respective basic cluster model detecting one or more visual element types, and responsive to a first threshold criterion measure related to the one or more basic clusters being satisfied, generating one or more superclusters based on an attribute shared by documents comprised by the plurality of basic clusters. The method also includes training a respective supercluster model detecting the one or more element types and generating a generalized cluster from the one or more superclusters. It includes training a generalized model for the generalized cluster, receiving an input document, assigning the input document to corresponding clusters, and detecting visual elements by processing the input document by each of the corresponding models.

## Claims

### Claim 1

A method comprising:
receiving, by a processing device, one or more sets of documents, wherein each document of the one or more sets of documents is associated with respective metadata;
assigning each document of the one or more sets of documents to one or more basic clusters based on the metadata of the document;
for each cluster of the one or more basic clusters, training a respective basic cluster model detecting one or more visual element types;
responsive to a first threshold criterion measure related to the one or more basic clusters being satisfied, generating one or more superclusters, each supercluster containing a respective plurality of basic clusters, based on an attribute shared by documents comprised by the plurality of basic clusters;
for each supercluster of the one or more superclusters, training a respective supercluster model detecting the one or more visual element types;
generating a generalized cluster from the one or more superclusters;
training a generalized model for the generalized cluster, the generalized model detecting one or more visual element types;
receiving an input document;
assigning the input document to a corresponding basic cluster, a corresponding supercluster, and a corresponding generalized cluster; and
detecting one or more visual elements by processing the input document by each of the corresponding basic cluster model, the corresponding supercluster model, and the corresponding generalized model.

### Claim 2 (depends on 1)

The method of claim 1, further comprising marking at least one set of documents to identify one or more visual elements.

### Claim 3 (depends on 1)

The method of claim 1, further comprising generating the one or more basic clusters of documents based on document attributes prior to assigning each document to the one or more basic clusters.

### Claim 4 (depends on 3)

The method of claim 3, wherein the threshold criterion measure related to the one or more basic clusters is a threshold number of the basic clusters.

### Claim 5 (depends on 1)

The method of claim 1, wherein the threshold criterion measure related to the one or more basic clusters is a threshold number of documents assigned to the one or more basic clusters.

### Claim 6 (depends on 1)

The method of claim 1, wherein each of the one or more visual element types is one of a number, a word, an image, a field, or a table.

### Claim 7 (depends on 1)

The method of claim 1, further comprising irreversibly anonymizing information contained in each document prior to training a corresponding model.

### Claim 8

A system comprising:
a memory;
a processor coupled to the memory, the processor configured to:
receive one or more sets of documents, wherein each document of the one or more sets of documents is associated with respective metadata;
assign each document of the one or more sets of documents to one or more basic clusters based on the metadata of the document;
for each cluster of the one or more basic clusters, train a respective basic cluster model detecting one or more visual element types;
responsive to a first threshold criterion measure related to the one or more basic clusters being satisfied, generate one or more superclusters, each supercluster containing a respective plurality of basic clusters, based on an attribute shared by the documents comprised by the plurality of basic clusters;
for each supercluster of the one or more superclusters, train a respective supercluster model detecting the one or more visual element types;
generate a generalized cluster from the one or more superclusters; and
train a generalized model for the generalized cluster, the generalized model detecting the one or more visual element types.

### Claim 9 (depends on 8)

The system of claim 8, wherein the processor is further configured to mark at least one set of documents to identify one or more visual elements.

### Claim 10 (depends on 8)

The system of claim 8, wherein the processor is further configured to generate the one or more basic clusters of documents based on document attributes prior to assigning each document to the one or more basic clusters.

### Claim 11 (depends on 10)

The system of claim 10, wherein the threshold criterion measure related to the one or more basic clusters is a threshold number of the basic clusters.

### Claim 12 (depends on 8)

The system of claim 8, wherein the threshold criterion measure related to the one or more basic clusters is a threshold number of documents assigned to the one or more basic clusters.

### Claim 13 (depends on 8)

The system of claim 8, wherein each of the one or more visual element types is one of a number, a word, an image, a field, or a table.

### Claim 14 (depends on 8)

The system of claim 8, wherein the processor is further configured to irreversibly anonymize information contained in each document prior to training a corresponding model.

### Claim 15

A non-transitory machine-readable storage medium including instructions that, when accessed by a processing device, cause the processing device to:
receive an input document;
assign the input document to a corresponding basic cluster, a corresponding supercluster, and a corresponding generalized cluster;
select a basic cluster model, a supercluster model, and a generalized cluster model respectively for the basic cluster, the supercluster, and the generalized cluster; and
detect one or more visual elements by processing the input document by each of the corresponding basic cluster model, the corresponding supercluster model, and the corresponding generalized model.

### Claim 16 (depends on 15)

The non-transitory machine-readable storage medium of claim 15, wherein the supercluster includes all documents contained in the basic cluster.

### Claim 17 (depends on 15)

The non-transitory machine-readable storage medium of claim 15, wherein the generalized cluster includes all documents contained in the supercluster.

### Claim 18 (depends on 15)

The non-transitory machine-readable storage medium of claim 15, wherein the basic cluster contains documents that share an attribute.

### Claim 19 (depends on 15)

The non-transitory machine-readable storage medium of claim 15, wherein the basic cluster model, the supercluster model, and the generalized cluster model are each respectively trained to detect a different type of visual element.

### Claim 20 (depends on 15)

The non-transitory machine-readable storage medium of claim 15, wherein the input document is assigned to the basic cluster based on a similarity of the document to a centroid of the basic cluster.