# US 2023/0410462 A1 · Automated categorization and assembly of low-quality images into electronic documents

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410462 A1
- Document ID: US20230410462A1
- Status: Published application
- Kind: A1
- Application number: 18/458,014
- Filed: 2023-08-29
- Published: 2023-12-21
- Earliest claimed: 2021-05-03
- Assignees: Bank of America Corporation
- Applicants: Bank of America Corporation
- Inventors: Van Nguyen; Sean Michael Byrne; Syed Talha; Aftab Khan; Beena Khushalani; Sharad K. Kalyani
- Web page: https://patentgrep.com/patent/US20230410462A1

## CPC

- G06F 40/30: Semantic analysis
- G06V 10/464: {using a plurality of salient features, e.g. bag-of-words [BoW] representations}
- G06F 16/93: Document management systems
- G06F 18/24: Classification techniques
- G06F 18/253: {of extracted features}
- G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30)
- G06N 20/00: Machine learning
- G06V 10/30: Noise filtering
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/806: {of extracted features}
- G06V 30/41: Analysis of document content (recognition of printed characters based on code marks G06V30/224)
- G06V 30/413: Classification of content, e.g. text, photographs or tables
- G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
- G06F 40/216: using statistical methods
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An apparatus includes a memory and processor. The memory stores document categories, text generated from an image a physical document page, and a machine learning algorithm. The machine learning algorithm is configured to extract features associated with natural language processing and features associated with the text. The machine learning algorithm is also configured to generate a feature vector that includes the first and second pluralities of features, and to generate, based on the feature vector, a set of probabilities, each of which is associated with a document category and indicates a probability that the physical document from which the text was generated belongs to that document category. The processor applies the machine learning algorithm to the text, to generate the set of probabilities, identifies a largest probability, and assigns the image to the associated document category.

## Claims

### Claim 1

An apparatus comprising:
a memory configured to store:
a set of document categories;
a first set of text generated from a first image of a page of a physical document, the page comprising a second set of text; and
a machine learning algorithm configured, when applied to the first set of text and executed by a hardware processor, to:
extract a set of features from the first set of text, the set of features comprising:
a first plurality of features obtained by performing natural language processing feature extraction operations on the first set of text; and
a second plurality of features, each feature of the second plurality of features assigned to an error type of a set of error types and associated with one or more errors of a set of errors, the one or more errors belonging to the assigned error type of the set of error types;
generate a first feature vector comprising the first plurality of features and the second plurality of features; and
generate, based on the first feature vector, a first set of probabilities, each probability of the first set of probabilities associated with a document category of the set of document categories and indicating a probability that the physical document from which the first set of text was generated belongs to the associated document category; and
the hardware processor communicatively coupled to the memory, the hardware processor configured to:
apply the machine learning algorithm to the first set of text, to generate the first set of probabilities;
identify a largest probability of the first set of probabilities; and
assign the first image to the document category associated with the largest probability of the first set of probabilities.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein at least one error type of the set of error types is associated with at least one of:
non-ascii characters;
non-English words;
stray letters other than “a” and “I”;
numbers comprising one or more letters; or
misplaced punctuation marks.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein:
the image comprises at least one of:
a scanned image generated by scanning the page of the physical document; or
a faxed image generated by faxing the page of the physical document; and
the noise in the image is associated with at least one of:
ruled lines on the page of the physical document;
hole punches on the page of the physical document;
uneven contrast in the image;
one or more folds in the page of the physical document;
handwritten text in a margin of the page of the physical document;
a background color of the image; or
interfering strokes in the image, wherein the physical document comprises double-sided pages.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the machine learning algorithm comprises at least one of a neural network algorithm, a decision tree algorithm, a naive Bayes algorithm, and a logistic regression algorithm.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein:
the memory is further configured to store a third set of text generated from a second image of the page of the physical document;
the machine learning algorithm is further configured, when applied to the third set of text and executed by the hardware processor, to:
extract a second set of features from the third set of text, the second set of features comprising:
a first plurality of features obtained by performing the natural language processing feature extraction operations on the third set of text; and
a second plurality of features, each feature of the second plurality of features assigned to an error type of the set of error types and associated with one or more errors of a second set of errors in the third set of text, the one or more errors belonging to the assigned error type of the set of error types;
generate a second feature vector comprising the first plurality of features of the second set of features and the second plurality of features of the second set of features, wherein the second feature vector is different from the first feature vector; and
generate, based on the second feature vector, a second set of probabilities, each probability of the second set of probabilities associated with a document category of the set of document categories and indicating a probability that the physical document from which the third set of text was generated belongs to the associated document category; and
the hardware processor is further configured to:
apply the machine learning algorithm to the third set of text, to generate the second set of probabilities;
identify a largest probability of the second set of probabilities; and
assign the second image to the document category associated with the largest probability of the second set of probabilities, wherein the document category assigned to the second image is the same as the document category assigned to the first image.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein:
the memory is further configured to store training data comprising:
a plurality of sets of text, each set of text of the plurality of sets of text generated from an image of a page of a known physical document of a set of known physical documents; and
a plurality of labels, each label of the plurality of labels assigned to a set of text of the plurality of sets of text, and identifying a document category of the set of document categories into which the known physical document of the set of known physical documents has previously been categorized; and
the hardware processor is further configured to use the training data to train the machine learning algorithm.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein:
the machine learning algorithm comprises a set of adjustable weights; and
training the machine learning algorithm comprises adjusting the set of adjustable weights.

### Claim 8

A method comprising:
applying a machine learning algorithm to a first set of text, to generate a first set of probabilities, wherein:
the first set of text was generated from a first image of a page of a physical document, wherein the physical document is assigned to a first document category of a set of document categories;
the machine learning algorithm is configured, when applied to the first set of text, to:
extract a set of features from the first set of text, the set of features comprising:
a first plurality of features obtained by performing natural language processing feature extraction operations on the first set of text; and
a second plurality of features, each feature of the second plurality of features assigned to an error type of a set of error types and associated with one or more errors of a set of errors, the one or more errors belonging to the assigned error type of the set of error types;
generate a first feature vector comprising the first plurality of features and the second plurality of features; and
generate, based on the first feature vector, the first set of probabilities; and
each probability of the first set of probabilities is associated with a document category of the set of document categories and indicates a probability that the physical document from which the first set of text was generated belongs to the associated document category;
identifying a largest probability of the first set of probabilities; and
assigning the first image to the document category associated with the largest probability of the first set of probabilities.

### Claim 9 (depends on 8)

The method of claim 8, wherein at least one error type of the set of error types is associated with at least one of:
non-ascii characters;
non-English words;
stray letters other than “a” and “I”;
numbers comprising one or more letters; or
misplaced punctuation marks.

### Claim 10 (depends on 8)

The method of claim 8, wherein:
the image comprises at least one of:
a scanned image generated by scanning the page of the physical document; or
a faxed image generated by faxing the page of the physical document; and
the noise in the image is associated with at least one of:
ruled lines on the page of the physical document;
hole punches on the page of the physical document;
uneven contrast in the image;
one or more folds in the page of the physical document;
handwritten text in a margin of the page of the physical document;
a background color of the image; or
interfering strokes in the image, wherein the physical document comprises double-sided pages.

### Claim 11 (depends on 8)

The method of claim 8, wherein the machine learning algorithm comprises at least one of a neural network algorithm, a decision tree algorithm, a naive Bayes algorithm, or a logistic regression algorithm.

### Claim 12 (depends on 8)

The method of claim 8, further comprising:
applying the machine learning algorithm to a third set of text, to generate a second set of probabilities, wherein:
the third set of text was generated from a second image of the page of the physical document;
the machine learning algorithm is configured, when applied to the third set of text, to:
extract a second set of features from the third set of text, the second set of features comprising:
a first plurality of features obtained by performing the natural language processing feature extraction operations on the third set of text; and
a second plurality of features, each feature of the second plurality of features assigned to an error type of the set of error types and associated with one or more errors of a second set of errors, the one or more errors belonging to the assigned error type of the set of error types;
generate a second feature vector comprising the first plurality of features of the second set of features and the second plurality of features of the second set of features, wherein the second feature vector is different from the first feature vector; and
generate, based on the second feature vector, the second set of probabilities; and
each probability of the second set of probabilities is associated with a document category of the set of document categories and indicates a probability that the physical document from which the third set of text was generated belongs to the associated document category;
identifying a largest probability of the second set of probabilities; and
assigning the second image to the document category associated with the largest probability of the second set of probabilities, wherein the document category assigned to the second image is the same as the document category assigned to the first image.

### Claim 13 (depends on 8)

The method of claim 8, further comprising using training data to train the machine learning algorithm, wherein the training data comprises:
a plurality of sets of text, each set of text of the plurality of sets of text generated from an image of a page of a known physical document of a set of known physical documents; and
a plurality of labels, each label of the plurality of labels assigned to a set of text of the plurality of sets of text, and identifying a document category of the set of document categories into which the known physical document of the set of known physical documents has previously been categorized.

### Claim 14 (depends on 13)

The method of claim 13, wherein:
the machine learning algorithm comprises a set of adjustable weights; and
training the machine learning algorithm comprises adjusting the set of adjustable weights.

### Claim 15

A system comprising:
a memory configured to store:
a first image of a page of a physical document;
a first machine learning algorithm configured, when executed by a hardware processor, to convert a first image of a page of a physical document into a first set of text, wherein the page comprises a second set of text; and
a second machine learning algorithm configured, when applied to the first set of text and executed by the hardware processor, to:
extract a set of features from the first set of text, the set of features comprising:
a first plurality of features obtained by performing natural language processing feature extraction operations on the first set of text; and
a second plurality of features, each feature of the second plurality of features assigned to an error type of a set of error types and associated with one or more errors of a set of errors, the one or more errors of the set of errors belonging to the assigned error type of the set of error types;
generate a first feature vector comprising the first plurality of features and the second plurality of features; and
generate, based on the first feature vector, a first set of probabilities, each probability of the first set of probabilities associated with a document category of the set of document categories and indicating a probability that the physical document from which the first set of text was generated belongs to the associated document category; and
the hardware processor communicatively coupled to the memory, the hardware processor configured to:
apply the first machine learning algorithm to the first image, to generate the first set of text;
apply the second machine learning algorithm to the first set of text, to generate the first set of probabilities;
identify a largest probability of the first set of probabilities; and
assign the first image to the document category associated with the largest probability of the first set of probabilities.

### Claim 16 (depends on 15)

The system of claim 15, wherein at least one error type of the set of error types is associated with at least one of:
non-ascii characters;
non-English words;
stray letters other than “a” and “I”;
numbers comprising one or more letters; or
misplaced punctuation marks.

### Claim 17 (depends on 15)

The system of claim 15, wherein:
the image comprises at least one of:
a scanned image generated by scanning the page of the physical document; or
a faxed image generated by faxing the page of the physical document; and
the noise in the image is associated with at least one of:
ruled lines on the page of the physical document;
hole punches on the page of the physical document;
uneven contrast in the image;
one or more folds in the page of the physical document;
handwritten text in a margin of the page of the physical document;
a background color of the image; or
interfering strokes in the image, wherein the physical document comprises double-sided pages.

### Claim 18 (depends on 15)

The system of claim 15, wherein the second machine learning algorithm comprises at least one of a neural network algorithm, a decision tree algorithm, a naive Bayes algorithm, and a logistic regression algorithm.

### Claim 19 (depends on 15)

The system of claim 15, wherein:
the memory is further configured to store training data comprising:
a plurality of sets of text, each set of text of the plurality of sets of text generated from an image of a page of a known physical document of a set of known physical documents; and
a plurality of labels, each label of the plurality of labels assigned to a set of text of the plurality of sets of text, and identifying a document category of the set of document categories into which the known physical document of the set of known physical documents has previously been categorized; and
the hardware processor is further configured to use the training data to train the second machine learning algorithm.

### Claim 20 (depends on 19)

The system of claim 19, wherein:
the second machine learning algorithm comprises a set of adjustable weights; and
training the second machine learning algorithm comprises adjusting the set of adjustable weights.
