# US 2023/0053464 A1 · Systems, Methods, and Devices for Automatically Converting Explanation of Benefits (EOB) Printable Documents into Electronic Format using Artificial Intelligence Techniques

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0053464 A1
- Document ID: US20230053464A1
- Status: Published application
- Kind: A1
- Application number: 17/728,928
- Filed: 2022-04-25
- Published: 2023-02-23
- Earliest claimed: 2021-08-19
- Assignees: Data-Core Systems, Inc.
- Applicants: Data-Core Systems, Inc.
- Inventors: Anshuman Narayan; Jishnu Bhattacharyya; Dhrubajyoti Chakravarty; Pradeep K. Banerjee; Sin-Min Chang
- Web page: https://patentgrep.com/patent/US20230053464A1

## CPC

- G06V 30/413: Classification of content, e.g. text, photographs or tables
- G06V 30/418: Document matching, e.g. of document images
- G06F 40/174: Form filling; Merging
- G06V 10/82: using neural networks
- G06V 30/1444: {Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields}
- G06V 30/18: Extraction of features or characteristics of the image
- G06V 30/19013: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching (specially adapted for image segmentation G06T7/10; specially adapted for the analysis of motion G06T7/20; specially adapted for image alignment G06T7/30; specially adapted for the calculation of depth from stereo images G06T7/50; specially adapted for position determination G06T7/70)}
- G06V 30/19093: {Proximity measures, i.e. similarity or distance measures}
- G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments for automatically converting printed documents into electronic format using artificial intelligence techniques disclosed herein include: (i) receiving a plurality of images of documents; (ii) for each received image, using an image classification algorithm to classify the image as one of (a) an image of a first type of document, or (b) an image of a second type of document; (iii) for each image classified as an image of the first type of document, using an object localization algorithm to identity an area of interest in the image; (iv) for an identified area of interest, using an optical character recognition algorithm to extract text from the identified area of interest; and (v) populating a record associated with the document with the extracted text.

## Claims

### Claim 1

A method comprising:
receiving a plurality of images of documents;
for each received image, using an image classification algorithm to classify the image as one of (i) an image of a first type of document from one of a plurality of sources, or (ii) an image of a second type of document;
for each image classified as an image of the first type of document from one of the plurality of sources, using an object localization algorithm to identity an area of interest in the image;
for an identified area of interest, using an optical character recognition algorithm to extract text from the identified area of interest; and
populating a record associated with the document with the extracted text.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first type of document from one of the plurality of sources is an Explanation of Benefits (EOB) document from one of a plurality of payers, wherein the second type of document is a document other than an EOB document, and wherein using an image classification algorithm to classify the image as one of (i) an image of a first type of document from one of a plurality of sources or (ii) an image of a second type of document comprises:
using an autoencoder to generate a feature vector corresponding to the image;
determining a plurality of similarity scores for the image, wherein each similarity score corresponds to a similarity between (i) the feature vector corresponding to the image and (ii) a representative feature vector corresponding to one of the plurality of payers; and
classifying the image as corresponding to an EOB document from one of the plurality of payers based at least in part on the plurality of determined similarly scores.

### Claim 3 (depends on 2)

The method of claim 2, wherein the similarity scores comprise one or more of a cosine similarity, Euclidean distance, or a Kullback-Leibler divergence.

### Claim 4 (depends on 2)

The method of claim 2, wherein generating a feature vector corresponding to the image comprises encoding the image into a multi-dimensional vector with an autoencoder that has been trained using a set of images comprising (i) images of explanation of benefits documents from each of the plurality of payers and (ii) images of documents that are not explanation of benefits documents.

### Claim 5 (depends on 3)

The method of claim 3, wherein each representative feature vector corresponding to one of the plurality of payers (i) comprises features having a mean over standard deviation score above a provisioned threshold and (ii) excludes features having a mean over standard deviation score below the provisioned threshold.

### Claim 6 (depends on 1)

The method of claim 1, wherein the area of interest comprises a target band that substantially spans a width of the image, and wherein for each image classified as an image of the first type of document from one of the plurality of sources, using an object localization algorithm to identity an area of interest in the image comprises:
identifying a reference band in the image that spans the width of the image;
identifying the target band based at least in part on the reference band, wherein the target band is above the reference band and does not overlap the reference band.

### Claim 7 (depends on 1)

The method of claim 1, wherein for each image classified as an image of the first type of document from one of the plurality of sources, using an object localization algorithm to identity an area of interest in the image comprises:
extracting features from the image;
using a Faster Region-based Convolutional Neural Network (Faster R-CNN) to identify one or more predicted regions based on the extracted features, wherein each predicted region spans a width of the image;
for each predicted region identified via the Faster R-CNN, using a modified Non-Maximum (Non-Max) Suppression procedure to merge any one or more overlapping predicted regions into a single area of interest based at least in part on an Intersection of Union score; and
cropping the single area of interest from the image.

### Claim 8 (depends on 1)

The method of claim 1, wherein populating a record associated with the document with the extracted text comprises:
generating an EDI 835 compliant record based at least in part on the extracted text; and
identifying a previously-generated EDI 837 compliant record that corresponds to the generated EDI 835 compliant record.

### Claim 9 (depends on 2)

The method of claim 2, wherein classifying the image as corresponding to an EOB document from one of the plurality of payers based at least in part on the plurality of similarity scores for the image comprises:
classifying the image as corresponding to an EOB document from one of the plurality of payers based at least in part on (i) the plurality of similarly scores for the image and (ii) at least some of the text extracted from the identified area of interest in the image.

### Claim 10 (depends on 9)

The method of claim 9, wherein classifying the image as corresponding to an EOB document from one of the plurality of payers based at least in part on (i) the plurality of similarly scores for the image and (ii) at least some of the text extracted from the identified area of interest in the image comprises:
generating a first classification of the image, wherein the first classification identifies one of the plurality of payers;
when the one of the plurality of payers identified in the first classification is consistent with payer information within EDI-837 data corresponding to the image, selecting the first classification as the classification of the image; and
when the one of the plurality of payers identified in the first classification is inconsistent with payer information within EDI-837 data corresponding to the image, generating a second classification of the image, wherein the second classification identifies one of the plurality of payers that is different than the payer identified in the first classification.

### Claim 11

Tangible, non-transitory computer-readable media comprising program instructions stored therein, wherein the program instructions, when executed by one or more processors, cause a computing system to perform functions comprising:
receiving a plurality of images of documents;
for each received image, using an image classification algorithm to classify the image as one of (i) an image of a first type of document from one of a plurality of sources, or (ii) an image of a second type of document;
for each image classified as an image of the first type of document from one of the plurality of sources, using an object localization algorithm to identity an area of interest in the image;
for an identified area of interest, using an optical character recognition algorithm to extract text from the identified area of interest; and
populating a record associated with the document with the extracted text.

### Claim 12 (depends on 11)

The tangible, non-transitory computer-readable media of claim 11, wherein the first type of document from one of the plurality of sources is an Explanation of Benefits (EOB) document from one of a plurality of payers, wherein the second type of document is a document other than an EOB document, and wherein using an image classification algorithm to classify the image as one of (i) an image of a first type of document from one of a plurality of sources or (ii) an image of a second type of document comprises:
using an autoencoder to generate a feature vector corresponding to the image;
determining a plurality of similarity scores for the image, wherein each similarity score corresponds to a similarity between (i) the feature vector corresponding to the image and (ii) a representative feature vector corresponding to one of the plurality of payers; and
classifying the image as corresponding to an EOB document from one of the plurality of payers based at least in part on the plurality of determined similarly scores.

### Claim 13 (depends on 12)

The tangible, non-transitory computer-readable media of claim 12, wherein the similarity scores comprise one or more of a cosine similarity, Euclidean distance, or a Kullback-Leibler divergence.

### Claim 14 (depends on 12)

The tangible, non-transitory computer-readable media of claim 12, wherein generating a feature vector corresponding to the image comprises encoding the image into a multi-dimensional vector with an autoencoder that has been trained using a set of images comprising (i) images of explanation of benefits documents from each of the plurality of payers and (ii) images of documents that are not explanation of benefits documents.

### Claim 15 (depends on 13)

The tangible, non-transitory computer-readable media of claim 13, wherein each representative feature vector corresponding to one of the plurality of payers (i) comprises features having a above a provisioned threshold and (ii) excludes features having a mean over standard deviation score below the provisioned threshold.

### Claim 16 (depends on 11)

The tangible, non-transitory computer-readable media of claim 11, wherein the area of interest comprises a target band that substantially spans a width of the image, and wherein for each image classified as an image of the first type of document from one of the plurality of sources, using an object localization algorithm to identity an area of interest in the image comprises:
identifying a reference band in the image that spans the width of the image;
identifying the target band based at least in part on the reference band, wherein the target band is above the reference band and does not overlap the reference band.

### Claim 17 (depends on 11)

The tangible, non-transitory computer-readable media of claim 11, wherein for each image classified as an image of the first type of document from one of the plurality of sources, using an object localization algorithm to identity an area of interest in the image comprises:
extracting features from the image;
using a Faster Region-based Convolutional Neural Network (Faster R-CNN) to identify one or more predicted regions based on the extracted features, wherein each predicted region spans a width of the image;
for each predicted region identified via the Faster R-CNN, using a modified Non-Maximum (Non-Max) Suppression procedure to merge any one or more overlapping predicted regions into a single area of interest based at least in part on an Intersection of Union score; and
cropping the single area of interest from the image.

### Claim 18 (depends on 11)

The tangible, non-transitory computer-readable media of claim 11, wherein populating a record associated with the document with the extracted text comprises:
generating an EDI 835 compliant record based at least in part on the extracted text; and
identifying a previously-generated EDI 837 compliant record that corresponds to the generated EDI 835 compliant record.

### Claim 19 (depends on 12)

The tangible, non-transitory computer-readable media of claim 12, wherein classifying the image as corresponding to an EOB document from one of the plurality of payers based at least in part on the plurality of similarity scores for the image comprises:
generating a first classification of the image, wherein the first classification identifies one of the plurality of payers;
when the one of the plurality of payers identified in the first classification is consistent with payer information within EDI-837 data corresponding to the image, selecting the first classification as the classification of the image; and
when the one of the plurality of payers identified in the first classification is inconsistent with payer information within EDI-837 data corresponding to the image, generating a second classification of the image, wherein the second classification identifies one of the plurality of payers that is different than the payer identified in the first classification.

### Claim 20

A method performed by a computing system, the method comprising:
receiving a plurality of images of documents;
for each received image, using an image classification algorithm to classify the image as one of an image of an Explanation of Benefits (EOB) document from one of a plurality of payers or an image of a document other than an EOB document, wherein using the image classification algorithm to classify the image as one of an image of an Explanation of Benefits (EOB) document from one of a plurality of payers or an image of a document other than an EOB document comprises (i) using an autoencoder to generate a feature vector corresponding to the image, (ii) determining a plurality of similarity scores for the image, wherein each similarity score corresponds to a similarity between (a) the feature vector corresponding to the image and (b) a representative feature vector corresponding to one of the plurality of payers, and (iii) classifying the image as corresponding to an EOB document from one of the plurality of payers based at least in part on the plurality of determined similarly scores, wherein classifying the image as corresponding to an EOB document from one of the plurality of payers based at least in part on the plurality of determined similarly scores comprises (a) generating a first classification of the image, wherein the first classification identifies one of the plurality of payers, (b) when the one of the plurality of payers identified in the first classification is consistent with payer information within EDI-837 data corresponding to the image, selecting the first classification as the classification of the image, and (c) when the one of the plurality of payers identified in the first classification is inconsistent with payer information within EDI-837 data corresponding to the image, generating a second classification of the image, wherein the second classification identifies one of the plurality of payers that is different than the payer identified in the first classification;
for each image classified as an image of an Explanation of Benefits (EOB) document from one of the plurality of payers, using an object localization algorithm to identity an area of interest in the image;
for an identified area of interest, using an optical character recognition algorithm to extract text from the identified area of interest; and
populating a record associated with the Explanation of Benefits (EOB) document from one of the plurality of payers with the extracted text.
