# US 2022/0036217 A1 · Machine learning technologies for assessing text legibility in electronic documents

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036217 A1
- Document ID: US20220036217A1
- Status: Published application
- Kind: A1
- Application number: 17/390,551
- Filed: 2021-07-30
- Published: 2022-02-03
- Earliest claimed: 2020-07-31
- Assignees: Cimpress Schweiz GmbH
- Applicants: Cimpress Schweiz GmbH
- Inventors: Vyacheslav Nykyforov
- Web page: https://patentgrep.com/patent/US20220036217A1

## CPC

- G06V 30/40: Document-oriented image-based pattern recognition
- G06F 40/109: Font handling; Temporal or kinetic typography
- G06N 5/04: Inference or reasoning models
- G06N 20/00: Machine learning
- G06T 11/26: Drawing of charts or graphs
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V 10/446: {using Haar-like filters, e.g. using integral image techniques}
- G06V 10/758: {Involving statistics of pixels or of feature values, e.g. histogram matching}
- G06V 10/993: {Evaluation of the quality of the acquired pattern}
- G06V 30/10: Character recognition
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods for using machine learning to assess text legibility in an electronic document are disclosed. According to certain aspects, an electronic device may train a machine learning model using training data that includes at least a representation of a text legibility level in the training data. Additionally, the electronic device may input the electronic document into the machine learning model, which may analyze the electronic document and output a representation of the text legibility level of a set of textual content included in the electronic document. The electronic device may display the output for review and assessment by a user, who may use the electronic device to facilitate any modifications to the electronic document.

## Claims

### Claim 1

A computer-implemented method of using machine learning to assess text legibility in an electronic document, the computer-implemented method comprising:
training, by a computer processor, a machine learning model using a training set of electronic documents, each of the training set of electronic documents comprising (i) a training set of textual content, (ii) a training set of visual content, and (iii) a training representation of a text legibility level of the training set of textual content;
storing the machine learning model in a memory;
accessing, by the computer processor, input data comprising an electronic document that comprises a set of textual content and a set of visual content;
analyzing, by the computer processor using the machine learning model, the input data; and
based on the analyzing, outputting, by the machine learning model, a representation of a text legibility level of the set of textual content of the electronic document.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
displaying, in a user interface, the representation of the text legibility level.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein outputting the representation of a text legibility level comprises:
outputting, by the machine learning model, a red-green-blue (RGB) representation of the text legibility level, the RGB representation comprising (i) a first image of a set of detected characters, (ii) a second image of a set of legibility values corresponding to the set of detected characters, and (iii) a third image of a set of text lines.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, further comprising:
calculating, by the computer processor using the red-green-blue (RGB) representation, a measure of the text legibility level of the set of textual content.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein calculating the measure of the text legibility level comprises:
generating, by the computer processor, a fourth image representing a difference between the second image of the set of legibility values and the first image of the set of detected characters, wherein the fourth image depicts a set of visual representations of a set of glyphs present in the set of textual content of the electronic document;
ascertaining, by the computer processor from the fourth image, at least a portion of the set of visual representations of the set of glyphs that do not meet a threshold characteristic;
modifying, by the computer processor, the fourth image by removing at least the portion of the set of visual representations; and
calculating, by the computer processor from the fourth image that was modified, the measure of the text legibility level of the set of textual content.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein calculating, from the fourth image that was modified, the measure of the text legibility level comprises:
determining, by the computer processor, a ratio of a size of a remaining portion of the set of visual representations in the fourth image to a size of the electronic document; and
calculating, by the computer processor based on the ratio, the measure of the text legibility level of the set of textual content.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein training the machine learning model comprises:
removing, from each of the training set of electronic documents, at least a portion of the training set of textual content; and
training, by the computer processor, the machine learning model using the training set of electronic documents with at least the portion of the training set of textual content removed therefrom.

### Claim 8

A system for using machine learning to assess text legibility in an electronic document, comprising:
a memory storing a set of computer-readable instructions and data associated with a machine learning model; and
a processor interfaced with the memory, and configured to execute the set of computer-readable instructions to cause the processor to:
train a machine learning model using a training set of electronic documents, each of the training set of electronic documents comprising (i) a training set of textual content, (ii) a training set of visual content, and (iii) a training representation of a text legibility level of the training set of textual content,
store the machine learning model in the memory,
access input data comprising an electronic document that comprises a set of textual content and a set of visual content,
analyze, using the machine learning model, the input data, and
based on the analyzing, output, by the machine learning model, a representation of a text legibility level of the set of textual content of the electronic document.

### Claim 9 (depends on 8)

The system of claim 8, further comprising:
a user interface;
wherein the processor is configured to execute the set of computer-readable instructions to further cause the processor to:
cause the user interface to display the representation of the text legibility level.

### Claim 10 (depends on 8)

The system of claim 8, wherein the processor outputs representation of the text legibility level as a red-green-blue (RGB) representation of the text legibility level, the RGB representation comprising (i) a first image of a set of detected characters, (ii) a second image of a set of legibility values corresponding to the set of detected characters, and (iii) a third image of a set of text lines.

### Claim 11 (depends on 10)

The system of claim 10, wherein the processor is configured to execute the set of computer-readable instructions to further cause the processor to:
calculate, using the red-green-blue (RGB) representation, a measure of the text legibility level of the set of textual content.

### Claim 12 (depends on 11)

The system of claim 11, wherein to calculate the measure of the text legibility level, the processor is configured to:
generate a fourth image representing a difference between the second image of the set of legibility values and the first image of the set of detected characters, wherein the fourth image depicts a set of visual representations of a set of glyphs present in the set of textual content of the electronic document,
ascertain, from the fourth image, at least a portion of the set of visual representations of the set of glyphs that do not meet a threshold characteristic,
modify the fourth image by removing at least the portion of the set of visual representations, and
calculate, from the fourth image that was modified, the measure of the text legibility level of the set of textual content.

### Claim 13 (depends on 12)

The system of claim 12, wherein to calculate, from the fourth image that was modified, the measure of the text legibility level, the processor is configured to:
determine a ratio of a size of a remaining portion of the set of visual representations in the fourth image to a size of the electronic document; and
calculating, by the computer processor based on the ratio, the measure of the text legibility level of the set of textual content.

### Claim 14 (depends on 8)

The system of claim 8, wherein to train the machine learning model, the processor is configured to:
remove, from each of the training set of electronic documents, at least a portion of the training set of textual content, and
train the machine learning model using the training set of electronic documents with at least the portion of the training set of textual content removed therefrom.

### Claim 15

A non-transitory computer-readable storage medium configured to store instructions executable by a computer processor, the instructions comprising:
instructions for training a machine learning model using a training set of electronic documents, each of the training set of electronic documents comprising (i) a training set of textual content, (ii) a training set of visual content, and (iii) a training representation of a text legibility level of the training set of textual content;
instructions for storing the machine learning model in a memory;
instructions for accessing input data comprising an electronic document that comprises a set of textual content and a set of visual content;
instructions for analyzing, using the machine learning model, the input data; and
instructions for, based on the analyzing, outputting, by the machine learning model, a representation of a text legibility level of the set of textual content of the electronic document.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein the instructions further comprise:
instructions for displaying, in a user interface, the representation of the text legibility level.

### Claim 17 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein the instructions for outputting the representation of a text legibility level comprise:
instructions for outputting, by the machine learning model, a red-green-blue (RGB) representation of the text legibility level, the RGB representation comprising (i) a first image of a set of detected characters, (ii) a second image of a set of legibility values corresponding to the set of detected characters, and (iii) a third image of a set of text lines.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the instructions further comprise:
instructions for calculating, using the red-green-blue (RGB) representation, a measure of the text legibility level of the set of textual content.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, wherein the instructions for calculating the measure of the text legibility level comprise:
instructions for generating a fourth image representing a difference between the second image of the set of legibility values and the first image of the set of detected characters, wherein the fourth image depicts a set of visual representations of a set of glyphs present in the set of textual content of the electronic document;
instructions for ascertaining, from the fourth image, at least a portion of the set of visual representations of the set of glyphs that do not meet a threshold characteristic;
instructions for modifying the fourth image by removing at least the portion of the set of visual representations; and
instructions for calculating, from the fourth image that was modified, the measure of the text legibility level of the set of textual content.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the instructions for calculating, from the fourth image that was modified, the measure of the text legibility level comprise:
instructions for determining a ratio of a size of a remaining portion of the set of visual representations in the fourth image to a size of the electronic document; and
instructions for calculating, based on the ratio, the measure of the text legibility level of the set of textual content.
