# US 2022/0012522 A1 · Preprocessing images for ocr using character pixel height estimation and cycle generative adversarial networks for better character recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0012522 A1
- Document ID: US20220012522A1
- Status: Published application
- Kind: A1
- Application number: 17/483,878
- Filed: 2021-09-24
- Published: 2022-01-13
- Earliest claimed: 2019-10-27
- Assignees: John Snow Labs Inc.
- Applicants: John Snow Labs Inc.
- Inventors: Jose Alberto Pablo Andreotti; David Talby
- Web page: https://patentgrep.com/patent/US20220012522A1

## CPC

- G06V 10/82: using neural networks
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 3/608: by skew deformation, e.g. two-pass or three-pass rotation
- G06T 5/70: Denoising; Smoothing
- G06T 5/80: Geometric correction
- G06T 7/60: Analysis of geometric attributes
- G06V 30/1478: {of characters or characters lines}
- G06V 30/162: Quantising the image signal
- G06V 30/19173: {Classification techniques}
- G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30176: Document
- G06V 30/10: Character recognition
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A text extraction computing method that comprises calculating an estimated character pixel height of text from a digital image. The method may scale the digital image using the estimated character pixel height and a preferred character pixel height. The method may binarizes the digital image. The method may remove distortions using a neural network trained by a cycle GAN on a set of source text images and a set of clean text images. The set of source text images and clean text images are unpaired. The source text images may be distorted images of text. Calculating the estimated character pixel height may include summarizing the rows of pixels into a horizontal projection, and determining a line-repetition period from the projection, and quantifying the portion of the line-repetition period that corresponds to the text as the estimated character pixel height. The method may extract characters from the digital image using OCR.

## Claims

### Claim 1

A text extraction computing method, comprising:
calculating an estimated character pixel height of text from a digital source image having lines of text;
scaling the digital image using the estimated character pixel height and a preferred character pixel height;
binarizing the digital image; and
extracting characters from the digital image using an optical character recognizer.

### Claim 2 (depends on 1)

The text extraction computing method of claim 1, where calculating the estimated character pixel height includes:
summarizing the rows of pixels into a horizontal projection, and
determining a line-repetition period from the horizontal projection, and
quantifying the portion of the line-repetition period that corresponds to the text as the estimated character pixel height.

### Claim 3 (depends on 2)

The text extraction computing method of claim 2, where determining the line-repetition period uses signal autocorrelation.

### Claim 4 (depends on 2)

The text extraction computing method of claim 2, where determining the line-repetition period uses Fast Fourier Transform.

### Claim 5 (depends on 1)

The text extraction computing method of claim 1, further comprising the steps of:
de-skewing the digital source image by rotating the digital source image;
removing distortion from the digital source image; and
postprocessing the extracted characters.

### Claim 6 (depends on 5)

The text extraction computing method of claim 5, where removing distortion uses a neural network trained by a cycle generative adversarial network on a set of source text images and a set of clean text images, where the set of source text images and the set of clean text images are unpaired, and where the source text images are distorted images of text.

### Claim 7 (depends on 5)

The text extraction computing method of claim 5, where removing distortion and scaling uses a preprocessing neural network trained by a cycle generative adversarial network on a set of source text images and a set of clean text images, where the set of source text images and the set of clean text images are unpaired, and where the source text images include distorted images and the image text has noise.

### Claim 8 (depends on 5)

The text extraction computing method of claim 5, where removing distortion, de-skewing, and scaling uses a preprocessing neural network trained using a cycle generative adversarial network on a set of source text images and a set of clean text images, where the set of source text images and the set of clean text images are unpaired, and where the source text images include distorted images and the image text has bad image lighting.

### Claim 9 (depends on 5)

The text extraction computing method of claim 5, where removing distortions, de-skewing, de-warping uses a preprocessing neural network trained using a cycle generative adversarial network on a set of source text images and a set of clean text images, where the set of source text images and the set of clean text images are unpaired, and where the source text images include distorted images, and the image text is skewed, and the image text is warped.

### Claim 10 (depends on 5)

The text extraction computing method of claim 5, where postprocessing includes a Levenshtein automaton model and a deep learning language model.

### Claim 11

A text extraction computing method, comprising:
calculating an estimated character pixel height of text from a digital source image having lines of text;
scaling the digital image larger or smaller using the estimated character pixel height and a preferred character pixel height;
binarizing the digital image; and
extracting characters from the digital image using an optical character recognizer.

### Claim 12 (depends on 11)

The text extraction computing method of claim 11, where calculating the estimated character pixel height includes:
summarizing the rows of pixels into a horizontal projection, and
determining a line-repetition period from the horizontal projection, and
quantifying the portion of the line-repetition period that corresponds to the text as the estimated character pixel height.

### Claim 13 (depends on 12)

The text extraction computing method of claim 12, where determining the line-repetition period uses signal autocorrelation.

### Claim 14 (depends on 12)

The text extraction computing method of claim 12, where determining the line-repetition period uses Fast Fourier Transform.

### Claim 15 (depends on 11)

The text extraction computing method of claim 11, further comprising the steps of:
de-skewing the digital source image by rotating the digital source image;
removing distortion from the digital source image; and
postprocessing the extracted characters.

### Claim 16 (depends on 15)

The text extraction computing method of claim 15, where removing distortion uses a neural network trained by a cycle generative adversarial network on a set of source text images and a set of clean text images, where the set of source text images and the set of clean text images are unpaired, and where the source text images are distorted images of text.

### Claim 17 (depends on 15)

The text extraction computing method of claim 15, where removing distortion and scaling uses a preprocessing neural network trained by a cycle generative adversarial network on a set of source text images and a set of clean text images, where the set of source text images and the set of clean text images are unpaired, and where the source text images include distorted images and the image text has noise.

### Claim 18 (depends on 15)

The text extraction computing method of claim 15, where removing distortion, de-skewing, and scaling uses a preprocessing neural network trained using a cycle generative adversarial network on a set of source text images and a set of clean text images, where the set of source text images and the set of clean text images are unpaired, and where the source text images include distorted images and the image text has bad image lighting.

### Claim 19 (depends on 15)

The text extraction computing method of claim 15, where removing distortions, de-skewing, de-warping uses a preprocessing neural network trained using a cycle generative adversarial network on a set of source text images and a set of clean text images, where the set of source text images and the set of clean text images are unpaired, and where the source text images include distorted images, and the image text is skewed, and the image text is warped.

### Claim 20 (depends on 15)

The text extraction computing method of claim 15, where postprocessing includes a Levenshtein automaton model and a deep learning language model.
