# US 2017/0372163 A1 · Systems and methods for incremental character recognition to recognize characters in images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0372163 A1
- Document ID: US20170372163A1
- Status: Published application
- Kind: A1
- Application number: 15/194,021
- Filed: 2016-06-27
- Published: 2017-12-28
- Earliest claimed: 2016-06-27
- Assignees: Facebook, Inc.
- Applicants: Facebook, Inc.
- Inventors: Benjamin Ray; Ahmad Abdulmageed Mohammed Abdulkader; Sofus Attila Macskassy
- Web page: https://patentgrep.com/patent/US20170372163A1

## CPC

- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 30/147: {Determination of region of interest}
- G06V 30/10: Character recognition
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems, methods, and non-transitory computer-readable media can acquire an image that depicts at least one character. A set of pixels, within the image, through which the at least one character is depicted can be identified. At least one linear portion, within the image, can be identified based on the set of pixels. For each sub-portion within the at least one linear portion, a respective first confidence score representing a respective first likelihood that a respective sub-portion depicts the at least one character can be determined.

## Claims

### Claim 1

A computer-implemented method comprising:
acquiring, by a computing system, an image that depicts at least one character;
identifying, by the computing system, a set of pixels, within the image, through which the at least one character is depicted;
identifying, by the computing system, at least one linear portion, within the image, based on the set of pixels; and
determining, by the computing system, for each sub-portion within the at least one linear portion, a respective first confidence score representing a respective first likelihood that a respective sub-portion depicts the at least one character.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
utilizing a language model to determine, for each sub-portion within the at least one linear portion, a respective second confidence score representing a respective second likelihood that the respective sub-portion depicts the at least one character.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, further comprising:
combining the respective first confidence score and the respective second confidence score to produce a respective overall confidence score for the respective sub-portion;
determining that the respective overall confidence score at least meets a specified threshold value; and
recognizing the respective sub-portion as being associated with the at least one character.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, wherein the language model is associated with a defined plurality of character sequences.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein at least one location, within the image, of the at least one character is determined based on one or more coordinates, within the image, of the respective sub-portion.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein identifying the set of pixels includes analyzing, for each pixel in the image, a respective image patch larger than and including a respective pixel, wherein the respective image patch is analyzed to determine whether the respective pixel forms at least part of the at least one character depicted in the image, and wherein the respective pixel is included in the set of pixels when the respective pixel is determined to form at least part of the at least one character.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein identifying the at least one linear portion is based on identifying at least one of a top portion above the at least one linear portion or a bottom portion below the at least one linear portion.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein identifying the at least one linear portion utilizes a text heat-map.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein determining, for each sub-portion within the at least one linear portion, the respective first confidence score is based on analyzing at least one of a left sub-portion adjacent to the respective sub-portion or a right sub-portion adjacent to the respective sub-portion.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein determining, for each sub-portion within the at least one linear portion, the respective first confidence score utilizes a character heat-map.

### Claim 11

A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
acquiring an image that depicts at least one character;
identifying a set of pixels, within the image, through which the at least one character is depicted;
identifying at least one linear portion, within the image, based on the set of pixels; and
determining, for each sub-portion within the at least one linear portion, a respective first confidence score representing a respective first likelihood that a respective sub-portion depicts the at least one character.

### Claim 12 (depends on 11)

The system of claim 11, wherein the instructions cause the system to further perform:
utilizing a language model to determine, for each sub-portion within the at least one linear portion, a respective second confidence score representing a respective second likelihood that the respective sub-portion depicts the at least one character.

### Claim 13 (depends on 12)

The system of claim 12, wherein the instructions cause the system to further perform:
combining the respective first confidence score and the respective second confidence score to produce a respective overall confidence score for the respective sub-portion;
determining that the respective overall confidence score at least meets a specified threshold value; and
recognizing the respective sub-portion as being associated with the at least one character.

### Claim 14 (depends on 11)

The system of claim 11, wherein identifying the at least one linear portion utilizes a text heat-map.

### Claim 15 (depends on 11)

The system of claim 11, wherein determining, for each sub-portion within the at least one linear portion, the respective first confidence score utilizes a character heat-map.

### Claim 16

A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
acquiring an image that depicts at least one character;
identifying a set of pixels, within the image, through which the at least one character is depicted;
identifying at least one linear portion, within the image, based on the set of pixels; and
determining, for each sub-portion within the at least one linear portion, a respective first confidence score representing a respective first likelihood that a respective sub-portion depicts the at least one character.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein the instructions cause the system to further perform:
utilizing a language model to determine, for each sub-portion within the at least one linear portion, a respective second confidence score representing a respective second likelihood that the respective sub-portion depicts the at least one character.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the instructions cause the system to further perform:
combining the respective first confidence score and the respective second confidence score to produce a respective overall confidence score for the respective sub-portion;
determining that the respective overall confidence score at least meets a specified threshold value; and
recognizing the respective sub-portion as being associated with the at least one character.

### Claim 19 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein identifying the at least one linear portion utilizes a text heat-map.

### Claim 20 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein determining, for each sub-portion within the at least one linear portion, the respective first confidence score utilizes a character heat-map.
