# US 12,182,703 B2 · Methods and apparatus to detect a text region of interest in a digital image using machine-based analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,703 B2
- Document ID: US12182703B2
- Status: Granted
- Kind: B2
- Application number: 17/598,792
- Filed: 2019-03-28
- Published: 2022-06-16
- Granted: 2024-12-31
- Earliest claimed: 2019-03-28
- Assignees: Nielsen Consumer LLC
- Applicants: Nielsen Consumer LLC
- Inventors: Roberto Arroyo; Javier Tovar Velasco; Francisco Javier Delgado Del Hoyo; Diego González Serrador; Emilio Almazán; Antonio Hurtado
- Web page: https://patentgrep.com/patent/US12182703B2

## CPC

- G06N 3/08: Learning methods
- G06F 18/24133: {Distances to prototypes}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/82: using neural networks
- G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images
- G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
- G06V 30/413: Classification of content, e.g. text, photographs or tables
- G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
- G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
- G06V 2201/10: Recognition assisted with metadata
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Methods, apparatus, systems and articles of manufacture are disclosed to analyze characteristics of text of interest using a computing system. An example apparatus includes a text detector to provide text data from a first image, the first image including a first text region of interest and a second text region not of interest, a color-coding generator to generate a plurality of color-coded text-map images, the plurality of color-coded text-map images including color-coded segments with different colors, the color-coded segments corresponding to different text characteristics, and a convolutional neural network (CNN) to determine a first location in the first image as more likely to be the first text region of interest than a second location in the first image corresponding to the second text region that is not of interest based on performing a CNN analysis on the first image and the plurality of color-coded text-map images.

## Claims

### Claim 1

An apparatus to analyze characteristics of text of interest, the apparatus comprising:
text detector circuitry to provide text data from a first image, the first image including a first text region of interest and a second text region not of interest;
color-coding generator circuitry to generate a plurality of color-coded text-map images, the color-coded text-map images including color-coded segments with different colors, the color-coded segments corresponding to different text characteristics; and
a convolutional neural network (CNN) to:
analyze the first image and the color-coded text-map images to detect visual features;
determine a first likelihood that a first location in the first image corresponds to the first text region of interest based on the CNN analysis;
determine a second likelihood that a second location in the first image corresponds to the second text region not of interest based on the CNN analysis;
classify at least one of the first location to be in the first text region based on the first likelihood or the second location to be in the second text region based on the second likelihood; and
adjust a characteristic of the first image based on the classification.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the color-coded text-map images include a first color-coded text-map image and a second color-coded text-map image, the first color-coded text-map image including first color-coded segments of a first color, and the second color-coded text-map image including second color-coded segments of a second color.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the first color-coded segments correspond to a first text characteristic, and the second color-coded segments correspond to a second text characteristic.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the first color is different than the second color.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the CNN analysis identifies the second text region that is not of interest as separate from the first text region of interest when a same keyword appears in both the first text region of interest and the second text region that is not of interest.

### Claim 6 (depends on 1)

The apparatus of claim 1, further including an interface to provide the color-coded text-map images to the CNN via a plurality of corresponding input channels of the CNN.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the first image is at least one of a food product label, a non-food product label, a sales receipt, a webpage, or a ticket.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the first text region of interest includes at least one of a nutrition facts table, a list of ingredients, a product description, candidate persons, numerical dates, or percentages.

### Claim 9

At least one non-transitory computer readable medium comprising computer readable instructions to cause at least one processor circuit to at least:
generate text data from a first image, the first image including a first text region of interest and a second text region not of interest;
generate a plurality of color-coded text-map images, the color-coded text-map images including color-coded segments with different colors, the color-coded segments corresponding to different text characteristics;
analyze, via a convolutional neural network (CNN), the first image and the color-coded text-map images to detect visual features:
determine a first likelihood that a first location in the first image corresponds to the first text region of interest based on the CNN analysis;
determine a second likelihood that a second location in the first image corresponds to the second text region not of interest based on the CNN analysis;
classify at least one of the first location to be in the first text region based on the first likelihood or the second location to be in the second text region based on the second likelihood; and
adjust a characteristic of the first image based on the classification.

### Claim 10 (depends on 9)

The at least one non-transitory computer readable medium of claim 9, wherein the color-coded text-map images include a first color-coded text-map image and a second color-coded text-map image, the first color-coded text-map image including first color-coded segments of a first color, and the second color-coded text-map image including second color-coded segments of a second color.

### Claim 11 (depends on 10)

The at least one non-transitory computer readable medium of claim 10, wherein the first color-coded segments correspond to a first text characteristic, and the second color-coded segments correspond to a second text characteristic.

### Claim 12 (depends on 11)

The at least one non-transitory computer readable medium of claim 11, wherein the first color is different than the second color.

### Claim 13 (depends on 9)

The at least one non-transitory computer readable medium of claim 9, wherein the computer readable instructions are to cause one or more of the at least one processor circuit to identify the second text region that is not of interest as separate from the first text region of interest when a same keyword appears in both the first text region of interest and the second text region that is not of interest.

### Claim 14 (depends on 9)

The at least one non-transitory computer readable medium of claim 9, wherein the computer readable instructions are to cause one or more of the at least one processor circuit to provide the plurality of color-coded text-map images to the CNN via a plurality of corresponding input channels of the CNN.

### Claim 15 (depends on 9)

The at least one non-transitory computer readable medium of claim 9, wherein the first image is at least one of a food product label, a non-food product label, a sales receipt, a webpage, or a ticket.

### Claim 16 (depends on 9)

The at least one non-transitory computer readable medium of claim 9, wherein the first text region of interest includes at least one of a nutrition facts table, a list of ingredients, a product description, candidate persons, numerical dates, or percentages.

### Claim 17

A method to analyze characteristics of text of interest, the method comprising:
generating text data from a first image, the first image including a first text region of interest and a second text region not of interest;
generating a plurality of color-coded text-map images, the color-coded text-map images including color-coded segments with different colors, the color-coded segments corresponding to different text characteristics;
analyzing, via a convolutional neural network (CNN), the first image and the color-coded text map images to detect visual features;
determining a first likelihood that a first location in the first image to the first text region of interest based on the CNN analysis;
determining a second likelihood that a second location in the first image corresponds to the second text region not of interest based on the CNN analysis;
classify at least one of the first location to be in the first text region based on the first likelihood or the second location to be in the second text region based on the second likelihood; and
adjust a characteristic of the first image based on the classification.

### Claim 18 (depends on 17)

The method of claim 17, wherein the color-coded text-map images include a first color-coded text-map image and a second color-coded text-map image, the first color-coded text-map image including first color-coded segments of a first color, and the second color-coded text-map image including second color-coded segments of a second color.

### Claim 19 (depends on 18)

The method of claim 18, wherein the first color-coded segments correspond to a first text characteristic, and the second color-coded segments correspond to a second text characteristic.

### Claim 20 (depends on 19)

The method of claim 19, wherein the first color is different than the second color.