# US 2024/0005690 A1 · Generating article polygons within newspaper images for extracting actionable data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005690 A1
- Document ID: US20240005690A1
- Status: Published application
- Kind: A1
- Application number: 18/345,952
- Filed: 2023-06-30
- Published: 2024-01-04
- Earliest claimed: 2022-07-01
- Assignees: Ancestry.com Operations Inc.
- Applicants: Ancestry.com Operations Inc.
- Inventors: Michael Benjamin Brodie; Suraj Subraveti; Maria Antonia Fabiano
- Web page: https://patentgrep.com/patent/US20240005690A1

## CPC

- G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
- G06V 10/82: using neural networks
- G06V 30/10: Character recognition
- G06V 30/1444: {Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields}
- G06V 30/18: Extraction of features or characteristics of the image
- G06V 30/413: Classification of content, e.g. text, photographs or tables
- G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure is directed toward systems, methods, and non-transitory computer-readable media for generating and providing actionable data from newspaper articles identified and segmented from digital newspaper images. For example, the disclosed systems segment articles of a newspaper image by using specially designed models to generate polygons defining article boundaries within the newspaper image. In some cases, the disclosed systems further determine article text from a polygon of an article for additional processing to determine an article topic, determine an article type, predict entity names within the article, and/or predict a locality associated with the article.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving a newspaper image depicting digitized newspaper content;
detecting an article within the digitized newspaper content of the newspaper image utilizing an article prediction model;
generating, utilizing the article prediction model, a polygon defining boundaries of the article within the newspaper image; and
determining text within the polygon utilizing an optical character recognition model.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
extracting text features from the text within the polygon utilizing a domain-adapted information extraction model; and
predicting, from the text features, entity names within the text of the polygon.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising generating, using an article locality model, a locality prediction for the article based on the text within the polygon.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein detecting the article within the newspaper image comprises:
determining text columns within the newspaper image using a column prediction model; and
aligning the article within one or more of the text columns.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein generating the polygon comprises generating an irregularly shaped polygon enclosing portions of multiple text columns within the newspaper image based on the article spanning the multiple text columns.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, further comprising:
extracting visual features from pixels of the newspaper image enclosed by the polygon; and
classifying the article into an article topic based on the visual features.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein generating the polygon defining the boundaries of the article comprises utilizing a column prediction model repurposed from an architectural building detection model.

### Claim 8

A non-transitory computer readable medium storing instructions which, when executed by at least one processor, cause the at least one processor to:
receive a newspaper image depicting digitized newspaper content;
detect an article within the digitized newspaper content of the newspaper image utilizing an article prediction model;
generate, utilizing the article prediction model, a polygon defining boundaries of the article within the newspaper image; and
determine text within the polygon utilizing an optical character recognition model.

### Claim 9 (depends on 8)

The non-transitory computer readable medium of claim 8, further storing instructions which, when executed by the at least one processor, cause the at least one processor to:
extract text features from the text within the polygon utilizing an information extraction model adapted to a newspaper domain from a free text domain; and
predict, from the text features, name parts for entity names within the text of the polygon.

### Claim 10 (depends on 8)

The non-transitory computer readable medium of claim 8, further storing instructions which, when executed by the at least one processor, cause the at least one processor to generate, using an article locality model based on the text within the polygon, a locality prediction indicating whether the article is a local article or a non-local article.

### Claim 11 (depends on 8)

The non-transitory computer readable medium of claim 8, further storing instructions which, when executed by the at least one processor, cause the at least one processor to detect the article within the newspaper image by:
generating, utilizing a column prediction model, a plurality of column predictions for the newspaper image;
determining, from the plurality of column predictions, text columns within the newspaper image according to densities of column predictions at respective locations; and
aligning the article within one or more of the text columns.

### Claim 12 (depends on 8)

The non-transitory computer readable medium of claim 8, further storing instructions which, when executed by the at least one processor, cause the at least one processor to generate the polygon by generating a rectangular polygon enclosing one or more text columns within the newspaper image including text of the article.

### Claim 13 (depends on 8)

The non-transitory computer readable medium of claim 8, further storing instructions which, when executed by the at least one processor, cause the at least one processor to:
extract visual features from pixels of the newspaper image enclosed by the polygon; and
classify the article into an article type based on the visual features.

### Claim 14 (depends on 8)

The non-transitory computer readable medium of claim 8, further storing instructions which, when executed by the at least one processor, cause the at least one processor to generate the polygon defining the boundaries of the article by utilizing a column prediction model repurposed from an architectural building detection model.

### Claim 15

A system comprising:
at least one processor; and
a non-transitory computer readable medium storing instructions which, when executed by the at least one processor, cause the system to:
receive a newspaper image depicting digitized newspaper content;
detect an article within the digitized newspaper content of the newspaper image utilizing an article prediction model;
generate, utilizing the article prediction model, a polygon defining boundaries of the article within the newspaper image; and
determine text within the polygon utilizing an optical character recognition model.

### Claim 16 (depends on 15)

The system of claim 15, further storing instructions which, when executed by the at least one processor, cause the system to:
extract text features from the text within the polygon utilizing an information extraction model adapted to a newspaper domain from a free text domain; and
predict, from the text features, name parts for entity names within the text of the polygon.

### Claim 17 (depends on 15)

The system of claim 15, further storing instructions which, when executed by the at least one processor, cause the system to generate, using an article locality model based on the text within the polygon, a locality prediction indicating whether the article is a local article or a non-local article.

### Claim 18 (depends on 15)

The system of claim 15, further storing instructions which, when executed by the at least one processor, cause the system to detect the article within the newspaper image by:
generating, utilizing a column prediction model, a plurality of column predictions for the newspaper image;
determining, from the plurality of column predictions, text columns within the newspaper image according to densities of column predictions at respective locations; and
aligning the article within one or more of the text columns.

### Claim 19 (depends on 15)

The system of claim 15, further storing instructions which, when executed by the at least one processor, cause the system to generate the polygon by generating a rectangular polygon enclosing one or more text columns within the newspaper image including text of the article.

### Claim 20 (depends on 15)

The system of claim 15, further storing instructions which, when executed by the at least one processor, cause the system to:
extract visual features from pixels of the newspaper image enclosed by the polygon; and
classify the article into an article topic based on the visual features.
