# US 2021/0279516 A1 · Ground truth generation for image segmentation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279516 A1
- Document ID: US20210279516A1
- Status: Published application
- Kind: A1
- Application number: 16/811,473
- Filed: 2020-03-06
- Published: 2021-09-09
- Earliest claimed: 2020-03-06
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Peter Willem Jan Staar; Michele Dolfi; Christoph Auer; Leonidas Georgopoulos; Konstantinos Bekas
- Web page: https://patentgrep.com/patent/US20210279516A1

## CPC

- G06V 30/19147: {Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06F 18/24: Classification techniques
- G06V 20/64: Three-dimensional [3D] objects
- G06V 30/19173: {Classification techniques}
- G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method, system and computer program product to generate a training data set for image segmentation applications, comprising providing a set of input documents of a first format. The input documents each comprise one or more pages. The input documents are split into individual document pages and parsed. Parsing comprises identifying a predefined set of items including position information of the position of the predefined set of items in the individual document pages; generating a bitmap image of a second format for each individual document page of the first format. The bitmap image comprises a predefined number of pixels. A mask is generated for each individual document. The mask comprises the predefined number of pixels of the corresponding bitmap image. Generating the mask comprises assigning an encoded class label to each pixel of the mask based on the position information of identified items of the predefined set of items.

## Claims

### Claim 1

A computer-implemented method comprising:
providing a set of input documents of a first format, each input document comprising one or more document pages;
splitting the input documents into individual document pages of the first format;
parsing the individual document pages of the first format, the parsing comprising identifying a predefined set of items including position information of the position of the predefined set of items in the individual document pages;
generating a bitmap image of a second format for each individual document page of the first format, the bitmap image comprising a predefined number of pixels;
generating a mask for each individual document, the mask comprising the predefined number of pixels of the corresponding bitmap image; and
the generating of the mask comprises assigning an encoded class label to each pixel of the mask based on the position information of identified items of the predefined set of items.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the assigning of the class label comprises evaluating whether the respective pixel of the mask overlaps with an identified item of the predefined set of items or not.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the mask is a binary mask.

### Claim 4 (depends on 1)

The method according to claim 1, wherein
the predefined set of items comprises text cells; and
a position information of the text cells include bounding box coordinates of bounding boxes of the text cells.

### Claim 5 (depends on 1)

The method according to claim 1, wherein
the predefined set of items comprises paths; and
the position information comprises a path coordinates and a width of the path.

### Claim 6 (depends on 1)

The method according to claim 1, wherein
the mask is a 3-dimensional mask with dimensions number of pixels in height, number of pixels in width and number of class labels.

### Claim 7 (depends on 1)

The method according to claim 1, wherein
the bitmap images are 3-dimensional images with dimensions number of pixels in height, number of pixels in width and number of color channels.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the generating of the mask comprises:
initializing the encoded class label of all pixels of the mask to a first value; and
assigning a second value as encoded class label to all pixels of the mask that overlap with an identified item of the predefined set of items according to the position information of the predefined set of items.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising:
providing the bitmap image and the corresponding mask as tuple to a training data set for an image segmentation application.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the training data set comprises:
a plurality of individual document pages of the first format;
a bitmap image of the second format for each of the plurality of individual document pages of the first format, the bitmap comprising a predefined number of pixels;
a mask for each of the plurality of individual document pages of the first format, the mask comprising the predefined number of pixels of the corresponding bitmap image; and
an encoded class label assigned to each pixel of the mask based on position information of identified items of the predefined set of items.

### Claim 11 (depends on 1)

The method according to claim 1, further comprising:
adding a masked signature to the bitmap image.

### Claim 12 (depends on 1)

The method according to claim 1, further comprising:
adding a masked handwritten text to the bitmap image.

### Claim 13 (depends on 1)

The method according to claim 1, further comprising:
adding affine transformations to the bitmap image and the corresponding mask in order to simulate scanning distortions.

### Claim 14 (depends on 1)

The method according to claim 1, further comprising:
adding noise to the bitmap image.

### Claim 15 (depends on 1)

The method according to claim 1, wherein the first format is a programmatic data format.

### Claim 16 (depends on 1)

The method according to claim 1, wherein the first format is a format according to a Portable Document Format-standard, according an ISO 32000-X standard of an International Standard Association.

### Claim 17 (depends on 1)

The method according to claim 1, wherein the second format is a format selected from the group consisting of:
Portable Networks Graphic format;
Joint Photographics Expert Group format; and
Tagged Image File Format format.

### Claim 18 (depends on 1)

The method according to claim 1, further comprising:
gathering the predefined set of items of each of the one or more individual pages of the plurality of documents into individual page files of a third format; and
the assigning of the encoded class label uses the individual pages files of the third format.

### Claim 19 (depends on 18)

The method according to claim 18, wherein the third format is selected from the group consisting of:
Java Script Object Notification Data Interchange Format;
Hypertext Markup Language; and
YAML Ain't Markup Language.

### Claim 20 (depends on 1)

The method according to claim 1, wherein the mask has a predefined fourth format.

### Claim 21

A system comprising one or more processors for executing computer-readable instructions, the computer-readable instructions controlling the one or more processors to perform operations comprising:
providing a set of input documents of a first format, each of the set of input documents comprising one or more document pages;
splitting the input documents into its individual document pages;
parsing the individual document pages of the first format, the parsing comprising identifying a predefined set of items including position information of the position of the predefined set of items in the individual document pages;
generating a bitmap image of a second format for each individual document page of the first format, the bitmap image comprising a predefined number of pixels;
generating a mask for each individual document page, the mask comprising the predefined number of pixels of the corresponding bitmap image; and
assigning an encoded class label to each pixel of the mask based on the position information of identified items of the predefined set of items.

### Claim 22

A computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:
providing a set of input documents of a first format, each of the input documents comprising one or more document pages;
splitting the input documents into individual document pages;
parsing the individual document pages of the first format, the parsing comprising identifying a predefined set of items including position information of the position of the predefined set of items in the individual document pages;
generating a bitmap image of a second format for each individual document page of the first format, the bitmap image comprising a predefined number of pixels;
generating a mask for each individual document page, the mask comprising the predefined number of pixels of the corresponding bitmap image; and
assigning an encoded class label to each pixel of the mask based on the position information of identified items of the predefined set of items.

### Claim 23

A method comprising:
receiving a training data set, the training data set comprising:
a plurality of individual document pages of a first format;
a bitmap image of a second format for each of the plurality of individual document pages of the first format, the bitmap comprising a predefined number of pixels;
a mask for each of the plurality of individual document pages of the first format, the mask comprising the predefined number of pixels of the corresponding bitmap image; and
an encoded class label assigned to each pixel of the mask based on position information of identified items of the predefined set of items; and
training a cognitive model of an image segmentation application with the training data set.

### Claim 24 (depends on 23)

The method of claim 23, comprising:
performing image segmentation, comprising:
receiving a training data set, the training data set comprising:
a plurality of individual document pages of a first format;
a bitmap image of a second format for each of the plurality of individual document pages of the first format, the bitmap comprising a predefined number of pixels;
a mask for each of the plurality of individual document pages of the first format, the mask comprising the predefined number of pixels of the corresponding bitmap image; and
an encoded class label assigned to each pixel of the mask based on position information of identified items of the predefined set of items; and
training a cognitive model of an image segmentation application with the training data set.

### Claim 25

A cognitive model, the cognitive model being encoded on a non-transitory computer readable medium, the cognitive model comprising:
a training data set, the training data set comprising:
a plurality of individual document pages of a first format;
a bitmap image of a second format for each of the plurality of individual document pages of the first format, the bitmap comprising a predefined number of pixels;
a mask for each of the plurality of individual document pages of the first format, the mask comprising the predefined number of pixels of the corresponding bitmap image; and
an encoded class label assigned to each pixel of the mask based on position information of identified items of the predefined set of items.
