# US 2019/0156487 A1 · Automated generation of pre-labeled training data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0156487 A1
- Document ID: US20190156487A1
- Status: Published application
- Kind: A1
- Application number: 16/258,037
- Filed: 2019-01-25
- Published: 2019-05-23
- Earliest claimed: 2017-01-04
- Assignees: Cisco Technology, Inc.
- Applicants: Cisco Technology, Inc.
- Inventors: Rob Liston; John G. Apostolopoulos
- Web page: https://patentgrep.com/patent/US20190156487A1

## CPC

- G06T 7/11: Region-based segmentation
- G06T 7/194: involving foreground-background segmentation
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10024: Color image
- G06T 2207/20081: Training; Learning
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Presented herein are techniques for automatically generating object segmentation training data. In particular, a segmentation data generation system is configured to obtain training images derived from a scene captured by one or more image capture devices. Each training image is a still image that includes a foreground object and a background. The segmentation data generation system automatically generates a mask of the training image to delineate the object from the background and, based on the mask automatically generates a masked image. The masked image includes only the object present in the training image. The segmentation data generation system composites the masked image with an image of an environmental scene to generate a composite image that includes the masked image and the environmental scene.

## Claims

### Claim 1

A method comprising:
obtaining a training image derived from a scene captured by one or more image capture devices, wherein the training image includes a foreground object and a background;
automatically generating a mask of the training image to delineate the foreground object from the background;
based on the mask of the training image, automatically generating a masked image that includes only the foreground object present in the training image;
compositing the masked image with an image of an environmental scene to generate a composite image that includes the masked image and the environmental scene; and
performing coherence matching of the masked image and the environmental scene by adjusting lighting characteristics of at least one of the masked image or the environmental scene.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating a region and label pair which represents the masked image in the composite image, wherein the composite image and the region and label pair form an object segmentation training data set; and
providing the object segmentation training data set to a machine learning-based image segmentation process.

### Claim 3 (depends on 1)

The method of claim 1, wherein compositing the masked image with the image of the environmental scene comprises:
performing one or more translations, rotations, scaling, or other perspective transformations to one or more of the masked image and the environmental scene.

### Claim 4 (depends on 1)

The method of claim 1, wherein adjusting the lighting characteristics causes the masked image and the environmental scene to have substantially the same amount of illumination.

### Claim 5 (depends on 1)

The method of claim 1, wherein adjusting the lighting characteristics causes the masked image and the environmental scene to include substantially the same spectrum of visible light.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
adjusting noise characteristics of at least one of the masked image or the environmental scene such that the masked image and the environmental scene include substantially the same amount of random noise.

### Claim 7 (depends on 1)

The method of claim 1, wherein automatically generating a mask of the training image to delineate the foreground object from the background comprises:
automatically generating a binary mask of the training image.

### Claim 8 (depends on 7)

The method of claim 7, wherein automatically generating the binary mask of the training image comprises:
using at least one of a chroma key approach or a depth camera approach to generate the binary mask.

### Claim 9

An apparatus comprising:
a memory;
a network interface unit; and
a processor configured to:
obtain a training image derived from a scene captured by one or more image capture devices, wherein the training image includes a foreground object and a background;
automatically generate a mask of the training image to delineate the foreground object from the background;
based on the mask of the training image, automatically generate a masked image that includes only the foreground object present in the training image;
composite the masked image with an image of an environmental scene to generate a composite image that includes the masked image and the environmental scene; and
perform coherence matching of the masked image and the environmental scene by adjusting lighting characteristics of at least one of the masked image or the environmental scene.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the processor is configured to:
generate a region and label pair which represents the masked image in the composite image, wherein the composite image and the region and label pair form an object segmentation training data set; and
provide the object segmentation training data set to a machine learning-based image segmentation process.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein to composite the masked image with the image of the environmental scene, the processor is configured to:
perform one or more translations, rotations, scaling, or other perspective transformations to one or more of the masked image and the environmental scene.

### Claim 12 (depends on 9)

The apparatus of claim 9, wherein to adjust the lighting characteristics of at least one of the masked image or the environmental scene, the processor is configured to:
cause the masked image and the environmental scene to have substantially the same amount of illumination.

### Claim 13 (depends on 9)

The apparatus of claim 9, wherein to adjust the lighting characteristics of at least one of the masked image or the environmental scene, the processor is configured to:
cause the masked image and the environmental scene to include substantially the same spectrum of visible light.

### Claim 14 (depends on 9)

The apparatus of claim 9, wherein the processor is further configured to:
adjust noise characteristics of at least one of the masked image or the environmental scene such that the masked image and the environmental scene include substantially the same amount of random noise.

### Claim 15

One or more non-transitory computer readable storage media encoded with instructions that, when executed by a processor, cause the processor to:
obtain a training image derived from a scene captured by one or more image capture devices, wherein the training image includes a foreground object and a background;
automatically generate a mask of the training image to delineate the foreground object from the background;
based on the mask of the training image, automatically generate a masked image that includes only the foreground object present in the training image;
composite the masked image with an image of an environmental scene to generate a composite image that includes the masked image and the environmental scene; and
perform coherence matching of the masked image and the environmental scene by adjusting lighting characteristics of at least one of the masked image or the environmental scene.

### Claim 16 (depends on 15)

The non-transitory computer readable storage media of claim 15, further comprising instructions operable to:
generate a region and label pair which represents the masked image in the composite image, wherein the composite image and the region and label pair form an object segmentation training data set; and
provide the object segmentation training data set to a machine learning-based image segmentation process.

### Claim 17 (depends on 15)

The non-transitory computer readable storage media of claim 15, wherein the instructions operable to composite the masked image with the image of the environmental scene comprise instructions operable to:
perform one or more translations, rotations, scaling, or other perspective transformations to one or more of the masked image and the environmental scene.

### Claim 18 (depends on 15)

The non-transitory computer readable storage media of claim 15, wherein the instructions operable to composite the masked image with the image of the environmental scene comprise instructions operable to:
perform coherence matching of the masked image and the environmental scene to normalize the masked image and the environmental scene.

### Claim 19 (depends on 15)

The non-transitory computer readable storage media of claim 15, wherein the instructions operable to perform coherence matching of the masked image and the environmental scene comprise instructions operable to:
adjust lighting characteristics of at least one of the masked image or the environmental scene such that the masked image and the environmental scene have substantially the same amount of illumination or substantially the same spectrum of visible light.

### Claim 20 (depends on 15)

The non-transitory computer readable storage media of claim 15, wherein the instructions operable to perform coherence matching of the masked image and the environmental scene comprise instructions operable to:
adjust noise characteristics of at least one of the masked image or the environmental scene such that the masked image and the environmental scene include substantially the same amount of random noise.
