# US 2019/0377980 A1 · Generating training data for machine learning classifier

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377980 A1
- Document ID: US20190377980A1
- Status: Published application
- Kind: A1
- Application number: 16/002,728
- Filed: 2018-06-07
- Published: 2019-12-12
- Earliest claimed: 2018-06-07
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Trebor Lee CONNELL; Emanuel SHALEV; Michael John EBSTYNE; Don Dongwoo KIM
- Web page: https://patentgrep.com/patent/US20190377980A1

## CPC

- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 20/00: Machine learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Data representing a scene is received. The scene includes labeled elements such as walls, a floor, a ceiling, and objects placed at various locations in the scene. The original received scene may be modified in different ways to create new scenes that are based on the original scene. These modifications include adding clutter to the scene, moving one or more elements of the scene, swapping one or more elements of the scene with different labeled elements, changing the size, color, or materials associated with one or more of the elements of the scene, and changing the lighting used in the scene. Each new scene may be used to generate labeled training data for a classifier by placing a virtual sensor (e.g., a camera) in the new scene, and generating sensor output data for the virtual sensor based on its placement in the new scene.

## Claims

### Claim 1

A system for generating training data for machine learning classifiers, the system comprising:
at least one computing device; and
a training engine that:
receives a first scene, wherein the first scene includes a plurality of elements, and each element is associated with a label;
generates a second scene from the first scene by performing one or more of:
changing a lighting characteristic of the first scene;
adding one more elements to the plurality of elements of the first scene; and
changing a characteristic of one or more elements of the plurality of elements of the first scene;
generates training data from the second scene; and
trains a machine learning classifier using the generated training data.

### Claim 2 (depends on 1)

The system of claim 1, wherein changing a lighting characteristic of the first scene comprises one or more of changing a temperature, a location, a shape, and an intensity of a light source associated with the first scene, removing a light source from the first scene, and adding a light source to the first scene.

### Claim 3 (depends on 1)

The system of claim 1, wherein changing a characteristic of one or more elements of the plurality of elements of the first scene comprises changing one or more of a size, a location, an orientation, a color, and a texture of one or more elements of the plurality of elements of the first scene.

### Claim 4 (depends on 1)

The system of claim 1, wherein adding one or more elements to the plurality of elements of the first scene comprises randomly adding the one or more elements to the plurality of elements of the first scene.

### Claim 5 (depends on 1)

The system of claim 1, wherein the training engine that generates the training data from the generated second scene comprises the training engine that:
places a virtual sensor in a location in the second scene;
generates sensor data for the virtual sensor based on the location of the virtual sensor in the second scene; and
generates the training data using the sensor data and the labels associated with the elements of the plurality of elements of the second scene.

### Claim 6 (depends on 5)

The system of claim 5, wherein the virtual sensor is a camera.

### Claim 7 (depends on 5)

The system of claim 5, wherein the training engine selects the location for the virtual sensor.

### Claim 8 (depends on 7)

The system of claim 7, wherein the training engine randomly selects the location or selects the location based on one or more elements of interest in the second scene.

### Claim 9

A system for generating training data for machine learning classifiers, the system comprising:
at least one computing device; and
a training engine that:
receives a scene, wherein the scene includes a plurality of elements, and each element is associated with a label;
changes a lighting characteristic of the scene;
changes a characteristic of one or more elements of the plurality of elements of the scene;
adds a new element to the plurality of elements of the scene, wherein the new element is associated with a label;
places a virtual sensor in a location of the scene;
generates sensor data for the virtual sensor based on the location of the virtual sensor in the scene; and
generates training data based on the generated sensor data and the labels associated with the elements of the plurality of elements of the scene.

### Claim 10 (depends on 9)

The system of claim 9, wherein the training engine further trains a machine learning classifier using the generated training data.

### Claim 11 (depends on 9)

The system of claim 9, wherein the training engine that changes the lighting characteristic of the scene comprises the training engine performing one or more of changing a temperature, a location, a shape, and an intensity of a light source associated with the scene, removing a light source from the scene, and adding a light source to the scene.

### Claim 12 (depends on 9)

The system of claim 9, wherein the training engine that changes a characteristic of one or more elements of the plurality of elements of the scene comprises the training engine that changes one or more of a size, a location, an orientation, a color, and a texture of one or more elements of the plurality of elements of the scene.

### Claim 13 (depends on 9)

The system of claim 9, wherein the training engine that adds a new element to the plurality of elements of the scene comprises the training engine that randomly adds a new element to the plurality of elements of the scene.

### Claim 14 (depends on 9)

The system of claim 9, wherein the virtual sensor is a camera.

### Claim 15

A method for generating training data for machine learning classifiers, the method comprising:
receiving a scene by a computing device, wherein the scene includes a plurality of elements, and each element is associated with a label;
changing a lighting characteristic of the scene by the computing device;
changing a characteristic of one or more elements of the plurality of elements of the scene by the computing device;
adding a new element to the plurality of elements of the scene by the computing device, wherein the new element is associated with a label;
generating training data based on the scene by the computing device; and
providing the generated training data.

### Claim 16 (depends on 15)

The method of claim 15, further comprising training a machine learning classifier using the generated training data.

### Claim 17 (depends on 15)

The method of claim 15, further comprising:
placing a virtual sensor in a location of the scene;
generating sensor data for the virtual sensor based on the location of the virtual sensor in the scene; and
generating the training data based on the generated sensor data and the labels associated with the elements of the plurality of elements.

### Claim 18 (depends on 15)

The method of claim 15, wherein changing the lighting characteristic of the scene comprises one or more of changing a temperature, a location, a shape, and an intensity of a light source associated with the scene, removing a light source from the scene, and adding a light source to the scene.

### Claim 19 (depends on 15)

The method of claim 15, wherein changing a characteristic of one or more elements of the plurality of elements of the scene comprises changing one or more of a size, a location, an orientation, a color, and a texture of one or more elements of the plurality of elements of the scene.

### Claim 20 (depends on 15)

The method of claim 15, wherein adding one or more elements to the plurality of elements comprises randomly adding the one or more elements to the plurality of elements.