# US 2022/0092848 A1 · Machine learning based image attribute determination

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0092848 A1
- Document ID: US20220092848A1
- Status: Published application
- Kind: A1
- Application number: 17/540,173
- Filed: 2021-12-01
- Published: 2022-03-24
- Earliest claimed: 2016-03-25
- Assignees: Outward, Inc.
- Applicants: Outward, Inc.
- Inventors: Clarence Chui; Manu Parmar
- Web page: https://patentgrep.com/patent/US20220092848A1

## CPC

- G06F 16/78: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06T 15/205: {Image-based rendering}
- G06F 16/58: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/32: using correlation-based methods
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Techniques for machine learning based image attribute determination are disclosed. In some embodiments, one or more unknown attributes of a received image comprising a prescribed environment are determined using a machine learning based framework. The machine learning based framework is at least in part trained on image data sets comprising a model environment that substantially simulates the prescribed environment.

## Claims

### Claim 1

A method, comprising:
receiving an image for which one or more attributes are unknown, wherein the image comprises a prescribed environment; and
determining one or more unknown attributes for the received image using a machine learning based framework, wherein the machine learning based framework is at least in part trained on image data sets comprising a model environment that substantially simulates the prescribed environment.

### Claim 2 (depends on 1)

The method of claim 1, wherein the received image comprises a rendering and the prescribed environment comprises the model environment.

### Claim 3 (depends on 1)

The method of claim 1, wherein the received image comprises a photograph and the prescribed environment comprises an actual physical environment.

### Claim 4 (depends on 1)

The method of claim 1, wherein the one or more unknown attributes of the received image determined by the machine learning based framework comprise pixel location in three-dimensional space.

### Claim 5 (depends on 1)

The method of claim 1, wherein the one or more unknown attributes of the received image determined by the machine learning based framework comprise pixel xyz coordinates.

### Claim 6 (depends on 1)

The method of claim 1, wherein the one or more unknown attributes of the received image determined by the machine learning based framework comprise pixel depth.

### Claim 7 (depends on 1)

The method of claim 1, wherein the one or more unknown attributes of the received image determined by the machine learning based framework comprise pixel surface normal vectors.

### Claim 8 (depends on 1)

The method of claim 1, wherein the prescribed environment is constrained and known.

### Claim 9 (depends on 1)

The method of claim 1, wherein known information about the prescribed environment comprises known structure and geometry of the prescribed environment.

### Claim 10 (depends on 1)

The method of claim 1, wherein the prescribed environment comprises an apparatus or rig for photographing objects or items.

### Claim 11 (depends on 1)

The method of claim 1, wherein known information about the prescribed environment comprises known camera information.

### Claim 12 (depends on 1)

The method of claim 1, wherein known information about the prescribed environment comprises known lighting information.

### Claim 13 (depends on 1)

The method of claim 1, wherein the training image data sets comprising the model environment are rendered from available three-dimensional models of corresponding assets.

### Claim 14 (depends on 1)

The method of claim 1, wherein the training image data sets comprising the model environment are labeled or otherwise associated with relevant metadata.

### Claim 15 (depends on 1)

The method of claim 1, wherein various attributes learned by the machine learning based framework from the training image data sets are derived or inferred from labels or metadata associated with the training image data sets.

### Claim 16 (depends on 1)

The method of claim 1, wherein the machine learning based framework comprises one or more neural networks.

### Claim 17 (depends on 1)

The method of claim 1, wherein the machine learning based framework comprises one or more convolutional neural networks.

### Claim 18 (depends on 1)

The method of claim 1, wherein the received image comprises a still image or a frame of a video sequence.

### Claim 19

A system, comprising:
a processor configured to:
receive an image for which one or more attributes are unknown, wherein the image comprises a prescribed environment; and
use a machine learning based framework to determine one or more unknown attributes for the received image, wherein the machine learning based framework is at least in part trained on image data sets comprising a model environment that substantially simulates the prescribed environment; and
a memory coupled to the processor and configured to provide the processor with instructions.

### Claim 20

A computer program product, the computer program product being embodied in a non-transitory computer readable medium and comprising computer instructions for:
receiving an image for which one or more attributes are unknown, wherein the image comprises a prescribed environment; and
determining one or more unknown attributes for the received image using a machine learning based framework, wherein the machine learning based framework is at least in part trained on image data sets comprising a model environment that substantially simulates the prescribed environment.
