# US 2022/0070369 A1 · Camera Image Or Video Processing Pipelines With Neural Embedding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0070369 A1
- Document ID: US20220070369A1
- Status: Published application
- Kind: A1
- Application number: 17/458,985
- Filed: 2021-08-27
- Published: 2022-03-03
- Earliest claimed: 2020-08-28
- Assignees: Spectrum Optix Inc.
- Applicants: Spectrum Optix Inc.
- Inventors: Kevin Gordon; Martin Humphreys; Colin D'Amore
- Web page: https://patentgrep.com/patent/US20220070369A1

## CPC

- G06T 1/00: General purpose image data processing
- G06T 3/4046: using neural networks
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- H04N 23/617: Upgrading or updating of programs or applications for camera control
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- H04N 23/80: Camera processing pipelines; Components thereof
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/048: Activation functions
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing pipeline including a still or video camera includes a first portion of an image processing system arranged to use information derived at least in part from a neural embedding. A second portion of the image processing system can be used to modify at least one of an image capture setting, sensor processing, global post processing, local post processing, and portfolio post processing, based at least in part on neural embedding information.

## Claims

### Claim 1

An image processing pipeline including a still or video camera, comprising:
a first portion of an image processing system arranged to use information derived at least in part from neural embedding information; and
a second portion of the image processing system used to modify at least one of an image capture setting, sensor processing, global post processing, local post processing, and portfolio post processing, based at least in part on the neural embedding information.

### Claim 2 (depends on 1)

The image processing pipeline of claim 1, wherein the neural embedding information includes a latent vector.

### Claim 3 (depends on 1)

The image processing pipeline of claim 1, wherein the neural embedding information includes at least one latent vector that is sent between modules in the image processing system.

### Claim 4 (depends on 1)

The image processing pipeline of claim 1, wherein the neural embedding includes at least one latent vector that is sent between one or more neural networks in the image processing system.

### Claim 5

An image processing pipeline including a still or video camera, comprising:
a first portion of an image processing system arranged to reduce data dimensionality and effectively downsample an image, images, or other data using a neural processing system to create neural embedding information; and
a second portion of the image processing system arranged to modify at least one of an image capture setting, sensor processing, global post processing, local post processing, and portfolio post processing, based at least in part on the neural embedding information.

### Claim 6 (depends on 5)

The image processing pipeline of claim 5, wherein the neural embedding information includes a latent vector.

### Claim 7 (depends on 5)

The image processing pipeline of claim 5, wherein the neural embedding information includes at least one latent vector that is sent between modules in the image processing system.

### Claim 8 (depends on 5)

The image processing pipeline of claim 5, wherein the neural embedding includes at least one latent vector that is sent between one or more neural networks in the image processing system.

### Claim 9

An image processing pipeline including a still or video camera, comprising:
a first portion of an image processing system arranged for at least one of categorization, tracking, and matching using neural embedding information derived from a neural processing system; and;
a second portion of the image processing system arranged to modify at least one of an image capture setting, sensor processing, global post processing, local post processing, and portfolio post processing, based at least in part on the neural embedding information.

### Claim 10 (depends on 9)

The image processing pipeline of claim 9, wherein the neural embedding information includes a latent vector.

### Claim 11 (depends on 9)

The image processing pipeline of claim 9, wherein the neural embedding information includes at least one latent vector that is sent between modules in the image processing system.

### Claim 12 (depends on 9)

The image processing pipeline of claim 9, wherein the neural embedding includes at least one latent vector that is sent between one or more neural networks in the image processing system.

### Claim 13

An image processing pipeline including a still or video camera, comprising:
a first portion of an image processing system arranged to reduce data dimensionality and effectively downsample an image, images, or other data using a neural processing system to provide neural embedding information; and
a second portion of the image processing system arranged to preserve the neural embedding information within image or video metadata.

### Claim 14 (depends on 13)

The image processing pipeline of claim 13, wherein the neural embedding information includes a latent vector.

### Claim 15 (depends on 13)

The image processing pipeline of claim 13, wherein the neural embedding information includes at least one latent vector that is sent between modules in the image processing system.

### Claim 16 (depends on 13)

The image processing pipeline of claim 13, wherein the neural embedding includes at least one latent vector that is sent between one or more neural networks in the image processing system.

### Claim 17

An image processing pipeline including a still or video camera, comprising:
a first portion of an image processing system arranged to reduce data dimensionality and effectively downsample an image, images, or other data using a neural processing system to provide neural embedding information; and
a second portion of the image processing system arranged for at least one of categorization, tracking, and matching using neural embedding information derived from the neural processing system.

### Claim 18 (depends on 17)

The image processing pipeline of claim 17, wherein the neural embedding information includes a latent vector.

### Claim 19 (depends on 17)

The image processing pipeline of claim 17, wherein the neural embedding information includes at least one latent vector that is sent between modules in the image processing system.

### Claim 20 (depends on 17)

The image processing pipeline of claim 17, wherein the neural embedding includes at least one latent vector that is sent between one or more neural networks in the image processing system.

### Claim 21

A neural network training system, comprising:
a first portion having a neural network algorithm arranged to reduce data dimensionality and effectively downsample an image, images, or other data using a neural processing system to provide neural embedding information;
a second portion having a neural network algorithm arranged for at least one of categorization, tracking, and matching using neural embedding information derived from a neural processing system; and
a training procedure that optimizes operation of the first and second portions of the neural network algorithm.
